All too often, you'll find that those who reject Christianity do not understand it. This is true not only for the faith as a whole, but also for individual doctrines. Sola Scriptura, for example, is a basic Christian doctrine that is often neglected, and all too often rejected, simply because its opponents do not, or will not, understand it.
The most common caricature of Sola Scriptura is that it is a doctrine held to the exclusion of any church at all. No leaders, no teachers, no authority what so ever, just each individual Christian as an island, so long as that island is the Bible. So naturally, the most common attempt to refute Sola Scriptura is to show where scripture itself emphasises the importance of the Church.
But the simple response to these arguments is never to downplay the importance of the Church, because Sola Scriptura would logically argue, since scripture does emphasise the importance of the Church, that the Church is important. What Sola Scriptura would deny is that the Church has any authority over scripture.
Every time this argument is brought against Sola Scriptura, the heretic is actually admitting this. By appealing to scripture to support their Church's claim to authority, they are admitting the scriptures are, indeed, a higher authority. Should they, at any given point, deny the authority of the scriptures, they lose their argument for their own legitimacy.
By contrast, scripture presents itself as the very word of God. Those who deny this, once again, lose their own argument for their own legitimacy (not to mention, at least sometimes, are inconsistent with the official teachings of their own Church). Even if their Church explicitly denies the authority of scripture, this weakens them significantly; if God has not spoken, what gives them authority to speak for Him? A man can no more understand a distant God than a dog on the street can understand the depths of a monarchy.
But if you believe, correctly, that "All Scripture is given by inspiration of God, and is profitable for doctrine, for reproof, for correction, for instruction in righteousness, that the man of God may be complete, thoroughly equipped for every good work" (2 Timothy 3:16-17), then you have absolutely no excuse for denying Sola Scriptura, particularly once you study that collection of God-inspired documents and see just how firmly Christ Himself affirmed the scriptures, not even allowing the Pharisees, whose authority He affirmed (Matthew 23:1-3), to alter, deny, disconfirm, disobey, break, violate, or otherwise dissent from scripture.
Furthermore, we notice that His approved messengers affirmed scripture, both Old and New, and did not even distinguish between Old and New. Furthermore, they considered themselves stewards of the mysteries of God (1 Corinthians 4:1), but did not consider themselves to have authority above the scriptures. James tells us that teachers shall receive a stricter judgement (James 3:1). Luke tells us that Paul and Silas allowed the Bereans to test their message against scripture to see if what they were saying was true (Acts 17:10-12). Paul says to the Galatians that even if the Apostles, or an angel from Heaven, preached a different gospel, that individual was to be considered accursed (Galatians 1:8).
It is beyond clear, therefore, that Sola Scriptura is a fact, simply because it is the only doctrine that grants God His true authority. When God speaks, the Church must listen. Does this not just make sense? Is there any Church with the audacity to flat out claim "God is wrong, we have authority over Him"? It wouldn't surprise me, especially given that I have heard individuals say it, but it's astonishing to me that a fallible human being, who cannot live without food and drink, would so elevate himself above the bread of life, who gives us living water. Unless you share in such astonishing arrogance, you believe in Sola Scriptura.