"Professing to be wise, they became fools, and changed the glory of the incorruptible God into an image made like corruptible man—and birds and four-footed animals and creeping things." - Romans 1:22-23
Although this passage refers to humanity in general, it is put on full display when it comes to atheists. In general, atheists act as if they have the absolute moral and intellectual high ground. "We are all born atheists, until someone starts telling us lies", as one meme by Atheist Republic puts it. Atheism, they claim, is the default position of humanity, and if they weren't taught religious claims, no one would ever become religious. Religion requires indoctrination, and thus, by extension, religious people are indoctrinated.
There are, however, a myriad of problems with this argument, starting with the simple, and blindingly obvious fact that the default position of humanity is complete and utter childishness. And I don't just mean throwing tantrums rather than thinking things through, or being gullible. I mean literally, at the beginning of our lives, we know nothing.
A baby knows how to poop, cry, and latch on to anything that remotely resembles its mother's breasts. Would you trust a child to cook you a meal? File your taxes? Defend you in court when it turns out the government doesn't accept a dried clump of mashed banana as a valid address? On its own, a baby doesn't even know how to use a spoon. Thus, it's hardly sensible to appeal to how we do think as children as a sensible guide for how we should think as adults.
But of course, the atheist might still argue that although babies don't begin with knowledge, they can still obtain it. In a similar vein, atheists like Penn Jillette argue that if we lost all knowledge of science, we could reconstruct it over time, whereas if we lost all knowledge of religion, we could never recover it. And yes, with some aspects of religion, this is true. The problem is this flawed , and provably false assumption that in order for something to be true, you should be able to conclude it for yourself. This is so silly that it's not only unsustainable, atheists would not attempt to sustain it in any other situation.
The irony of Penn Jillette's argument is that his own identity is one such thing that could never be reconstructed if all knowledge was lost. How would he know how old he is? How would he remember his original name? If his birth certificate survived the great knowledge purge, how would he re-construct the language in which to read it? It wouldn't be too hard to prove that he is a fully grown human male, but his more abstract traits, such as his name and history, would be lost forever.
Interestingly, this argument works quite well against Penn's own religion, that being Evolution. Evolution, rather than being a fact of science, is actually a myth about history. For this reason, even without some mystical erasure of all human knowledge, Evolutionists cannot agree on what happened, when it happened, or even the process by which it supposedly happened. Furthermore, the evidence we continue to obtain militates against it. Thus, it is not Christians who indoctrinate our children, but Evolutionists. Evolutionists like Eugenie Scott even flat out admit that Evolution is so weak, teaching students to think critically about it might make them reject it.
So, what we have here is two comparable arguments. One of them poses some hypothetical catastrophe in which humanity forgets everything we've ever known, then somehow figure things out all over again. It is assumed the result would be an atheistic society with a strong grasp on science. We'll gloss over the fact we'd never recover our knowledge about history. But the other argument focuses on the way things are now, not losing any knowledge, but that if we maintain the knowledge we have, we can reasonably conclude that religious claims such as Evolution are false, whereas religious claims such as Christianity is true. In other words, the atheist argument depends on an unrealistic scenario with an unprovable (not to mention unlikely) conclusion. The Christian argument focuses on the existing scenario, with a demonstrable conclusion. I think it's clear which side has the more logical case.
At this point, it's worth noting that although a sudden loss of knowledge is unlikely, there was a time of supernatural confusion. The human race conspired to build a tower to the heavens themselves, rather than filling the Earth as God had commanded, and so He confused their language. From there, they spread across the Earth, and notably retained Theism.
This makes sense under a Christian worldview. If there really was a Tower of Babel event, it makes perfect sense that the human race is instinctively Theistic, and has therefore formed a large number of religions in spite of relative isolation. But under an atheistic worldview, this event did not happen. Thus, atheists, and in particular those who argue that no one would draw religious conclusions on their own, are forced to explain how multiple isolated cultures did draw religious conclusions on their own.
See, beyond all imagination, the atheist is actually wrong that we begin as atheists. Scientific studies have shown that while we are obviously not born with Bibles for brains, children do instinctively know God. One such study was conducted by researchers at Oxford University, who concluded that Japanese children recognise design in nature. This is significant, as Japan is not an especially hospitable culture for Creationism. In Japan, there are two dominant religions: Shinto and, to a lesser extent, Buddhism, neither of which teach God as a creator. Nevertheless, without being taught about a Creator God, the children naturally concluded that He must exist, and created living organisms.
This makes perfect sense in light of the Christian faith. Scripture teaches two kinds of revelation: Special revelation (that which is given directly by God, such as Scripture), and general revelation (that which we either know by instinct, or conclude by observation). We are told that God's invisible attributes can be understood from creation, and that He has written the law on our hearts. For this reason, God can, and will, condemn those who "sin without law", since "they are a law unto themselves".
But it's not just general revelation that we find throughout the world in ancient cultures. We do see some strange religions that could theoretically come about even if the Bible wasn't true, but we also see things which we would not expect to see in this scenario.
An interesting starting point is "dragons". There is no shortage of dragon legends, and even depictions, throughout history. This makes sense when you compare the Christian and Evolutionist beliefs about dinosaurs. If Evolution is true, human beings have never encountered anything much more spectacular than a crocodile, or a komodo dragon. But according to the Biblical timeline, rather than dinosaurs being millions of years old, they were actually made alongside mankind. If you accept the likely theory that Behemoth is a dinosaur, the Bible even explicitly says as much. Thus, it makes perfect sense that we have so much evidence of man clashing with dinosaurs.
But even this evidence pales in comparison with very specific historical events such as creation, the Noahic flood, and the Tower of Babel, all of which exist, albeit in diluted form, in the memories of several isolated cultures. Ancient Chinese culture even has a god by the name of Shang Di, who so closely resembles the Biblical God that it almost seems fair to count them as one and the same.
With all of this in mind, it seems this particular atheist argument does more to harm atheism than promote it. It is illogical from the outset to appeal to the default state of human ignorance to suggest we should therefore maintain that ignorance, but on top of this, it turns out it's not even true. It is utterly false that we are naturally atheistic, but must be taught religion. In reality, atheism is a relatively new thing to our species, because for thousands of years since the Fall, we have remembered God, up to and including His historical dealings with man up until the Babel Dispersion. In other words, this is just one more example of atheists trying, and failing, to claim the intellectual high ground.
Unfortunately, one thing none of us need to be taught is how to sin. Each and every one of us do, say, and even think evil things, long before we are even capable of understanding evil. For this, God owes us His everlasting wrath. But this is not His desire. Therefore, before humanity was even created, God concocted a plan of redemption. Through the same people who brought sin and death into this world, God Himself would enter creation as a man, live a perfect life, and die a sinner's death. This effectively allows us to "swap verdicts". Jesus, who was innocent, was punished for our guilt. Therefore, in Him, we cannot be condemned. The only condition for this is faith. Those who confess Jesus as Lord, and believe He rose from the dead, will be saved.