"...for when Gentiles, who do not have the law, by nature do the things in the law, these, although not having the law, are a law to themselves, who show the work of the law written in their hearts, their conscience also bearing witness, and between themselves their thoughts accusing or else excusing them) in the day when God will judge the secrets of men by Jesus Christ, according to my gospel." - Romans 2:14-16
The Bible is more than just a book containing all we need for faith and practice. It also lends a helpful insight into the mind of an unbeliever. God, as my friend "Jar Jar" used to say, is the ultimate psychologist. He knows the human mind because He created it.
One of the things He tells us is that although there genuinely are people who believe in their hearts that there is no God (Psalm 14:1), they always have at least some innate knowledge of Him. Atheism is, as I like to say, real, but deliberate.
Ask any atheist to justify morality, they will never be able to give a consistent answer. Morality comes down to a subjective opinion, and yet with the exception of literal sociopaths/psychopaths, morality is also a common opinion. Everyone has a sense of morality, everyone cries for justice. Nobody has to be taught that right and wrong exist, save the aforementioned sociopaths/psychopaths.
Let's talk about them for a moment. Because of their unusual mental state, these people rarely feel guilt (sociopaths can, to an extent). A psychopath can murder a person and feel nothing. No atheist will ever be able to say "that's wrong", because in an atheistic world, it wouldn't be. Every atheist, even those who don't know it, has at the very least an oblique awareness that there is a God to whom they are accountable.
Now, obviously, this awareness is marred. We aren't born with Bibles in our heads. That's why God gave it to us in book form. But we do have some awareness. One might compare this to martial arts. No one has to be taught that punching someone in the face hurts, but that's rarely going to help you if someone attacks you on the street. Some people don't even know how to make a fist, but learning a martial art greatly increases your chances of surviving an attack. Similarly, most people instinctively know right and wrong exist, but we all need to be taught the finer details, and to choose right over wrong.
When an atheist acknowledges morality, they inadvertently acknowledge God. The problem they have is that this is all He needs to convict them when they break the laws they know He has. The Bible does indicate that He judges according to knowledge. Hell will be more tolerable for the ignorant than the wise. But knowing justice exists allows us to become unjust (Romans 7:7-12). And we all fall short in that department.
But God doesn't just leave it at that. It has never been so simple, for as much as God loves justice, He loves mercy more. And so a sinner has a choice. 2,000 years ago, Jesus, the Son of God, took the punishment for sin upon Himself. He died on the cross, and more importantly was raised to life. Humanity is now faced with a choice: Reject that offer and suffer eternal justice, or repent and confess Jesus as the risen Lord, inheriting a place in His eternal Kingdom. Don't let your faith remain oblique: Put your faith entirely in God.