Every time an Old Earther says Genesis wasn't supposed to be taken literally, they are admitting that this is what Genesis literally says. Two questions arise. Question one is if it's not supposed to be taken literally, how is it supposed to be taken? Question two is why?
Now, there are so many weird and wacky interpretations I've seen, but they always have one thing in common: nothing in the text itself leads them to that conclusion. It's always something on the outside. Specifically, we need to change our interpretation of Genesis to fit the prevailing culture's understanding of origins. Because, you know, non-Christians have always been better at interpreting God's word than Christians....
But here's a thing to consider. Genesis was written around 1,400 B.C. Principles of Geology was written in 1830 A.D. That means that for well over 3,000 years, Genesis had been read without Uniformitarianism in mind. In other words, Jews and Christians had to read Genesis as it was written. And surprise surprise, they all came to similar conclusions, and all vehemently opposed the old earth views that did exist at the time.
This includes Augustine, who is by far the most misquoted ancient writer in this entire debate. Old Earthers frequently refer to his work "The Literal Meaning of Genesis" to claim he gave us permission not to take it literally. Now of course, his opinion is as irrelevant as anyone writing today. Prophet, Apostle, God, Augustine was none of these things. He was a guy born three centuries after the faith was "once for all delivered to the Saints" (Jude 1:3), so the fact that Old Earthers refer to him as if his opinion matters further shows that their interpretation is not based on God's word, but on man's.
But as irrelevant as the man is, he is at least useful in showing that the early Church took Genesis literally, because Augustine himself was what is commonly referred to as a "Young Earth Creationist". In The City of God, he wrote "They are deceived, too, by those highly mendacious documents which profess to give the history of many thousand years, though, reckoning by the sacred writings, we find that not 6000 years have yet passed".
Reckoning by the sacred writings (i.e. "this is what the scriptures say"), not 6000 years have yet passed. Anyone who says even many thousands of years (OECs usually want us to accept millions, even billions) is deceived by mendacious documents. So, what would he say to the modern Old Earther who uses his own works to discredit Genesis? He'd say they're deceived by mendacious documents.
And mendacious they are! Lyell specifically came up with his principle of Uniformitarianism to "free science from Moses" (because up until his day, many scientists believed Genesis). Not only did he have an agenda, he was willing to lie for it. For example, when he visited Niagara falls, he was informed that they were retreating by about a yard per year, which would mean that the erosion of the whole gorge would have taken around 10,000 years. But this wasn't long enough for Lyell, so he wrote "the average of one foot a year would be a much more probable conjecture. In that case, it would have required 35,000 years for the retreat of the Falls, from the escarpment of Queenstown to their present site."
Can you get more hypocritical than that? You come up with a theory that the present is the key to the past, so if we observe one yard of erosion per year (a rate that was observed to be fairly constant for forty years, and has actually been observed to increase occasionally), that means it's always been one yard of erosion per year, but then you decide that leads to a younger age for the falls than you want, so you throw away observation and go with the longer date? What an obvious snake oil salesman.
I use that term deliberately, because it is not my own. Rather, these are the words of Warren Allmon, Director of the Paleontological Research Institution in Ithaca, NY, who in 1993, wrote the following:
"As is now increasingly acknowledged, however, Lyell also sold geology some snake oil. He convinced geologists that because physical laws are constant in time and space and current processes should be consulted before resorting to unseen processes, it necessarily follows that all past processes acted at essentially their current rates (that is, those observed in historical time). This extreme gradualism has led to numerous unfortunate consequences, including the rejection of sudden or catastrophic events in the face of positive evidence for them, for no reason other than that they were not gradual." (1)
All of this shows that when Old Earthers say Genesis isn't supposed to be taken literally because a literal understanding conflicts with the modern understanding of origins, they are ultimately endorsing an interpretation that is not only so brand new that it would have been alien to anyone living in the first century and before, but that is actually based on a very specific lie, backed up by a very flawed philosophy, invented by men who sometimes brazenly admitted that they were trying to oppose God. This really is a case of God's word vs. man's word, and for a Christian, it should be a no-brainer which one to pick. But if you're going to pick the latter, at the very least admit it.
References
1. Allmon, W.D. Post Gradualism, Science 262:122–123, October 1, 1993