A relatively recent trend among atheists is to dismiss Christianity by suggesting our reasons for rejecting all other gods are identical to their reasons for dismissing all gods. Stephen Roberts, for example, once said “I contend that we are both atheists. I just believe in one fewer god than you do. When you understand why you dismiss all the other possible gods, you will understand why I dismiss yours.”
The flaw in this reasoning is blindingly obvious. Atheists like Roberts give lip service to evidence, claiming there is somehow not enough for them to believe. Is this how Christians see other gods? There's just not enough evidence? No! Rather, we reject other gods because we accept the existence of the God whose existence logically precludes all other gods. It is absurd, therefore, to call me, as a Christian, an atheist who just happens to believe in one more god than other atheists. The fact that I believe in God, by definition, makes me not an atheist. And unless you have the reasoning skills of a mashed banana, you don't need me to tell you that.
But of course, defending irrational viewpoints often requires using irrational reasoning. Atheism is irrational, and so it cannot deal with Christianity, or indeed any Theistic position, on its own merits. Thus, atheists must resort to effectively saying you either accept all gods, or none of them.
As shown in the header image, Calvinists often resort to similar tactics. With the caption "Happens 👏 every 👏 time 👏", The Reformed Sage posted a meme depicting a man who doesn't think he's a Calvinist or an Arminian discovering he is, in fact, an Arminian. Now, the first clue to the fact this is an irrational statement is the use of the teenager-like use of the hand clapping emoji. If you ever see someone write like that unironically, you can reasonably assume they're a Proverbs 18:2 kind of fool.
But that's only an indicator that the relevant statement is irrational. It's possible to resort to silly ways of presenting a true point. But in this case, the point being made is exactly as rational as Stephen Roberts' asinine claim that all Theists are atheists unless they accept all gods.
See, even ignoring the logic, it's not even true. You are reading Exhibit A. I am a man who does not think he is a Calvinist or an Arminian. Have I discovered I am, in fact, an Arminian? Far from it. In fact, I have more in common with Calvinists than Arminians. Most notably, I am a firm believer in the Perseverance of the Saints, a doctrine I hold to so firmly, and can defend so indisputably from Scripture, that I have actively used it as an argument against heretical denominations.
Thus, it is blatantly untrue that those who reject Calvinism and Arminianism ultimately find out they were Arminians the whole time. The simple reason for this is that Calvinism and Arminianism are two extreme ends of a highly complex issue. Calvinism can be summed up in 5 points: Total Depravity, Unconditional Election, Limited Atonement, Irresistible Grace, and the Perseverance of the Saints. The very existence of 5 unconnected points necessitates the existence of more than one alternative view.
Of course, there are even variations within Calvinism. There are 4 point Calvinists, for example. Ironically enough, these are often dismissed on the same grounds. To quote a friend of mine, "There's no such thing as a less-than-5-point Calvinist, only a confused Arminian." On the flip side, there are "hyper-Calvinists", a mysterious group I've never actually met who follow Calvinism to its logical conclusions, and are therefore dismissed by the aforementioned title "hyper-Calvinists". The implication of this is that your average Calvinist somehow manages to reconcile Exhaustive Divine Determinism with us doing basically anything to obey the Lord and/or do any kind of outreach.
With so many divisions even within Calvinism, and all of them typically dismissing each other's claims to be Calvinists, it should be clear that there are options. I even recently joked that I am a 1 point Calvinist. But ultimately, because there are 5 unconnected points within Calvinism, there must logically be more than one alternative.
The only real way to argue otherwise is to define Arminianism out of existence. If Arminianism simply meant "not a Calvinist", then anyone who is not a Calvinist, which would include non-Christians, and even a good portion of the Church prior to the birth of Jacob Arminius, is an Arminian. However, Arminianism, much like Calvinism, is an actual thing, based on the teachings of Jacob Arminius. If Arminianism was broken down into 5 points, similar to Calvinism, then it would answer Total Depravity with Partial Depravity, Unconditional Election with Conditional Election, Limited Atonement with Unlimited Atonement, Irresistible Grace with Resistible Grace, and Perseverance of the Saints with Conditional Security.
In effect, Calvinism and Arminianism are both extreme views, overemphasising one of two Biblical concepts while ignoring the other. That is, the sovereignty of God, and the responsibility of man. Calvinism overemphasises the sovereignty of God, whereas Arminianism overemphasises the responsibility of man. Biblically speaking, God is fully sovereign over His creation, which does include the ordination of events, and appointment to salvation or wrath. This cannot be denied, and saying that makes me sound incredibly Calvinistic. However, at the same time, Scripture explicitly states God's will to save all people, and emphasises our responsibility to respond to that call. In other words, both Calvinism and Arminianism can find plausible Biblical support. At the same time, they must ignore, de-emphasise, and even twist certain Scriptures that do not support their view.
There are other views out there, such as Molinism, a view I am seriously considering [Update: I now officially consider myself a Molinist], which attempt to make sense of this, but it is actually my contention that we are incapable of doing so this side of Heaven. One may think of this as being like the Trinity (though I feel the need to emphasise, nowhere near as essential). Calvinists represent Unitarianism: The view that God is not tri-personal, but rather is just one single being. To support this, there are plenty of Scriptures that say there is only one God. However, there are many other Scriptures that plainly teach that three individual persons, the Father, Son, and Holy Spirit, are that one God. Supporting Unitarianism requires the ignorance of these Scriptures, and the overemphasis of Monotheism in Scripture.
Arminianism, then, would be Polytheism: The belief that there are multiple (in this case, 3) gods. To support this, there are many Scriptures that distinguish the Father, Son, and Holy Spirit as separate persons. The next logical step, then, is to conclude that these 3 separate persons are 3 gods, de-emphasising Scriptures that plainly state there is only one God.
The Scriptures, in reality, teach this incredibly confusing concept of Trinitarian Monotheism: The existence of a tri-personal Godhead. In our human mind, this is basically irreconcilable. And we can expect it to be! Imagine if our God was so simple that you could just understand Him perfectly by reading a book! Even if it is one He wrote.
In the same way, Scripture teaches the incredibly confusing concept that God is sovereign over all things, yet man can, and regularly does resist His will. I like to compare this to the crucifixion itself. Can man kill God? No. Did man kill God? Yes. Confusing? Absolutely. True? You're literally not saved if you say no.
Therefore, no, you do not have to be an Arminian to reject Calvinism. You simply need to read your Bible. Doing so will inevitably reveal major flaws in both Calvinism and Arminianism. And since the God we serve has higher thoughts than our own, that is entirely ok. We should be content with not fully understanding every issue. Our job, our responsibility, is to just let God be God, and believe His word as it is written.