You're mixing first past the post voting, and referendums with an impeachment process aimed at removing people from office, three different things.
The founding fathers put in place safeguards for similar reasons:
People often refer to the United States as a democracy, but technically speaking, that’s not true. It’s a republic.
Big deal, you say? If you care about your rights, it is. The Founding Fathers knew their history well, so they knew better than to establish the U.S. as a democracy.
In a democracy, of course, the majority rules. That’s all well and good for the majority, but what about the minority? Don’t they have rights that deserve respect?
Of course they do. Which is why a democracy won’t cut it. As the saying goes, a democracy is two wolves and a sheep voting on what’s for dinner.
The Founders were determined to forestall the inherent dangers of what James Madison called “the tyranny of the majority.” So they constructed something more lasting: a republic. Something with checks and balances. A system of government carefully balanced to safeguard the rights of both the majority and the minority.
That led, most notably, to the bicameral structure of our legislative branch. We have a House of Representatives, where the number of members is greater for more populous states (which obviously favors those states), and the Senate, where every state from Rhode Island and Alaska to California and New York have exactly two representatives (which keeps less-populated states from being steamrolled).
Being a republic, we also don’t pick our president through a direct, majority-take-all vote. We have an Electoral College. And a lot of liberals don’t like that.
Their attacks on the College are nothing new, but the defeat of Hillary Clinton in 2016 renewed their fury. After all, as they never tire of pointing out, Mrs. Clinton captured more of the popular vote than Donald Trump did. They see the Electoral College as an impediment to their political victories, therefore it’s got to go.
Were it not for the Electoral College, presidential candidates could act as if many Americans don’t even exist. They could simply campaign in a small handful of states with big populations. Who would care what the people in Iowa think? Or Wyoming? Or any number of other states with smaller populations?
The people in “flyover country” don’t get enough attention as it is, but without the Electoral College, they’d be completely at the mercy of the majority