The majority - as I think you inferred. Ideally though, it would be an overwhelming majority.
For some perspective, do you think 100% of U.S. citizens agree with all clauses of the constitution, as it's currently written? Obviously it's less than that. And if you're okay with that, then it means you're willing to accept some tyranny of the majority when it comes to establishing our constitution.
The fact of the matter is, a certain tyranny of the majority is utterly unavoidable. At least when it comes to the most fundamental decisions. And so our goal can't be to completely avoid all tyranny of the majority - because to do so would be impossible. But what we can try to do is find the right things that truly do and will make sense for the vast majority of people for now and as far into the future as we can foresee.
Contrary to what you've suggested, I'm not personally seeking any authority to tell people they are wrong. I'm just trying to get people to buy into the idea, first, that an unavoidable part of getting along involves finding common ground. And second, that much of our trouble in finding common ground lies in stubbornness. We stubbornly believe whatever our parents, family, and friends tell us, rather than trying to form a logically consistent worldview.
I've explored this idea much further in these articles:
1. https://medium.com/@nebulasaurus/will-humans-ever-settle-on-one-religion-4e50c81b1d96
2. https://medium.com/@nebulasaurus/4-things-all-good-citizens-believe-implicitly-dd37c4dbef6
3. https://medium.com/@nebulasaurus/faith-non-belief-and-the-parable-of-the-many-moons-c1fa5851e29d