What is stopping people from setting up multiple pools and getting 50% that way?

How do you really know all those pool operators are actually different people?

For a 'trustless' system, it seems like you need to place an awful lot of to trust in this small, secretive cabal of mining operators.