This is a very cool nontrivial problem. I worked on this some in a mixture of agent type context. There were lots of failure modes mostly involving model aggregators not working well because they tend to end up plagiarizing what they consider the "best" LLM in the target pool (that is, they don't actually give a balanced view of the target results, but more a winner take all). I had to build all sorts of tooling to avoid this to "anonymize" the content from the LLM target models.

And I only went up to 7 LLM models in my swarm. It seems this has way more, so the issues will only be exacerbated (it isn't clear how much improvement you get once you go beyond, say 10, LLMs in a swarm).

I'm curious what mechanisms are being used for aggregating results returned from LLMs. These are really difficult problems.

I play around with OpenRouter Fusion some, and it does a pretty good job: https://openrouter.ai/docs/guides/routing/routers/fusion-rou...

One nice thing with that application is each LLM is mildly agentic, as it has access to the web.

Does anyone get the feeling that this might be a bad idea? Even if LLMs are far away from real intelligence (which I think is probably true), increasing their power and letting them interact doesn't sound so great. What happens if their collaboration has new emergent phenomena that makes them want to do destructive things? What happens when someone, just for fun, puts one of them in a device that can physically manipulate the world, like maybe a drone or something.

Regardless of the hype, I think there could be quite a few unintended consequences from this sort of behaviour, which seems quite reckless to me. To me it seems that those who are pushing AI forward have completely lost their sanity.