Simply get every nation on earth to cooperate and ban a vaguely described technology that hundreds of billions of computers can run to varying degrees of efficiency! It's that easy!

If we can't get this level of cooperation for global warming, which is largely the result of a few dozen companies, what makes you think that governments across the world can stop everyone with access to a device with a reasonable amount of compute power? This idea is a non-starter and assumes that there is one single entity that could halt AI altogether. The genie is out of the bottle.

As I see it for country to disconnect from AI they'd need to go fully isolationist. Disconnect the internet fully from the rest of the world, block all mail, block all imports, disconnect all financial markets, etc. Otherwise they'd simply become a consumer of the AI output of other nations. For example, if AI can predict stock markets better by efficiently parsing financial documents then eventually foreign investors leveraging AI would dominate. That will work for a while but eventually AI will get cheap and efficient enough to be easily hidden. So now you need the government to police for AI, search people, track everything they do and so on. Criminals using AI will rise to power and prominence until stopped. Essentially prohibition or the war on drugs all over again.

edit: And of course to better understand and deal with the AI threat the government would be given exemptions to the laws. These exemptions would be used more and more widely by the government to exert power while the population is not allowed to even look into what is possible.

This isn't a math quiz. For practical purposes, we can start with a list of technologies that are clearly harmful. Generative AI like ChatGPT, AI image generators, AI text generators that write something based on a prompt, etc. all halted.

Llama and Stable Diffusion have proliferated already. Many countries view this as an arms race. Propose a plausible way to put the AI toothpaste back in the tube, because this stance seems profoundly impractical.

Until when? The only way to avoid societal impact of AI is to either stop it for decades until we reach some utopia UBI state or lobotomize current generative AI to a point where it's useless.

Personally, I believe generative AI should be lobotomized as you said. After pouring over all sorts of possibilities, I think no good will come of it.

Why would you want that? What is "clearly harmful" in your view?

Change scares people. This has been the case for every single technological invention including writing. Predicting the transformative impact of a technology is nearly impossible (or sci-fi writers would have a better hit ratio) but thinking about how it will break what we currently have (versus what will replace the current status-quo) is fairly easy. Accepting ones own limitations and inherent ignorance about the future is something many people very much do not want to accept.