"Superalignment" is (was?) OpenAI's term, so it might be more accurate to say he is joining Anthropic to work on alignment.

Looks like superalignment was Jan Leike's term, since the team at OpenAI dissolved immediately without him.

Is there a difference between "superalignment" and "alignment" ?

Yes. “Superalignment” (admittedly a corny term) refers to the specific case of aligning AI systems that are more intelligent than human beings. Alignment is an umbrella term which can also refer to basic work like fine-tuning an LLM to follow instructions.

Is this not something of an oxymoron? If there exists an ai that is more intelligent than humans, how could we mere mortals hope to control it? If we hinder it so that it cannot act in ways that harm humans, can we really be said to have created superintelligence?

It seems to me that the only way to achieve superalignment is to not create superintelligence, if that is even within our control.

Not self-evident. Fungus can control ant. Toxoplasma gondii can control human. Who is more intelligent? So if control of more intelligent being is possible, could it be symbiotic to permit? Alpha-proteobacteria sister to ancestor proto-mitochondria and now we live aligned. But those beings lacked conscious agency. We have more than them. Not self-evident we will fail at this.

Another example is the alignment between our hindbrain, limbic system and neocortex. Neocortex is smarter but is usually controlled by lower level processes…

Note that misalignment between these systems is very common.

Many people share your views, but others believe it is possible.

Huh! All this time I thought the "super" was just for branding/differentiation.

Alignment was the original term, but has been largely coopted to mean a vaguely similar looking concept of public safety around the capabilities of current models.

That was definitely part of it.

i suppose the difference between imaginary and "super"-imaginary isn't very important from a practical point of view.

they worry about alignment for ai, I worry about alignment for the corporations that wield technology, any technology.

Oh yes. One is super.

No doubt leadership at OpenAI, Google, and Microsoft are in favor of any regulations that would cement their lead.

For the rest of us, the relevant question is: are the costs worth the benefits?

> The risk [...] is that radically new products and approaches in the arena never get a chance to be developed and benefit consumers.

Totally. And the opposite risk is that these companies deploy technologies that cause massive harm to people, without adequate testing, because they're caught up in a race.

I think some regulations would be helpful on balance -- like reporting of large training runs, as in SB 1047 -- and some wouldn't -- like (hypothetically) requiring a license to train small models.

I like to imagine that, as in Ender's Game, the top-ranked player is unknowingly allocating memory for some critical infrastructure somewhere.

Getting Severance vibes.

Oh man,(spoilers) my pet theory is that the 'work' is actually them self-adjusting their implants. Remember they describe a number as a 'feeling' and some numbers 'feel' wrong. And we see behind the scenes that they are testing the main character with the psychiatrist/wife to see if it breaks the 'chips control/severance'.

They are simply engineers perfecting the implants so that they operate better.

I think that the numbers are in-house red team fuzzing attempts against the implant, with the goal being lower mean time between failures of the implant. It could also represent a debugger IDE, with the “spooky” numbers that we see jumping around onscreen on the workstation representing some kind of bad data that doesn’t pass tests that the normal numbers onscreen do pass.

I’m reminded of the in house help desk tech support that someone calls for in the film Vanilla Sky, analogously.

Just started watching this, and I have no real idea what's going on :)

“To compete your registration, tell us whether this image has a stop sign. And hurry it up, our self driving car needs this.”

https://xkcd.com/1897/

You're right that LLMs can produce new knowledge, but you're misinterpreting the quote. The claim in the article is that LLMs won't produce anything new when asked for "a paper about Middlemarch". The rest of the article goes on to demonstrate how language models can create new knowledge.

Wouldn't we just get pop-ups saying, "please enable X-Allow-Tracking for this website"? Same thing that some websites do in response to ad blockers.

Yes, it means that if you consent for cookies, you don't get annoying popups everywhere. Or, what actually would be interesting, a law explicitly disallowing "please enable X-Allow-Tracking for this website" popups.

Right now the web is broken anyway - some pay (in data and ads), some are free-riders. And everyone is pested by cookie popups. This "no tracking unless required for functionality" would make it nice to change a model for actually paying for use. (It promotes quality content, less distractions, less clickbaits; and thinking twice if you want to spend more time on yet another meme aggregator.)

The difference in running time grows exponentially with the size of the input. I'd say that's "literally" exponential.

Interesting! I'd like to see a version where the double-time track is fading in continuously, instead of arriving all at once at the beginning of a measure (which would be a closer analogue of the Shepard Tone). Can't quite tell if the other reply is an example of that.

OP agrees with you: antithesis of non-public = public.