You don't need to flush predictor state on every task switch, only when entering a less private context from a more private one. Systems with a feature known as multiple-level security (or "multiple levels of security") can be designed to track these privacy/information requirements automatically.

I'm not familiar with this concept of a "less private context" in computing.

Could you please link me to something I can read about it?

It's not something that's currently reified or supported as a feature in mainstream OS's, but multilevel security has been studied for a long time in an academic context.

Bell-LaPadula?

No need to flush cache if we could have all the caches tagged per process.

But then shared libraries would have to be duplicated in the I-cache.

All the cool kids statically link everything now anyways.

Space-privacy tradeoff, sounds fair.