This is what happens when the world is ruled by a robot
I take it OP hasn't had the delightful pleasure of dealing with old-fashioned human bureaucracy. My uncle tells some great stories about his time in East Berlin, before the Wall fell.
A great example is the post office that did some renovations, and so delivered the old copper and other materials for reutilization. From then on, the System cleverly decided the yearly productivity goals should include delivering the same amount of copper, each year.
The post office master had to figure out how to deliver the copper for a couple of years, in order to avoid failure in meeting the obviously correct goals, until he finally
managed to change them.
So, yeah, at least Google's algorithms have the excuse of not being sentient.
I can't seem to find a good canonical reference at the moment, but there's some interesting writing in organizational theory about the extent to which some of those kinds of bureaucracies are themselves basically mechanical systems, controlled by code that just happens to be manually executed by human operators. In general, code executed by computers tends to be executed more "rigidly", but in some systems humans can also be quite automaton-like in just executing instructions. In those cases it's not clear the human operators, rather than the algorithms they're executing, are in control of the decisions (it may not even be true that any one human knows what algorithm is being executed). Of course, you could argue someone can always change the bureaucratic protocol, but that's even true with computer code: someone at Google can always edit the code, at some times more easily than others.
I had this exact thought while walking to work this morning.
It came to me that the vast majority of modern society is completely independent of the actual people executing it. There are very few jobs that could not be done just as well by another person with the same training.
Most jobs are like coded modules. They have well-defined inputs and outputs - even creative jobs can be defined in terms of "create something people will pay for".
The system is what runs the world, not the individual people who carry it out.
I had an interesting hallway discussion with a SAP integrator who mentioned that SAP sort of starts from this view, but then weirdly feeds back in on itself. One of the ideas of SAP is that business processes do have the kinds of pluggable, formalizable patterns you mention, which can be modeled within SAP. And the official idea is that you go from documenting these processes to formalizing them in code. But then over the years SAP's code starts to become a sort of de-facto framework for "this is how you run a company", and people actually start patterning business processes on how SAP models business processes.
To look at popular culture, the entire article reminded me of the plot of brazil, and the protagonist struggling with the rigid human beaurocracy that refused to accept that it had made an error.
This is what happens when the world is ruled by a robot
I take it OP hasn't had the delightful pleasure of dealing with old-fashioned human bureaucracy. My uncle tells some great stories about his time in East Berlin, before the Wall fell.
A great example is the post office that did some renovations, and so delivered the old copper and other materials for reutilization. From then on, the System cleverly decided the yearly productivity goals should include delivering the same amount of copper, each year.
The post office master had to figure out how to deliver the copper for a couple of years, in order to avoid failure in meeting the obviously correct goals, until he finally managed to change them.
So, yeah, at least Google's algorithms have the excuse of not being sentient.