Sure, but I’d wager it goes beyond the singular manifestation of power as money. Plenty of other manifestations, and all lead to similar places.

At this point, anything remotely near bare-metal material reality triggers such a cognitive dissonance in a lot of folks, on account of all of the bs they have in their heads, that all of that bare-metal material reality is discarded as too painful.

Fun speculation: we are CPUs for the information systems we inhabit, like scientific method, political ideologies, etc.

It’s trivial to get LLMs to act against the instructions

There is a lot more that goes into it than just being correct. 18000 waters may have been the actual order, because somebody decided to screw with the machine. A human who isn’t terminally autistic would reliably interpret that as a joke and would simply refuse to punch that in. The LLM will likely do what a human tells it to do, since it has no contextual awareness, it only has the system prompt and whatever interaction with the user it had so far.