At the moment, I find it entertaining in the way a well constructed conspiracy anthology is entertaining: it has momentum, it is confidently narrated, it stitches disparate cues into a single totalizing story, and it rewards the reader with the pleasurable sensation of “seeing the hidden structure”.
To be fair, it is not empty sensationalism. There is real, interesting thinking in it, including a serious intuition about learned systems: when you train opaque optimizers and then scale them, you may not be meaningfully “authoring” their goals at all, only shaping behavior within a narrow envelope of tests. The authors are good at making that feel vivid, and at pressing on a genuine engineering discomfort: we do not possess the sort of formal guarantees we would want before deploying something that could outplan us.
But the book also reads like it is constantly crossing the border between science and the aesthetics of science. It uses technical vocabulary and gestures at mechanistic plausibility, yet frequently substitutes rhetorical certainty for disciplined inference. The argument is often assembled the way conspiratorial arguments are assembled: a chain of “and then, of course” moves, where each step is individually imaginable, but the overall probability mass is treated as nearly one. That is exactly what makes it fun to read and also what makes me hesitant to treat it as analysis rather than advocacy.
So I’d describe it as a stimulating thought experiment, not a reliable map. It’s valuable as a stress test for complacency and a prompt for better questions, but it is too rhetorically closed to function as a balanced assessment. If you already have an engineer’s skepticism about what is proven versus what is merely asserted, you will catch the slippage: hard problem becomes impossible problem; uncertainty becomes inevitability; plausibility becomes destiny.
Net: enjoyable, provocative, often insightful, and epistemically sloppy in places, with “science flavored certainty” doing more work than the science itself.