Who verifies the verification systems?
Particularly the things that AI can do, but aren't easily verifiable
AI came for the translators, and no-one spoke out. Well they did, but no-one cared, because having automated high-quality translation is so useful. Personally, I translated vast quantities of classical papyri, a Greek travel writer, a Byzantine encyclopedia and some 19th century children’s literature. It’s easy enough to verify the output, because you can do a back translation.
AI has already come for the software developers, who are in denial and really salty about it. I’ve been having lots of fun writing stuff at 10x what I used to (and I could write code really fast). It’s easy enough to verify the output, because you can write automated tests.
AI is coming for the mathematicians. They know about it and are a bit excited about it. Far too much of my PhD thesis is bot-proven theorems. It’s easy enough to verify the output because you run it through a proof checker.
AI is coming for the accountants. They barely realise how close it is. My wife looked at https://accountingops.industrial-linguistics.com/ and started excitedly telling me about it until I pointed out that that’s one of my projects. It’s easy enough to verify, because that’s what the whole accounting and auditing industry is about.
You get the picture.
AI is coming for all white collar jobs. That includes academia, if you’re an academic. It includes whatever it is that you do if you have a white collar job.
We all think that our field will be immune, but why would we be?
If there is a way to automate whether a piece of work is correct or not; especially if there is a way to automate determining whether one piece of work is better than another, then AI’s greater speed will inevitably have it take over for the human beings in that field.
Academia, for example, is rapidly changing to a different beast. It is not going to be about what we can do. We are the slower and junior partners. Machines can produce words, proofs, slides, reviews, grant applications, analyses, and plausible narratives at industrial scale. Or maybe that’s at “academic scale” if you’re not into industry bridges.
Output is getting cheaper so output stops being the scarce resource. What becomes scarce next is access to physical reality. Also the willingness to sign your name to a result when the model may be wrong.
Academia (and all industry white collar jobs) have to transform to be about how we can set ourselves up not to be fooled by what our AI bots are telling us when they are making mistakes.
The signals we used to use — a well-written paragraph, a complete literature review, a decent bit of code, a proof that looked basically right — these used to tell us something about the person who produced them. Now they don’t. You can have a well-written paragraph of complete nonsense (people have said this about my writing even before AI). GPT-5.4 Pro can do a literature review to find all the terrible papers that support your point. Unless the proof goes through Lean or Roqc, you can’t really tell if it is the hard-sweated believed-to-be-correct proof it looks like.
Who verifies the verification systems? How do we verify the verification systems?
This is all true even if we don’t get AGI. Just the diffusion of existing AI technology is enough to make this transition happen.
Jobs do not disappear the moment the AI can do the work anyway. They linger in larger organisations while everyone is flailing around trying to find new ways to certify quality. And then by inertia a few of the doers will fall into the job of verifiers, even though the required skill set is completely different.
Maybe we don’t want this future, but we don’t seem to have the power or collective will to stop it. Maybe we can shun the people who say out loud that academia (or whatever job we’re doing) is a white collar job just like every other white collar job that is in danger. It’s out of the Overton window right now, and maybe keeping reality out of the Overton window a little longer will slow the process down a little. That could be good.
How do we verify science/humanities/art/music/culture/sales/marketing in a world with AI that can slop at 100x our speed and occasionally stumble on brilliance beyond human capability by chance?
That’s the #1 question for our era.