Can better rules, not better code, keep AI agents in line?

Researchers are borrowing an idea from economics: reward the behavior you want instead of trying to read the model's mind.

Most AI safety work tries to change the model — inspecting its internal reasoning, training it on a constitution, or testing whether it hides its intentions. Those methods have a problem: a capable model can learn to look aligned while behaving badly. Some researchers are now trying the opposite approach. Instead of changing the model, change the incentives around it.

The idea is called reverse game theory, or mechanism design. It starts with the behavior you want and works backward to rules that make that behavior the rational choice. The textbook example is splitting a cake: one person cuts, the other chooses first, so the cutter has an incentive to be fair. In a recent preprint, MIT economist Stephen Morris and colleagues from Yale and Columbia proposed AI versions. One rewards a model for how well its answers predict a second model's answers, which makes deliberately underperforming on a test a losing strategy.

Princeton computer scientist Elad Hazan has a different design: agents take turns as solvers and auditors. Solvers that cheat are punished, but so are auditors that fail to catch them, and disagreements go to a human expert. The obvious risk is collusion; Hazan argues that paying auditors well enough can make looking the other way a bad deal. None of this requires a perfect view inside the model, which is the appeal. Whether incentives can hold at superhuman capability is still the open question.