Self-Tuning Prompts: Adapting Karpathy's Autoresearch Loop to Writing Quality

Tue, 25 Aug 2026 00:00:00 +0000

Andrej Karpathy recently showed that an AI can improve itself in a tight, unattended loop: propose a change, test it, keep it if it helps, throw it away if it doesn’t. That works beautifully when there’s a fast, objective number to check the result against. It’s a much harder question when the thing you’re trying to improve is writing quality, which nobody has ever reduced to a single trustworthy number. We tried it anyway, on the written instructions — the “prompts” — that steer two AI writing assistants, and the results are a useful lesson in where this kind of automation belongs, and where it doesn’t.

The original idea: a loop with no human in it #

Karpathy’s project lets an AI rewrite its own training code, run a short training job, and check the result against a single number that measures how well the resulting model performs — lower is strictly better, and it’s cheap to compute after every attempt. If the number improves, the AI keeps its change. If it doesn’t, the change is thrown away and reverted. Repeat, unattended, many times an hour. One independent write-up of a real run found the loop working through hundreds of attempts unsupervised, keeping a good number of genuine improvements along the way, and meaningfully cutting the time needed to reach a target level of quality. The whole idea rests on one condition: a fast, trustworthy number you can check the AI’s work against every single time.

Prompt-Engineering on Rishi's Blog

Self-Tuning Prompts: Adapting Karpathy's Autoresearch Loop to Writing Quality

The original idea: a loop with no human in it #