Self-Tuning Prompts: Adapting Karpathy's Autoresearch Loop to Writing Quality

The original idea: a loop with no human in it #

Karpathy’s project lets an AI rewrite its own training code, run a short training job, and check the result against a single number that measures how well the resulting model performs — lower is strictly better, and it’s cheap to compute after every attempt. If the number improves, the AI keeps its change. If it doesn’t, the change is thrown away and reverted. Repeat, unattended, many times an hour. One independent write-up of a real run found the loop working through hundreds of attempts unsupervised, keeping a good number of genuine improvements along the way, and meaningfully cutting the time needed to reach a target level of quality. The whole idea rests on one condition: a fast, trustworthy number you can check the AI’s work against every single time.

Why “is this well written” breaks the recipe #

Writing doesn’t have that number. There’s no equivalent for “is this article good” — no cheap, objective score you can compute in seconds and trust to go down every time the writing actually gets better. Karpathy’s loop works because the metric and the goal are the same thing. For writing, any number you pick is at best a proxy: a stand-in you hope correlates with quality, knowing full well it isn’t quality itself. That gap between the proxy and the real thing turns out to matter a lot in practice, as the next two stories show.

The stand-ins we picked #

We applied the same shape — propose an edit, test it, keep or discard — to the instructions given to two AI assistants working together: one that researches a topic before an article gets written, and one that writes the article itself. Rather than chase “good writing” directly, we picked narrow, checkable stand-ins. For the research assistant, the stand-in was simple: does every substantive claim it makes come with a source attached? For the writing assistant, it was: does the finished draft land close to its intended length? Neither one claims to capture whether the writing is actually good. Both are just concrete enough that an unattended loop can optimize them without a human reading every single draft along the way.

A clean win: citations that actually stuck #

The research assistant’s tuning loop played out the way the original idea promises. The first test was a basic model choice: given the exact same instructions, a larger underlying AI model cited its sources correctly most of the time across two test topics — roughly two-thirds on one, five-sixths on the other — while a smaller, cheaper model cited none of them at all, on either topic. That decision made, the real tuning began — testing the instructions against ten real topics, most of them used to score every round, a couple held back and checked only once at the very end. The starting point cited sources correctly a little over 80% of the time. One round of edits pushed that close to 96%. A tempting-looking fix came next: it closed a gap on the topics held back for final testing, but it made things slightly worse everywhere else — so it was correctly thrown out, exactly the kind of edit a system without guardrails would have mistakenly kept. A further attempt found a fix general enough to solve both problems at once. The final version got every single claim right — on the topics it had practiced on and the ones it had never seen.

The honest complication: great on practice, worse on the real test #

The writing assistant’s story is more instructive precisely because it didn’t end as cleanly. It had a known problem: on two real past drafts, it had overshot its intended length by more than a third in one case and by nearly double in the other. Tuned against eight practice topics, several rounds of propose-test-keep-or-discard drove its average length error down from roughly 30% off-target to about 7% — a large, real improvement, with a couple of tempting edits discarded along the way for accidentally breaking an unrelated formatting rule. But when that same tuned version finally ran against two topics it had never practiced on, its average error jumped back up to around 17% — more than double what it scored on its own practice material — and on one of those two new topics it missed its length target badly enough to fail outright. We checked for the obvious explanation, that it had somehow memorized details of the practice topics instead of learning a general skill, and ruled it out; nothing in the final wording referenced them directly. The likelier explanation is subtler: instructions tuned against eight closely related examples pick up habits specific to that small set rather than a rule that actually generalizes. We shipped it anyway, flagging the gap rather than hiding it, because the improvement was too large and consistent to ignore and every draft still cleared its basic quality checks.

The takeaway: automate the boring parts, not the judgment #

Put the two stories side by side and the lesson isn’t really about how the loop works — it’s about where automation belongs. Checking for citations and checking whether a draft is the right length are genuinely boring, mechanical tasks, and automating them saves real, repeated human effort. But neither stand-in measures whether the writing is actually good, and a result that looks great on the material you practiced on (7% average error) can still hide a real weak spot on something new (17%) that only a person reading the draft would catch. The fix wasn’t to distrust the automation — it was to keep one mandatory step where a person reads at least one full finished draft before trusting any tuned result. That’s the real adaptation this idea needed to survive contact with something as subjective as writing: let the unattended loop handle the parts that have a clean number, and keep a person in charge of the part that no number measures yet.