AI

The original idea: a loop with no human in it #

Karpathy’s project lets an AI rewrite its own training code, run a short training job, and check the result against a single number that measures how well the resulting model performs — lower is strictly better, and it’s cheap to compute after every attempt. If the number improves, the AI keeps its change. If it doesn’t, the change is thrown away and reverted. Repeat, unattended, many times an hour. One independent write-up of a real run found the loop working through hundreds of attempts unsupervised, keeping a good number of genuine improvements along the way, and meaningfully cutting the time needed to reach a target level of quality. The whole idea rests on one condition: a fast, trustworthy number you can check the AI’s work against every single time.

A conservative estimate puts a typical Fortune 500 company’s annual AI cost above $30 million. One healthcare enterprise consumed a trillion tokens over six months before its finance team understood what was driving the charges. Uber exhausted its full-year AI budget by April. These aren’t edge cases — they’re the leading edge of a structural shift that most enterprise finance teams haven’t priced in yet.

Karpathy’s own framing was quieter: “I think the next few years at the frontier of LLMs will be especially formative… excited to join the team here and get back to R&D.” He would lead a new pretraining-research group under Nick Joseph, Anthropic’s head of pretraining, built around a premise with a slight vertigo to it: use Claude to help build the next Claude.

Agents operating on codebases run into the same wall: context collapses after a milestone. Teams rely on ephemeral files like plan.md for in-flight work, but once the sprint ends, that memory is buried or lost. A compiled wiki solves this by forcing a durable synthesis of decisions, facts, and architecture at gate intervals. The result is a structured, typed knowledge base that reduces query-time token costs and prevents agents from re-deriving knowledge on every turn. Here is how to build the pattern, where it breaks at scale, and the infrastructure tradeoffs that keep it honest.

A handful of consumer GPUs now sit at the center of serious homelab inference. The interesting question is no longer “can I run a 30B-class model” but “which card should I buy for the workload I actually have.” Below is a method-grounded look at three tiers — the RTX 3060 12GB, RTX 4090 24GB, and RTX 5090 32GB — running today’s small-active-parameter mixture-of-experts models: Qwen3.5-35B-A3B and its newer sibling Qwen3.6-35B-A3B, Gemma 4 26B-A4B, and the dense Gemma 4 31B.

Microsoft is making its AI revenue model explicit in 2026: bundle Copilot into a new top-tier SKU called E7 at $99 per user per month, then bill every agent action separately through Copilot Credits. Together these moves push enterprise Microsoft 365 spending from a flat per-seat line into a hybrid model where the headline seat price covers the license and the metered consumption covers the work. For IT leaders staring at renewal cycles, this changes how the contract needs to be modeled — and where the overage risk actually lives.