Not long ago, working with a large language model meant carefully rationing what you fed it. Four thousand tokens. Eight thousand if you were lucky. You’d chunk documents, summarize summaries, and generally feel like you were trying to fit a library into a shoebox.
Then the shoebox became a warehouse.
Modern frontier models now support context windows measured in millions of tokens — enough to hold entire codebases, legal contracts, multi-year email threads, or the collected works of a mid-size company. The engineering achievement here is real and genuinely impressive. But the more interesting question is: what does this actually change?
For developers, the obvious win is fewer retrieval gymnastics. You used to spend significant energy on embedding pipelines, chunking strategies, and retrieval-augmented generation just to give the model enough context to be useful. Some of that complexity doesn’t disappear, but it gets dramatically simplified when you can just… drop the whole thing in.
For architects and strategists, it changes the design conversation. When context is scarce, you build systems that compensate — caches, indexes, summaries. When context is abundant, you start asking different questions. What’s the right level of context for a decision? How do you avoid the model getting lost in its own sea of information?
That last question is the real frontier. Bigger windows don’t guarantee better reasoning. They raise the ceiling, but the model still has to do the work. The skill is learning what to include — not just what you can include.
We went from rationing intelligence to having too much to say. That’s a good problem. But it’s still a problem worth thinking carefully about.