Why does your AI character forget names after twenty messages? This article explains LLM context length in plain words, compares free models by memory size, shows which settings to change, and gives practical roleplay habits that keep long stories consistent.
Your character just called you by the wrong name. The dragon you killed three scenes ago is suddenly alive again, and the tavern you burned down is serving ale. Nothing is broken, and the model is not being lazy. It simply ran out of room. Every language model reads and writes inside a fixed window of memory, and once your chat outgrows that window, the oldest material quietly falls out of view.
This article explains LLM context length in plain words: what it measures, how big it is on popular free models, which settings change how much the model remembers, and how to keep a long roleplay consistent without paying for a giant window you may not need.
💡 Quick answer: context length is the total number of tokens a model can look at in one request. That total includes your instructions, the chat history, and the reply the model is about to write. When the total goes over the limit, something gets cut.
Think of the context window as the desk a model works on. Everything it can see has to fit on that desk at the same moment. Anything that does not fit is gone, as far as the model is concerned. It has no hidden filing cabinet, and it does not carry memories from one request to the next. Each reply is generated fresh from whatever text you send along with it.
Models do not count words. They count tokens, which are chunks of text. In English, one token is roughly four characters, or about three quarters of a word. Short common words are often one token. Rare names, slang, and words in other languages can take three or four tokens each.
| Context size | Approx. English words | Roughly equals |
|---|---|---|
| 4K (4,096 tokens) | 3,000 | A long blog post |
| 8K | 6,000 | A short story |
| 32K | 24,000 | A novella chapter set |
| 128K | 96,000 | A full novel |
| 1M | 750,000 | A shelf of books |
These are estimates. Code, emoji, and non-English text eat tokens faster, so a chat in Japanese or Arabic fills the same window sooner than a chat in English.
Beginners assume the window holds only the chat. It holds much more:
That last item surprises people. If your window is 8K and you reserve 1,000 tokens for the answer, you really have 7K for everything else.
"Free" means different things. It can mean open weights you run on your own computer, a free tier with daily limits, or a model that costs very little per request. In every case, the number printed on the model page is the maximum, not a promise about quality.
Older open models are the classic example. Llama 2 7B Chat and the other Llama 2 sizes, such as Llama 2 13B Chat and Llama 2 70B Chat, work with about 4K tokens. That is fine for a quick question and painful for a story. Meta Llama 3 8B Instruct doubles that to roughly 8K, and Mistral 7B v0.1 sits in the same neighbourhood.
Small windows still have a place. They answer fast, they are cheap to run, and they force you to keep prompts tight, which is a good habit anyway.
The newer free and low-cost families are far more generous. Granite 4.0 H Small is built specifically for long inputs, and Granite 4.1 8B is a free option for chat and code. If your work is code, Granite 8B Code Instruct 128K carries its window size in the name, and its sibling Granite 20B Code Instruct 8K shows what the older generation offered.
Other families worth testing for long chats include DeepSeek V3, Kimi K2.6, and Qwen3.7 Plus. Check the model page for the exact window on each, since vendors change these numbers between versions.
Llama 4 Scout Instruct is advertised with a window of up to 10 million tokens, and Llama 4 Maverick Instruct is advertised at around 1 million. Google's Gemini 3.5 Flash and Gemini 3.1 Pro also sit in the long-window camp.
Here is the catch. By late 2026, windows of 1 million tokens are common, and the median across tracked models is reported around 256K. But the advertised window and the effective window are different things. Independent testing keeps finding that accuracy drops well before the maximum, especially when the one detail you need is buried in the middle of a huge pile of text. A giant window is a big library, not a better reader.
💡 Rule of thumb: treat roughly half of the advertised window as the safe working zone. Past that point, summarize or retrieve instead of pushing more raw text in.
Most people notice context limits only when something odd happens. There are two main failure patterns, and they look different.
When the chat gets too long, most apps drop the oldest messages first. Your opening scene, the character card (in some apps), and the original rules can vanish without a warning. The model then improvises. That is why a character suddenly changes personality: the text that defined them is no longer on the desk.
Signs of truncation:
The second pattern happens even when everything technically fits. Models pay the most attention to the start and the end of the window. Facts in the middle get weaker attention. If you stuff 80,000 tokens into a chat and bury the important clue at token 40,000, the model may miss it.
The fix is simple: put the most important information at the very beginning (system prompt, character card) or restate it near the end, close to your latest message.
Chat apps and API tools expose many sliders. Only a handful change memory behaviour in a way you will notice.
Many local and self-hosted setups ask you to choose a context size yourself, and the default is often small, such as 2K or 4K, even when the model supports far more. If your character forgets things quickly, check this first. Raising it increases memory use, so there is a limit set by your hardware. On hosted models, the vendor usually sets it for you.
This number is the largest reply the model may write. Set it too low and replies end mid-sentence. Set it too high and you waste window space reserved for text you will never see. For roleplay, 250 to 500 tokens suits most scenes. For long-form writing, go to 1,000 or more.
These two do not change how much the model remembers, but they change how well it uses what it remembers.
| Setting | Low value | High value | Good starting point for roleplay |
|---|---|---|---|
| Temperature | Predictable, safe | Creative, risky | 0.7 to 0.9 |
| Top P | Narrow word choices | Wider variety | 0.9 |
| Repetition penalty | Allows loops | Avoids repeats, can derail | 1.05 to 1.15 |
| Max output tokens | Short replies | Long replies | 300 to 500 |
💡 Watch the penalty. A repetition penalty that is too strong pushes the model away from words it has already used, including your character's own name. If names start to mutate, lower it.
Roleplay is the hardest test of context length, because the story is both long and detail-heavy. You can write for hours on a modest window if you manage what sits on the desk.
A character card is paid for in tokens on every single request. A 2,000-token card on an 8K window leaves a quarter of your memory permanently occupied. Aim for 300 to 700 tokens: name, age, voice, three traits, one goal, one fear. Cut backstory the model does not need yet.
A tight card might read like this: "Mara, 34, ship's navigator. Dry humor, speaks in short sentences, distrusts officers. Wants to find her missing brother. Never admits when she is scared."
Here is a worked example for an 8K window:
If each exchange (your message plus the reply) averages 400 tokens, the model remembers roughly 16 exchanges. After that, the oldest turns drop. Knowing this number tells you exactly when to summarize.
This habit does more than any setting. Every 15 to 20 exchanges, write (or ask the model to write) a short summary of what happened. Paste it near the top of the chat, then delete the old messages it replaces.
A good summary includes:
Keep it under 200 tokens. A story summarized this way can run for hundreds of turns on a window that would otherwise hold only 16.
Not every message deserves a place on the desk. Greetings, repeated descriptions, and long out-of-character chatter use tokens without adding story. Many apps let you pin or lock certain messages, or store facts in a lorebook that only loads when a trigger word appears. Use these tools for names, rules, and relationships. Let casual banter fall off the end.
💡 Restate, don't hope. If a detail matters in the next scene (the stolen key, the secret, the injury), mention it again in your own message. One sentence is cheaper than a plot hole.
Bigger is not always better. A giant window costs more, runs slower, and may bury your details. Match the model to the job.
| Your goal | Window you need | Models to try |
|---|---|---|
| Quick questions, short chats | 4K to 8K | Llama 2 7B Chat, Meta Llama 3 8B Instruct |
| Everyday chat and light roleplay | 32K to 128K | Granite 4.1 8B, DeepSeek V3 |
| Long stories and lorebooks | 128K or more | Granite 4.0 H Small, Kimi K2.6 |
| Huge documents, many files | 1M or more | Llama 4 Scout Instruct, Gemini 3.5 Flash |
| Polished writing and reasoning | Large, with strong recall | Claude Sonnet 5, GPT 5.6 Luna |
Window size is one factor. Writing quality, instruction following, and speed matter just as much, so test two or three models with the same opening prompt before you commit to a long story.
Check the card first. If the card is long and sits at the very top of an overfull chat, your app may be cutting it. Shorten the card, pin it, or restate the name in your latest message. Then raise the context size if your setup allows it.
This is nearly always the max output tokens setting. Raise it to 500 or more. If it is already high, the window may be full, so there is no room left to write. Summarize older turns to free space.
Lower the repetition penalty slightly, raise the temperature a little, and trim old messages that contain the repeated phrase. Models love copying patterns they can see in their own history.
Longer context means more computing work on every reply. If speed drops, you are probably sending far more text than needed. A clean summary plus the last ten messages is often faster and sharper than the whole transcript.
You do not need to guess which model suits your story. Open the large language models collection on PicassoIA, pick two models from the table above, and paste the same opening scene into each. Watch how long each one stays consistent, then adjust the max output and temperature and run the test again.
Try a quick experiment: write a 400-token summary of your current story, start a fresh chat with it, and see whether the characters hold together better than they did in the original 100-message thread. For many people, the answer is yes. Then take the same creative spark further, and build the cover art, character portraits, and scene images for your story with the image models on Picasso IA. Start with one chat, one character, and one summary, and you will see the difference in the next scene.
Choose your language
