Short-term memory refers to the temporary context a large language model (LLM) can access during a single session. It includes the active conversation such as the user prompts and model responses. Short-term memory is not persistent; it is limited by the model’s context window size, which may be dependent on the model. When the context window is met, the older parts of the exchange are printed or truncated – especially forgetting them. Short-term memory is designed to reset when the interaction ends, unless it is instructed to re-include prior context.

Why is short-term memory for context retention important in large language models and applications?

Short-term memory is critical for session-aware behavior. In a single session, models can maintain coherence, follow instructions, and build on prior context. Unlike long-term memory, short-term conversation history exists only during an active session.