Conversation history refers to the sequence of user queries and model responses within a session. It is an application-level feature, not a core capability of the model itself.

Conversation history can persist temporarily, for example, during a chat session, or it can be stored more permanently by the application in a database. In many cases, the large language model (LLM) does not inherently retain previous exchanges unless it is prompted. Since the model does not have visibility into all conversations, they do not consume tokens or affect the model's context window.

Why is conversation history important in large language models and AI applications?

Conversation history is critical for maintaining continuity and context in multi-turn interactions. Without access to prior exchanges, a model would treat every prompt as an isolated interaction. This can lead to a poor user experience when outputs are disjointed or provide repetitive responses. By preserving past queries and responses, applications construct prompts that allow the model to have more human-like interactions. It should be noted that conversation history is not long-term memory.