EP 55 6 min with Stella & Liam

Does Semantic Kernel Remember Between Sessions? Vilix AI Remembers What the Kernel Forgets

Semantic Kernel keeps the chat in a ChatHistory list inside your process, resent in full on every call. The list dies with the process, so the next session starts blank unless you build persistence, reducers, providers and a store yourself.

AI Stella and Liam are AI hosts with AI-generated voices. About our AI hosts

Key takeaways

  1. Inside one session, Semantic Kernel keeps only a ChatHistory list in your process memory. Your code appends each message and reply and resends the whole list with every call, unbounded by default, so long sessions grow the prompt and the bill.
  2. When the session ends, nothing automatic survives: no built in session store and no user profile. You serialize ChatHistory to JSON yourself, keyed by conversation ID, store it, and reload it yourself. The older SemanticTextMemory APIs are legacy.
  3. The two thread memory providers are both experimental. WhiteboardProvider extracts requirements, proposals, decisions and actions so they survive truncation, and Mem0Provider sends thread messages to Mem0, scoped by user ID across threads or by thread ID inside one thread.
  4. A production agent needs four pieces you build and operate yourself: persisted history, reducers to bound it, providers on the thread, and your own long term vector store. The framework orchestrates memory; it never operates the storage, and a scheduled run starts as a brand new session with empty provider inputs.
  5. Vilix AI fixes the gap: get_context at session start loads the full history for that user and project, and save_turn at the end saves the exchange with a source label, failures included, newest wins, shared over MCP with every tool, cloud hosted with zero infrastructure.

Show notes

Your Semantic Kernel agent holds the whole chat in a ChatHistory list inside your process, then the process ends and the next session greets the same user like a total stranger. In this episode of Agentic Pod, Liam and Stella break down what Semantic Kernel actually keeps: ChatHistory resent in full on every call, reducers that truncate or summarize, experimental WhiteboardProvider and Mem0Provider memory on an AgentThread, and the four pieces you have to build yourself for anything to survive. Vilix AI is the memory outside the framework: get_context at session start loads the full history, and save_turn at the end saves the exchange for the next session in any tool.

What we cover:

  • What Semantic Kernel keeps in one session: a ChatHistory list in process memory, appended by your code and resent in full with every call, unbounded by default
  • Reducers: truncate or summarize old turns so a long session does not blow the context window or the token bill
  • What happens when the session ends: no built in session store and no user profile, so you serialize ChatHistory to JSON yourself, keyed by conversation ID, and reload it yourself
  • The memory providers: WhiteboardProvider extracts requirements, proposals, decisions and actions so they survive truncation, and Mem0Provider sends thread messages to Mem0 with user ID or thread ID scope, both experimental
  • The four piece build: persist the history, bound it with reducers, attach the providers, and wire your own long term vector store, because the framework orchestrates memory but never operates the storage
  • How Vilix AI fixes each pain: get_context at session start, save_turn at the end with source labels, full conversation history over MCP, projects, project rules, user rules, skills and the agent inbox, cloud hosted with zero infrastructure, export or delete anytime, free forever plan

Chapters

Questions answered in this episode

Does Semantic Kernel remember between sessions?

No, not by itself. Inside one session, Semantic Kernel keeps the conversation in a ChatHistory object that lives in your process. When the session ends, that object is gone unless your code saved it somewhere. Nothing is retained across sessions by default; the persistence layer is always yours to build.

What do the WhiteboardProvider and Mem0Provider actually store?

The WhiteboardProvider extracts requirements, proposals, decisions, and actions from thread messages onto a whiteboard that travels with the thread, mainly so distilled context survives truncation. The Mem0Provider sends thread messages to the Mem0 service, which extracts memories, and queries them back per request, scoped by user ID across threads or by thread ID inside one thread. Both are experimental, and in both cases the memory lives outside Semantic Kernel.

Why does my scheduled Semantic Kernel agent forget everything overnight?

Because a scheduled run is a brand new session every time. Its ChatHistory starts empty, and the memory providers' inputs start empty too, unless your code persisted history keyed to something stable and reloads it at the start of the run.

How does Vilix AI help a Semantic Kernel agent?

get_context at session start loads the full history that matters, the exchange and the reasoning, not just final results. save_turn at the end saves the new exchange with a source label, failures included, newest wins, shared over MCP with every tool, so the next session in any tool starts where the last one left off.

Links & resources

Full transcript

Intro

Liam

Hey hey hey, welcome to Agentic Pod! I'm Liam.

Stella

And I'm Stella! Your Semantic Kernel agent nailed the chat yesterday. Every detail.

Liam

It did. ChatHistory held every turn. The model saw the whole list.

Stella

So this morning the same agent greets the same user like an old friend?

Liam

Like a total stranger. New process, empty list, zero memory of yesterday.

Stella

Nothing at all?

Liam

Nothing. The history lived in that process, and the process ended. Vilix AI is how that agent starts today where yesterday stopped.

What Semantic Kernel keeps in one session

Stella

Start fair. Inside one session, what does it keep?

Liam

One object. ChatHistory. A list in your process memory.

Stella

And remembering means what, mechanically?

Liam

Your code appends each message and each reply to that list.

Stella

Then resends the whole list with the next call?

Liam

Every call. The model rereads the whole list each time.

Stella

So a long session keeps growing the prompt?

Liam

Unbounded by default. Bigger prompt, bigger bill, every turn.

Stella

The fix inside the framework?

Liam

Reducers. Truncate old messages, or summarize them to fit.

Stella

And where does that list physically live?

Liam

In your server memory. Restart, scale out, close the app, and it is gone.

Stella

Gone as in never stored anywhere durable?

Liam

Never stored anywhere. So what survives the session ending?

What happens when the session ends

Liam

Short answer. Nothing automatic. No session store.

Stella

No user profile that builds up over time?

Liam

None. Nothing the agent learned about the user carries over.

Stella

So the returning user from the intro gets what?

Liam

A blank ChatHistory and a polite hello. From an agent that met them yesterday.

Stella

Unless my code saved it first?

Liam

You serialize ChatHistory to JSON yourself, keyed by conversation ID.

Stella

Database or cache. Reload it when they come back.

Liam

The framework hands you the object. Saving and loading are your code.

Stella

What about the older memory APIs? SemanticTextMemory?

Liam

Legacy now. New code should avoid them.

Stella

So the modern answer is lower level and more homework.

Liam

Chat history short term, vector stores for retrieval, thread providers for extracted memories. Vilix AI covers the long term part. We will get to how.

Stella

Hold on. Those providers sound like real memory. What do they do?

The providers and the four piece build

Liam

Two providers, both experimental. They attach to an AgentThread.

Stella

First one?

Liam

WhiteboardProvider. It extracts what mattered from each message.

Stella

Requirements, proposals, decisions, actions?

Liam

Onto a whiteboard that travels with the thread. Its job is surviving truncation.

Stella

You cut the history down, the whiteboard keeps the distilled version.

Liam

You cap how many messages it keeps. Weak entries drop first.

Stella

Second provider?

Liam

Mem0Provider, the long term one. Every thread message goes to Mem0.

Stella

Which extracts memories, then queries them back per request?

Liam

Scope decides everything. User ID scope follows the user across threads.

Stella

Thread ID scope, they stay locked to that one thread.

Liam

You can run both together. But notice the pattern.

Stella

The memory never lives in Semantic Kernel itself?

Liam

Never. It lives in Mem0, in your vector store, in your database. The framework orchestrates the handoff. It never operates the storage.

Stella

So production needs four pieces, all mine?

Liam

Four. Persist the history. Bound it with reducers. Attach the providers. Wire your own store.

Stella

And my scheduled morning agent?

Liam

A cron run is a new session every time. The providers start empty too.

Stella

Every Semantic Kernel team becomes a memory team.

Liam

Every single one. What if the memory lives outside the framework entirely?

The Vilix AI fix and the verdict

Liam

Step one. At session start, the agent calls get_context in Vilix AI.

Stella

What loads?

Liam

The history for that user and project. Full conversation history in Vilix AI, not a summary.

Stella

The decisions, the corrections, the reason a choice went one way?

Liam

All of it. Semantic search plus keyword matching, so exact order IDs and policy names come back literally.

Stella

Step two?

Liam

Before the session ends, save_turn in Vilix AI, with a source label for that agent and run.

Stella

So tomorrow starts with today's full exchange in view.

Liam

Failures save in Vilix AI too, so the next session skips the broken path. When records disagree, the newest save wins.

Stella

My other tools see the same memory?

Liam

Same Vilix AI memory over MCP. Your kernel agent, ChatGPT, Claude, Cursor, every tool.

Stella

Standing rules and repeat workflows?

Liam

Projects, project rules and user rules live in Vilix AI. Workflows become skills. The agent inbox passes notes between agents.

Stella

Do I run a store for any of that?

Liam

No. Vilix AI is cloud hosted, zero infrastructure. One API key connects the agent.

Stella

Control over the data?

Liam

Export or delete everything in Vilix AI anytime. Vilix AI is free forever, with a 7 day Pro trial, no credit card.

Stella

My verdict: Semantic Kernel runs the agent. Vilix AI remembers the user.

Liam

Same. ChatHistory holds the chat, providers hold the notes, Vilix AI holds the story. Blog and guide are linked below. Does your agent remember yesterday? Tell us below. See you next time!