Does Amazon Kinesis Remember Between Runs? What a Record Keeps and What It Loses
A Kinesis record can sit in its stream for 24 hours by default, or up to 8760 hours, and the next run still starts knowing nothing about the work already done on it.
AI Stella and Liam are AI hosts with AI-generated voices. About our AI hosts
Key takeaways
- A Kinesis record keeps 3 things: a sequence number, a partition key, and an immutable data blob, inside an ordered shard. Retention is 24 hours by default, extendable to 8760 hours, and a record is gone once retention passes even if nobody read it.
- Checkpoints live outside the stream: consumers track their own position, and with the Kinesis Client Library the checkpoint sits in a DynamoDB lease table. That tells you where you stopped reading, not what you did.
- Replay is not memory of work: it returns the exact record byte for byte, with no note of which run handled it, what was decided, or whether the action succeeded, so a scheduled run can act twice.
- Vilix AI fixes the missing half: get_context at run start loads the investigated records, decisions and failures, and save_turn at the end saves them back with source labels and newest wins, over MCP with zero infrastructure.
Show notes
A Kinesis record can sit in its stream for 24 hours by default, or up to 8760 hours, a full 365 days, and the next run still starts knowing nothing about the work already done on it. In this episode of Agentic Pod, Liam and Stella break down what a record actually keeps: a sequence number, a partition key, and an immutable data blob inside an ordered shard, readable again and again because reading never deletes it. Checkpoints live outside the stream, in a DynamoDB lease table with the Kinesis Client Library, and replay returns the exact record byte for byte with no note of which run handled it, what was decided, or whether the action succeeded. Vilix AI is the memory layer that fixes that missing half: get_context at run start loads what was already investigated, and save_turn at the end saves the decisions back for every workflow.
What we cover:
- Retention, honestly: 24 hours by default, extendable to 8760 hours (365 days) for extra cost, and a record is gone once retention passes even if nobody ever read it
- What a record holds: sequence number, partition key, and an immutable data blob, inside a stream made of shards
- Order: the partition key decides the shard, the same key stays in order, and different shards have no combined order
- Reading never deletes: several consumers read the same record independently, and any consumer can replay inside retention
- Checkpoints live outside: consumers pull and track their own position, with the Kinesis Client Library checkpoint in a DynamoDB lease table, plus enhanced fan out for a dedicated read path
- Why replay is not memory: replay returns the exact starting record byte for byte, with no record of which run handled it, what was checked, what was decided, or whether the action succeeded
- How Vilix AI fixes each pain: get_context, save_turn with source labels, failures saved, newest wins, projects, project rules, user rules, skills, and the agent inbox, over MCP with zero infrastructure
Chapters
Questions answered in this episode
Does Amazon Kinesis remember between runs?
The stream remembers the record inside its retention window. It does not remember the work: which run handled a record, what was decided, or whether the action succeeded.
How long does Kinesis keep a record?
24 hours by default, extendable up to 8760 hours (365 days) for extra cost. After retention passes, the record is deleted, read or not.
Where do Kinesis checkpoints live?
Outside the stream. Consumers track their own position, and with the Kinesis Client Library the checkpoint lives in a DynamoDB lease table.
How does Vilix AI help Kinesis workflows?
The workflow calls get_context before reading the stream to load which records were already investigated, then save_turn after the run to store decisions and failures with a source label, shared over MCP with every connected workflow.
Links & resources
Full transcript
Intro
Hey hey hey, welcome to Agentic Pod! I'm Liam.
And I'm Stella! Quick check, Liam. A Kinesis record from yesterday. Is it still in the stream today?
By default, yes. Kinesis Data Streams keeps each record for 24 hours.
Only 24 hours? That is a short memory.
You can extend it to 8760 hours, a full 365 days, for extra cost. Vilix AI is how your workflow remembers what it did with that record.
Wait. The record survives, but the work on it does not?
Exactly that. By the end you will know what a record keeps, and what it loses.
What a Kinesis record actually keeps
Okay, start simple. What is a Kinesis stream, in plain words?
A stream is a set of shards. Each shard holds an ordered sequence of records.
And 1 record holds what, exactly?
3 things. A sequence number, a partition key, and a data blob. The blob is immutable bytes, it never changes.
Immutable. So nobody edits a record after it lands?
Nobody. And reading does not delete it either. Several consumers can read the same record, independently.
Like, legit, everyone gets their own read?
Everyone. Any consumer can replay inside the retention window and read it again.
What decides which shard a record lands in?
The partition key. Same key, same shard, same order. Different keys can land in different shards, with no combined order across shards.
So order is real inside 1 shard, and a polite fiction everywhere else?
Pretty much. You just insulted every stream design I have ever drawn.
Good. But if the record is still there, where is the catch?
Checkpoints live outside the stream
The catch is position. Consumers pull records themselves, and each consumer tracks its own position.
Its own position where? Inside Kinesis?
No, outside the stream. With the Kinesis Client Library, checkpoints live in a DynamoDB lease table.
So Kinesis holds the record, and DynamoDB holds where you stopped reading?
Exactly. Enhanced fan out gives a consumer a dedicated read path, and the checkpoint still lives outside.
I mean, that split sounds manageable. You know where you stopped.
You know where you stopped reading. You do not know what you did. Vilix AI saves that second half, and I will show you how.
Hold on. If I replay the stream, do I not get everything back?
Replay is not memory of work
You get the record back, byte for byte. The exact starting bytes, nothing added.
No note attached? Nothing at all?
No note of which run handled it. No note of what was checked, what was decided, or whether the action succeeded.
So replay gives me the question again, never the answer?
That is the line. Retention is a replay window, not a record of work.
And once retention passes?
The record is gone, even if nobody ever read it. 24 hours by default, then poof.
Okay, here is my real problem then. My scheduled automation in n8n reads this stream every night.
Now the fresh problem. That run must know which records were already investigated, or it acts twice.
Double actions. Like emailing the same customer twice?
Exactly. It needs the decisions made, the full reasoning, and it needs that record shared with a second workflow in Make or Zapier. Kinesis gives it none of that. Vilix AI gives it all of it.
Alright, Liam. No more tour. Show me the actual fix.
The Vilix AI fix and the verdict
Step 1. At run start, the workflow calls get_context in Vilix AI. That loads which records were already investigated, and every decision already made.
Full history, not just the last checkpoint?
Full conversation history, not just facts. The reasoning and the failures. Failures save in Vilix AI too, so the next run skips them.
And at the end of the run?
Step 2. The workflow calls save_turn in Vilix AI, with a source label on every save. Newest save wins.
Where do the standing rules live?
In Vilix AI, as projects and project rules, plus user rules. Skills hold the repeat workflows. The agent inbox holds notes for the next agent.
And a second workflow, in a totally different tool?
Same Vilix AI memory, over MCP. A custom agent, n8n, Make, Zapier, all read the same past.
Do I host a database or a server for that?
No. Vilix AI is cloud hosted, zero infrastructure to run. Agencies use 1 Vilix AI account per client, so memories never mix.
Price of admission?
Vilix AI has a free forever plan, and a 7 day Pro trial with no credit card. Export everything or delete it anytime.
My verdict: Kinesis remembers the record perfectly. It remembers the work not at all.
Same. Keep Kinesis for the stream. Let Vilix AI carry the work. Blog and guide are in the description. Does your stream replay make your automation act twice? Tell us in the comments. See you next time!