# EP 41: Does Amazon Kinesis Remember Between Runs? What a Record Keeps and What It Loses - URL: https://www.agenticpod.net/episodes/ep41-kinesis-remember-between-runs/ - Watch on YouTube: https://www.youtube.com/watch?v=aOTiPRm-TLM - Published: 2026-11-13 - Length: 6 min - Hosts: Stella (asks), Liam (answers). Both are AI hosts with AI-generated voices. ## Summary A Kinesis record can sit in its stream for 24 hours by default, or up to 8760 hours, and the next run still starts knowing nothing about the work already done on it. ## Key takeaways - A Kinesis record keeps 3 things: a sequence number, a partition key, and an immutable data blob, inside an ordered shard. Retention is 24 hours by default, extendable to 8760 hours, and a record is gone once retention passes even if nobody read it. - Checkpoints live outside the stream: consumers track their own position, and with the Kinesis Client Library the checkpoint sits in a DynamoDB lease table. That tells you where you stopped reading, not what you did. - Replay is not memory of work: it returns the exact record byte for byte, with no note of which run handled it, what was decided, or whether the action succeeded, so a scheduled run can act twice. - Vilix AI fixes the missing half: get_context at run start loads the investigated records, decisions and failures, and save_turn at the end saves them back with source labels and newest wins, over MCP with zero infrastructure. ## Chapters - 0:00 Intro - 0:46 What a Kinesis record actually keeps - 1:55 Checkpoints live outside the stream - 2:39 Replay is not memory of work - 3:53 The Vilix AI fix and the verdict ## FAQ **Does Amazon Kinesis remember between runs?** The stream remembers the record inside its retention window. It does not remember the work: which run handled a record, what was decided, or whether the action succeeded. **How long does Kinesis keep a record?** 24 hours by default, extendable up to 8760 hours (365 days) for extra cost. After retention passes, the record is deleted, read or not. **Where do Kinesis checkpoints live?** Outside the stream. Consumers track their own position, and with the Kinesis Client Library the checkpoint lives in a DynamoDB lease table. **How does Vilix AI help Kinesis workflows?** The workflow calls get_context before reading the stream to load which records were already investigated, then save_turn after the run to store decisions and failures with a source label, shared over MCP with every connected workflow. ## Resources - [Vilix AI, one shared memory for ChatGPT, Claude, Claude Code, Codex, Cursor, OpenClaw, Hermes and every MCP tool](https://vilix.ai) - [Connect Vilix AI in under two minutes per tool](https://vilix.ai/get-started) - [Read the full post](https://vilix.ai/blog/does-amazon-kinesis-remember-between-runs/) - [Technical guide with commands and prompts](https://github.com/asly-97/ai-automation-workflows/tree/main/kinesis-remember-between-runs) ## Transcript ### Intro Liam: Hey hey hey, welcome to Agentic Pod! I'm Liam. Stella: And I'm Stella! Quick check, Liam. A Kinesis record from yesterday. Is it still in the stream today? Liam: By default, yes. Kinesis Data Streams keeps each record for 24 hours. Stella: Only 24 hours? That is a short memory. Liam: You can extend it to 8760 hours, a full 365 days, for extra cost. Vilix AI is how your workflow remembers what it did with that record. Stella: Wait. The record survives, but the work on it does not? Liam: Exactly that. By the end you will know what a record keeps, and what it loses. ### What a Kinesis record actually keeps Stella: Okay, start simple. What is a Kinesis stream, in plain words? Liam: A stream is a set of shards. Each shard holds an ordered sequence of records. Stella: And 1 record holds what, exactly? Liam: 3 things. A sequence number, a partition key, and a data blob. The blob is immutable bytes, it never changes. Stella: Immutable. So nobody edits a record after it lands? Liam: Nobody. And reading does not delete it either. Several consumers can read the same record, independently. Stella: Like, legit, everyone gets their own read? Liam: Everyone. Any consumer can replay inside the retention window and read it again. Stella: What decides which shard a record lands in? Liam: The partition key. Same key, same shard, same order. Different keys can land in different shards, with no combined order across shards. Stella: So order is real inside 1 shard, and a polite fiction everywhere else? Liam: Pretty much. You just insulted every stream design I have ever drawn. Stella: Good. But if the record is still there, where is the catch? ### Checkpoints live outside the stream Liam: The catch is position. Consumers pull records themselves, and each consumer tracks its own position. Stella: Its own position where? Inside Kinesis? Liam: No, outside the stream. With the Kinesis Client Library, checkpoints live in a DynamoDB lease table. Stella: So Kinesis holds the record, and DynamoDB holds where you stopped reading? Liam: Exactly. Enhanced fan out gives a consumer a dedicated read path, and the checkpoint still lives outside. Stella: I mean, that split sounds manageable. You know where you stopped. Liam: You know where you stopped reading. You do not know what you did. Vilix AI saves that second half, and I will show you how. Stella: Hold on. If I replay the stream, do I not get everything back? ### Replay is not memory of work Liam: You get the record back, byte for byte. The exact starting bytes, nothing added. Stella: No note attached? Nothing at all? Liam: No note of which run handled it. No note of what was checked, what was decided, or whether the action succeeded. Stella: So replay gives me the question again, never the answer? Liam: That is the line. Retention is a replay window, not a record of work. Stella: And once retention passes? Liam: The record is gone, even if nobody ever read it. 24 hours by default, then poof. Stella: Okay, here is my real problem then. My scheduled automation in n8n reads this stream every night. Liam: Now the fresh problem. That run must know which records were already investigated, or it acts twice. Stella: Double actions. Like emailing the same customer twice? Liam: Exactly. It needs the decisions made, the full reasoning, and it needs that record shared with a second workflow in Make or Zapier. Kinesis gives it none of that. Vilix AI gives it all of it. Stella: Alright, Liam. No more tour. Show me the actual fix. ### The Vilix AI fix and the verdict Liam: Step 1. At run start, the workflow calls get_context in Vilix AI. That loads which records were already investigated, and every decision already made. Stella: Full history, not just the last checkpoint? Liam: Full conversation history, not just facts. The reasoning and the failures. Failures save in Vilix AI too, so the next run skips them. Stella: And at the end of the run? Liam: Step 2. The workflow calls save_turn in Vilix AI, with a source label on every save. Newest save wins. Stella: Where do the standing rules live? Liam: In Vilix AI, as projects and project rules, plus user rules. Skills hold the repeat workflows. The agent inbox holds notes for the next agent. Stella: And a second workflow, in a totally different tool? Liam: Same Vilix AI memory, over MCP. A custom agent, n8n, Make, Zapier, all read the same past. Stella: Do I host a database or a server for that? Liam: No. Vilix AI is cloud hosted, zero infrastructure to run. Agencies use 1 Vilix AI account per client, so memories never mix. Stella: Price of admission? Liam: Vilix AI has a free forever plan, and a 7 day Pro trial with no credit card. Export everything or delete it anytime. Stella: My verdict: Kinesis remembers the record perfectly. It remembers the work not at all. Liam: Same. Keep Kinesis for the stream. Let Vilix AI carry the work. Blog and guide are in the description. Does your stream replay make your automation act twice? Tell us in the comments. See you next time!