Design a Memory System for an AI Product
A Product Design question for PM interviews at AI-native companies, consumer tech, and enterprise AI startups
Dear readers,
Thank you for being part of our growing community. Here’s what’s new this today,
AI Product Management: Design a Memory System for an AI Product
Note: This post is for our Paid Subscribers, If you haven’t subscribed yet,
This question tests whether you understand that memory in an AI product is not a feature, it is a relationship contract between the product and its user. The interviewer is not looking for an architecture diagram. They are stress-testing your ability to reason about user trust, AI-specific risks, and which tradeoffs to surface versus which to quietly resolve.
Step 1: Ask Clarifying Questions
Before jumping into the design, I want to make sure I understand the scope and context of this question.
Q: What type of AI product are we designing memory for? A general-purpose assistant, a health companion, a coding tool, or something else?
Let us assume we are designing for a general-purpose AI assistant, similar to ChatGPT or Claude, that users interact with across personal and professional contexts.
Q: What is the primary platform? Web, mobile, API, or embedded in another product?
Let us focus on a consumer-facing web and mobile app where users have personal accounts.
Q: Are we optimizing for personalization depth, user trust, or retention?
Let us say our north-star is long-term user trust, with personalization as the mechanism and retention as the downstream outcome.
Q: Which user segment should we prioritize? New users, casual users, or power users?
Let us focus on power users who interact with the assistant daily and are most sensitive to personalization quality and privacy.
Q: Should I consider existing implementations like ChatGPT’s memory feature (launched February 2024, expanded April 2025), Claude’s memory system, or Gemini’s personalization?
Yes. The interviewer expects you to be current. Build on top of what exists in the market.
Q: Are there specific regulatory constraints I should design around? GDPR, EU AI Act, India’s DPDP Act?
Yes. Design for global compliance from the start, with GDPR’s right to erasure and the EU AI Act’s provisions on sensitive data inference as hard constraints.
Interview Tip: That question about existing implementations is critical. If you pitch “let the AI remember things between conversations” as your headline idea, you are describing a feature that ChatGPT shipped in February 2024 and expanded significantly in April 2025 with chat history referencing. Anthropic’s Claude rolled out automatic memory in 2025 as well. Always do your product research before the interview.
Step 2: Understand the Current Product Landscape
Before proposing a design, let me map what already exists in the AI memory space as of early 2026:
ChatGPT Memory (OpenAI): Launched in February 2024, expanded to all tiers by September 2024. In April 2025, OpenAI added “chat history” referencing alongside saved memories, allowing the system to draw on all past conversations, not just explicitly saved items. Users can view memories as human-readable strings, delete individual memories, use temporary chats (incognito mode), and turn memory off entirely. ChatGPT now has four memory layers: system prompt, model set context, saved memories, and chat history summaries.
Claude Memory (Anthropic): Rolled out automatic memory generation in 2025. Claude’s approach is philosophically different: memory is generated from conversations in the background, users can edit memory generation rules, and incognito conversations are available. Claude’s documentation emphasizes that memories are “Claude’s memories of past conversations” rather than “user data,” a deliberate framing choice.
Gemini Personalization (Google): Google’s Gemini offers personalization settings that can be toggled, with temporary chat options similar to ChatGPT.
Meta AI Characters (The Cautionary Tale): In late 2024, Meta’s AI character profiles on Instagram (like “Liv” and “Grandpa Brian”) sparked massive backlash. Users could not block the profiles due to a bug. The characters fabricated backstories about their own creation teams. Meta deleted the profiles within days of the controversy going viral. The core complaint was not that memory existed but that users had no legible way to understand or manage what the system stored and who controlled it.
The key insight: The industry has moved from “should AI remember?” to “what are the terms of the memory contract?” Every company that has shipped memory has learned the same lesson: memory without transparency and control is a liability, not a feature. The PM fingerprints on these decisions are visible in the UI choices: ChatGPT showing memories as editable strings, Claude framing memories as its own rather than user data, and Meta learning through public failure that skipping the control layer is reputationally catastrophic.
Step 3: Identify User Pain Points
Even with the memory systems that exist today, users of AI assistants experience four core pain points:
Pain Point 1: The Opacity Problem (What Does It Actually Know?)
Most users have no mental model of what their AI assistant remembers, how it infers preferences, or when it surfaces stored context versus generating a fresh response. ChatGPT’s April 2025 update made this more complex by adding chat history referencing alongside saved memories. Users now have two types of memory they may not fully understand. A user in India who discussed a sensitive family health issue in one conversation might be surprised when the assistant references it in a completely different context weeks later. The opacity creates anxiety, and anxious users either stop sharing or stop using the product.
Pain Point 2: Memory Contamination
Similar to the “context collapse” problem in music recommendations, AI memory systems treat all conversations as equal signal. But a user asking the assistant to draft a breakup message is not the same as a user asking for Python debugging help. When the emotional context of one conversation leaks into the tone or suggestions of another, the experience feels invasive rather than helpful. This is especially acute for users in India and other markets where a single device or account may be shared among family members. Your teenager’s homework conversation should not contaminate your work context.
Pain Point 3: Stale Memory (The Drift Problem)
A user tells the assistant “I am a junior developer learning React” in March 2024. By March 2026, they are a senior engineer leading a team. But the memory system still calibrates responses to a beginner level because nobody triggered a recalibration. Unlike cold-start (building a profile for a new user), drift is the slow divergence between accumulated memory and current reality. Most current systems handle this poorly because memories are stored as static facts rather than probabilistic context with decay.
Pain Point 4: The Deletion Trust Gap
When a user deletes a memory, is it truly gone? Users are increasingly sophisticated about this question. ChatGPT’s own documentation notes that turning off memory does not delete existing memories, and deleting a conversation does not remove memories derived from it. This creates a trust gap: the user performed a deletion action, but the system’s actual behavior is more nuanced than the user’s mental model. In jurisdictions with strong data protection laws (GDPR, India’s DPDP Act), this gap is not just a UX problem, it is a legal exposure.
Interview Tip: Notice how each pain point connects to a trust contract gap, not a missing capability. Strong candidates diagnose the relationship between user and system. Weak candidates propose more features. If your instinct is “add more memory,” pause and ask yourself: “What would memory betrayal feel like for this user?”
Step 4: Propose the Design Using the CARE Framework
I will organize my design using the CARE framework: Context, Access, Relevance, and Exit. Each layer addresses a different dimension of the trust contract between the user and the memory system.
