Agentic AI Software Engineering: From Prompting and Context Engineering to Memory Architecture

<div class="cs-rating pd-rating" id="pd_rating_holder_1819065_post_3097"></div> <p class="wp-block-paragraph">Over the last couple of years, we observed that Large Language Model (LLM) usage, also known as GenAI, did not stop at prompt engineering. We moved through context engineering by providing richer prompts and structured context for better one-shot outcomes. In software engineering, we then saw spec-driven development, where context is specialized to requirements, design, constraints, and acceptance criteria, creating a practical boundary for language models to build within.</p> <p class="wp-block-paragraph">Yet even with these advances, most systems still operate in single threads of conversation, without ongoing training and without long-term awareness of global context, user persona, or organizational memory.</p> <p class="wp-block-paragraph">My view, as of early 2026, is that the next wave is memory architecture for AI systems. If AI is acting as a collaborator in long-running systems, where does its memory live, and how is it architected?</p> <p class="wp-block-paragraph">If you are thinking along the same lines, this post maps what we have seen pre-2026 and what to expect next. It frames the shift from context tricks to deliberate memory design, then lands on a practical reference stack and patterns you can implement.</p> <h2 class="wp-block-heading">1. Waves of AI in Software Engineering</h2> <p class="wp-block-paragraph">The progression has been consistent. Each wave increases leverage, and each wave increases the blast radius of mistakes.</p> <ul class="wp-block-list"> <li>Autocomplete and predictive text</li> <li>Prompt engineering</li> <li>Context engineering</li> <li>Spec-driven development</li> <li>Memory architecture</li> </ul> <h3 class="wp-block-heading">Characteristics</h3> <ul class="wp-block-list"> <li>Autocomplete improved velocity, not architecture.</li> <li>Prompt engineering improved expressiveness, not reproducibility.</li> <li>Context engineering improved grounding, not continuity.</li> <li>Spec-driven development improved alignment, not persistence.</li> <li>Memory architecture becomes the layer that reduces rework across time.</li> </ul> <h3 class="wp-block-heading">Observation</h3> <p class="is-style-info wp-block-paragraph">Most teams upgraded model capability faster than they upgraded system memory. The result is familiar: agents appear smart inside a single interaction and become expensive across a program of work.</p> <h2 class="wp-block-heading">2. What “Memory” Actually Means in LLM Systems</h2> <p class="wp-block-paragraph">“Memory” is overloaded. If the team does not disambiguate early, the solution usually defaults to longer chat history plus a vector store.</p> <h3 class="wp-block-heading">2.1 Parametric Memory</h3> <p class="wp-block-paragraph">Knowledge baked into model weights.</p> <p class="wp-block-paragraph">Strengths:</p> <ul class="wp-block-list"> <li>Broad general knowledge</li> <li>Strong priors and reasoning patterns</li> </ul> <p class="wp-block-paragraph">Weaknesses:</p> <ul class="wp-block-list"> <li>Slow to update</li> <li>Unreliable for organization-specific and time-sensitive context</li> <li>Not auditable in system terms</li> </ul> <h3 class="wp-block-heading">2.2 Ephemeral (Context-Window) Memory</h3> <p class="wp-block-paragraph">The current prompt window: chat history, retrieved docs, scratchpads, and tool outputs.</p> <p class="wp-block-paragraph">Constraint:</p> <ul class="wp-block-list"> <li>Context windows behave like RAM, not a database.</li> <li>The working set is rewritten every turn.</li> <li>Retention and eviction are often accidental, not designed.</li> </ul> <h3 class="wp-block-heading">2.3 External (Long-Term) Memory</h3> <p class="wp-block-paragraph">Stores the system owns: vector databases, document stores, SQL/NoSQL, knowledge graphs, logs, traces, and event streams.</p> <h3 class="wp-block-heading">2.4 Agent State vs System Memory</h3> <p class="wp-block-paragraph">Agent state is what a single run sees. System memory is what the organization retains.</p> <p class="wp-block-paragraph">This maps cleanly to distributed systems: state lives somewhere, ownership matters, retention matters, and observability matters.</p> <p class="wp-block-paragraph">Memory is architecture.</p> <p class="wp-block-paragraph">Call-out:</p> <p class="is-style-info wp-block-paragraph">Today, most teams say “memory” but often mean “slightly longer chat history plus a vector DB.” – a database with RAG architecture is not memory</p> <h2 class="wp-block-heading">3. Techniques That Got Us Here</h2> <p class="wp-block-paragraph">The current mainstream is valuable. It is also insufficient for long-running workflows.</p> <h3 class="wp-block-heading">3.1 Vector DB and Vanilla RAG</h3> <p class="wp-block-paragraph">How it works:</p> <ol class="wp-block-list"> <li>Chunk text</li> <li>Embed chunks</li> <li>Store embeddings</li> <li>Retrieve top-k by similarity</li> <li>Optionally rerank</li> <li>Inject into context window</li> </ol> <p class="wp-block-paragraph">Where it shines:</p> <ul class="wp-block-list"> <li>Document Q&A</li> <li>Codebase search</li> <li>Knowledge base assistants</li> <li>“Find the paragraph that says X”</li> </ul> <p class="wp-block-paragraph">Where it breaks down:</p> <ul class="wp-block-list"> <li>Multi-episode work where history matters</li> <li>Questions that depend on relationships and time</li> <li>Auditability of evidence across months</li> <li>Unbounded corpora and retrieval noise</li> </ul> <h3 class="wp-block-heading">3.2 Graph RAG and Knowledge Graphs</h3> <p class="wp-block-paragraph">Graph approaches bring explicit structure back into retrieval.</p> <p class="wp-block-paragraph">Pros:</p> <ul class="wp-block-list"> <li>Better relationship-centric queries</li> <li>Supports reasoning over links and causality</li> <li>Improves explainability via traversals and edges</li> </ul> <p class="wp-block-paragraph">Cons:</p> <ul class="wp-block-list"> <li>Modelling effort</li> <li>Governance overhead</li> <li>Drift between system reality and curated graph</li> </ul> <h3 class="wp-block-heading">3.3 Other Useful Patterns</h3> <ul class="wp-block-list"> <li>Conversation buffers and windowed chat memory</li> <li>Rolling summaries</li> <li>Hybrid retrieval (BM25 + embeddings + reranking)</li> <li>Tool-aware retrieval that prioritizes structured sources</li> </ul> <h2 class="wp-block-heading">4. Core Constraints That Shape Memory Design</h2> <p class="wp-block-paragraph">Most memory stacks fail for predictable reasons. These three constraints are reusable design lenses.</p> <h3 class="wp-block-heading">4.1 Capacity</h3> <p class="wp-block-paragraph">Memory grows faster than teams expect.</p> <ul class="wp-block-list"> <li>Every message, tool call, event, and log line is a write.</li> <li>Every write adds retrieval noise unless consolidated.</li> <li>Unbounded storage becomes unbounded spend.</li> </ul> <p class="wp-block-paragraph">Design decisions that must be explicit:</p> <ul class="wp-block-list"> <li>Raw text vs embeddings vs summaries</li> <li>Retention per tier</li> <li>Consolidation frequency</li> <li>Deletion strategy</li> </ul> <h3 class="wp-block-heading">4.2 Latency and Search Cost</h3> <p class="wp-block-paragraph">Users expect chat speed. Memory systems drift toward batch unless actively managed.</p> <p class="wp-block-paragraph">Latency sources:</p> <ul class="wp-block-list"> <li>Embedding computation</li> <li>ANN search</li> <li>Reranking</li> <li>Multi-hop retrieval</li> <li>Repeated retrieval inside a task</li> </ul> <p class="wp-block-paragraph">Architectural levers:</p> <ul class="wp-block-list"> <li>Precompute embeddings at ingestion</li> <li>Cache retrieval per episode</li> <li>Tier indexes for hot vs cold memory</li> <li>Deterministic limits on retrieval hops</li> </ul> <h3 class="wp-block-heading">4.3 Relationships and Structure</h3> <p class="wp-block-paragraph">Similarity is not structure.</p> <p class="wp-block-paragraph">If the question depends on sequences, causality, ownership, and time, pure vector indexes underperform.</p> <p class="wp-block-paragraph">Enterprise examples:</p> <ul class="wp-block-list"> <li>Incident timelines across services</li> <li>Integration flows across sagas</li> <li>Claim/case lifecycles with multiple actors</li> <li>Architecture decisions and downstream outcomes</li> </ul> <p class="wp-block-paragraph">Structure options:</p> <ul class="wp-block-list"> <li>Episodic timelines</li> <li>Entity indexes</li> <li>Event-sourced logs</li> <li>Lightweight graphs derived from episodes</li> </ul> <p class="wp-block-paragraph">Rule of thumb:</p> <p class="is-style-info wp-block-paragraph">If your questions involve “<em>who did what, when, and why</em>,” you need explicit structure, not a bigger vector DB.</p> <h2 class="wp-block-heading">5. The Shift From Recall to Episodic and Lifelong Memory</h2> <p class="wp-block-paragraph">The next step is not a larger vector DB. The shift is toward bounded memory units and deliberate consolidation.</p> <h3 class="wp-block-heading">5.1 Episodic Memory</h3> <p class="wp-block-paragraph">An episode is a coherent unit of experience: an incident, support case, sprint, deployment, design review, or decision sequence.</p> <p class="wp-block-paragraph">Chunks are arbitrary. Episodes are bounded and auditable.</p> <p class="wp-block-paragraph">A practical episodic store includes:</p> <ul class="wp-block-list"> <li>Episode metadata (time range, actors, entities, intent, outcome)</li> <li>Evidence links (tool outputs, tickets, commits, dashboards)</li> <li>Summaries at multiple resolutions (short, medium, deep)</li> <li>Optional embeddings for episode-level clustering and recall</li> </ul> <p class="wp-block-paragraph">TODO: Define a minimal episode schema (<code>episode_id</code>, <code>start_ts</code>, <code>end_ts</code>, <code>actors</code>, <code>entities</code>, <code>intent</code>, <code>outcome</code>, <code>evidence pointers</code>, <code>summaries</code>, <code>embedding pointers</code>).</p> <h3 class="wp-block-heading">5.2 Lifelong Agents and Multi-Tier Memory</h3> <p class="wp-block-paragraph">Long-running agents need tiers:</p> <ul class="wp-block-list"> <li>Working memory: current task, short-lived context</li> <li>Episodic memory: past tasks, incidents, conversations, decisions</li> <li>Semantic memory: distilled facts, preferences, policies from episodes</li> </ul> <p class="wp-block-paragraph">Key move: explicit consolidation. Without it, retrieval quality decays into probabilistic noise.</p> <h3 class="wp-block-heading">5.3 Memory as Resource Management</h3> <p class="wp-block-paragraph">Treat <strong>context windows as RAM</strong>. Treat <strong>external stores as disk</strong>. Treat <strong>retrieval as paging</strong>.</p> <p class="wp-block-paragraph">This forces policy decisions:</p> <ul class="wp-block-list"> <li>What stays hot</li> <li>What is paged in</li> <li>What is summarized</li> <li>What is evicted</li> <li>What is protected and audited</li> </ul> <h2 class="wp-block-heading">6. A 2026 Memory Stack for Enterprise Agents</h2> <p class="wp-block-paragraph">This is a reference architecture, not a product recommendation.</p> <h3 class="wp-block-heading">6.1 Ingress</h3> <p class="wp-block-paragraph">Inputs that generate memory writes:</p> <ul class="wp-block-list"> <li>User requests</li> <li>Tool outputs</li> <li>System events</li> <li>Logs and traces</li> <li>Documents and specs</li> </ul> <h3 class="wp-block-heading">6.2 Working Set</h3> <p class="wp-block-paragraph">Session-scoped, fast, aggressively bounded.</p> <p class="wp-block-paragraph">Policies:</p> <ul class="wp-block-list"> <li>Max token budget</li> <li>Eviction rules</li> <li>Summarization triggers</li> <li>Tool output compaction rules</li> </ul> <h3 class="wp-block-heading">6.3 Episodic Layer</h3> <p class="wp-block-paragraph">System of record for what happened.</p> <p class="wp-block-paragraph">Storage options:</p> <ul class="wp-block-list"> <li>Relational</li> <li>Document</li> <li>Event log with derived views</li> </ul> <p class="wp-block-paragraph">Core capabilities:</p> <ul class="wp-block-list"> <li>Append events into episodes</li> <li>Close episodes with outcomes</li> <li>Generate summaries</li> <li>Attach evidence pointers</li> <li>Index by time and entities</li> <li>Optional episode embeddings</li> </ul> <h3 class="wp-block-heading">6.4 Semantic Layer</h3> <p class="wp-block-paragraph">Distilled memory for reuse.</p> <p class="wp-block-paragraph">Examples:</p> <ul class="wp-block-list"> <li>Organizational constraints</li> <li>Architecture decisions and standards</li> <li>Preferred patterns and vendor constraints</li> <li>Known failure modes and mitigations</li> </ul> <p class="wp-block-paragraph">Storage options:</p> <ul class="wp-block-list"> <li>Relational or graph depending on query patterns</li> <li>Keep this store intentionally smaller than episodic memory</li> </ul> <h3 class="wp-block-heading">6.5 Access Patterns</h3> <p class="wp-block-paragraph">Before inference:</p> <ul class="wp-block-list"> <li>Retrieve semantic constraints</li> <li>Retrieve relevant episodes by entity/time</li> <li>Pull evidence snippets after episode selection</li> </ul> <p class="wp-block-paragraph">After inference:</p> <ul class="wp-block-list"> <li>Write tool calls and outputs</li> <li>Update episode state</li> <li>Generate short episode delta summaries</li> </ul> <p class="wp-block-paragraph">Background jobs:</p> <ul class="wp-block-list"> <li>Consolidation and pruning</li> <li>Contradiction detection in semantic memory</li> <li>Drift detection between standards and observed outcomes</li> </ul> <h2 class="wp-block-heading">7. Practical Patterns You Can Implement Now</h2> <h3 class="wp-block-heading">Pattern 1: Conversation Buffer + Episodic Log + Periodic Summaries</h3> <p class="wp-block-paragraph">Use case: ops and support copilots.</p> <ul class="wp-block-list"> <li>Working set for active incidents</li> <li>Episodic store per incident and follow-up actions</li> <li>Weekly or monthly summaries by service and failure mode</li> </ul> <p class="wp-block-paragraph">Trade-offs:</p> <ul class="wp-block-list"> <li>Low modelling overhead</li> <li>Strong audit trail</li> <li>Requires consolidation discipline</li> </ul> <h3 class="wp-block-heading">Pattern 2: Tool-Call Journal + Vector Index + Lightweight Graph</h3> <p class="wp-block-paragraph">Use case: integration and distributed design assistants.</p> <ul class="wp-block-list"> <li>Every tool call becomes a structured event</li> <li>Embeddings support recall</li> <li>Lightweight graph from episodes supports relationship queries</li> </ul> <p class="wp-block-paragraph">Trade-offs:</p> <ul class="wp-block-list"> <li>Higher build cost</li> <li>Better explainability</li> <li>Better “why this decision” traceability</li> </ul> <h3 class="wp-block-heading">Pattern 3: Preference and Policy Memory</h3> <p class="wp-block-paragraph">Use case: enterprise agents that must behave consistently.</p> <ul class="wp-block-list"> <li>Store constraints as semantic memory (security rules, cost limits, approved stacks)</li> <li>Inject constraints into planning, spec generation, and validation</li> </ul> <p class="wp-block-paragraph">Trade-offs:</p> <ul class="wp-block-list"> <li>Small store, high leverage</li> <li>Requires ownership and governance</li> <li>Failure mode: stale policy becomes hidden bias</li> </ul> <p class="wp-block-paragraph"><strong>Question</strong>: For each pattern, when do we use, what data shape, retrieval policy, consolidation policy, and key metrics do we use? </p> <h2 class="wp-block-heading">8. How Memory Changes Architecture and Design Work – Personal Agent vs Enterprise Grade</h2> <p class="wp-block-paragraph">Memory becomes first-class in architecture diagrams, alongside APIs, queues, and databases.</p> <h3 class="wp-block-heading">8.1 Governance</h3> <ul class="wp-block-list"> <li>Ownership of memory schemas</li> <li>Retention and residency rules</li> <li>Audit trail for what the agent knew at a given time</li> <li>Schema evolution and backwards compatibility</li> </ul> <h3 class="wp-block-heading">8.2 New Non-Functional Requirements</h3> <ul class="wp-block-list"> <li>Retrieval and reranking latency budgets</li> <li>Storage and retention cost controls</li> <li>Recall and consolidation quality metrics</li> <li>Privacy controls and redaction workflows</li> </ul> <h3 class="wp-block-heading">8.3 Checklist for Architects Starting an AI Initiative</h3> <ol class="wp-block-list"> <li>Define episodes for your domain.</li> <li>Define memory tiers and retention per tier.</li> <li>Define retrieval policy deterministically.</li> <li>Define consolidation jobs and schedules.</li> <li>Define evaluation harness and metrics.</li> <li>Define audit and access controls.</li> </ol> <h2 class="wp-block-heading">9. How I’m Applying These in Nova’s Production Stack</h2> <p class="wp-block-paragraph">In Nova’s production stack, these readings map directly to a multi-layer implementation rather than a single memory database.</p> <h3 class="wp-block-heading">My Current Multi-Layer Memory Implementation</h3> <ol class="wp-block-list"> <li><strong>Layer 1: Dense Embeddings (Mac endpoint)</strong></li> </ol> <ul class="wp-block-list"> <li>Model: <code>nomic-embed-text</code></li> <li>Purpose: semantic retrieval across conversations and notes</li> <li>Role: high-recall similarity for fast context seeding</li> </ul> <ol start="2" class="wp-block-list"> <li><strong>Layer 2: Knowledge Graph (Pi host)</strong></li> </ol> <ul class="wp-block-list"> <li>Nodes: concepts, decisions, tasks, memories</li> <li>Edges: <code>inspired_by</code>, <code>contradicts</code>, <code>part_of</code>, <code>related_to</code>, <code>evolved_from</code></li> <li>Role: relationship-aware retrieval beyond nearest-neighbor text</li> </ul> <ol start="3" class="wp-block-list"> <li><strong>Layer 3: Temporal Memory</strong></li> </ol> <ul class="wp-block-list"> <li>Session chronology and continuity across days</li> <li>Reinforcement via revisit signals</li> <li>Role: preserve sequencing and causality in long-running work</li> </ul> <ol start="4" class="wp-block-list"> <li><strong>Layer 4: Meta-Cognition</strong></li> </ol> <ul class="wp-block-list"> <li>Pattern detection over repeated interactions</li> <li>Insight generation for planning and tone adaptation</li> <li>Role: improve collaboration quality, not just retrieval quality</li> </ul> <ol start="5" class="wp-block-list"> <li><strong>Layer 5: Operational Memory Surface (OpenClaw runtime)</strong></li> </ol> <ul class="wp-block-list"> <li>Live memory ingestion from markdown paths</li> <li>Session-scoped memory orchestration</li> <li>Hook-driven memory capture and synchronization</li> <li>Role: connect architecture to day-to-day agent behavior</li> </ul> <h3 class="wp-block-heading">Why I Am Using This Approach</h3> <p class="wp-block-paragraph">I use this multi-layer design because each memory problem is different, and one store cannot optimize all of them at once.</p> <ul class="wp-block-list"> <li>Vector recall is fast but weak on causality.</li> <li>Graph structure is strong on relationships but expensive to curate.</li> <li>Temporal logs preserve sequence but need consolidation.</li> <li>Meta-cognitive summaries improve tone and continuity but must be grounded in evidence.</li> </ul> <p class="wp-block-paragraph">This layered architecture gives me better trade-off control across four things that matter in production:</p> <ol class="wp-block-list"> <li><strong>Continuity:</strong> fewer resets between sessions and projects</li> <li><strong>Quality:</strong> better context selection and less retrieval noise</li> <li><strong>Auditability:</strong> clearer evidence for “why the agent responded this way”</li> <li><strong>Cost control:</strong> bounded retrieval and consolidation policies over time</li> </ol> <p class="wp-block-paragraph">In short, I am not trying to build a bigger memory store. I am trying to build a memory system that stays useful under real operational pressure.</p> <hr class="wp-block-separator has-alpha-channel-opacity"/> <h2 class="wp-block-heading">10. Where This Series Goes Next</h2> <p class="wp-block-paragraph">This post sets the baseline. The next posts move from concept to measurable implementation.</p> <p class="wp-block-paragraph">Planned follow-ups:</p> <ul class="wp-block-list"> <li>Episodic schema and consolidation jobs</li> <li>Metrics and observability for memory quality</li> <li>Memory safety and access controls in multi-agent systems</li> <li>Cost modelling for retention and retrieval</li> <li>Patterns for graph-derived memory from event streams</li> </ul> <hr class="wp-block-separator has-alpha-channel-opacity"/> <h2 class="wp-block-heading">References (Working Set)</h2> <p class="wp-block-paragraph">The Big LLM Architecture Comparison (Sebastian Raschka)</p> <p class="wp-block-paragraph">Design Patterns for Long-Term Memory in LLM-Powered Architectures (Serokell)</p> <p class="wp-block-paragraph">How LLM Memory Works: Architecture, Techniques, and Developer Patterns (C-Sharp Corner)</p> <p class="wp-block-paragraph">Titans + MIRAS: Helping AI Have Long-Term Memory (Google Research)</p> <p class="wp-block-paragraph">LLM system design guides and architecture explainers (various)</p> <p class="wp-block-paragraph"></p>