AI is reshaping our software engineering practice in successive waves, it started with auto completion and predictive text, we the drove into prompt engineering, and more recently into context engineering, and is now we are evolving/narrowing towards spec-driven development to create new software and understand and modernise legacy software with greater accuracy, speed and higher quality
Each phase introduced new practices, new risks, and new tooling. In this post we look at the evolution and outline technical examples, and discusses the implications for engineering practice.
1. Introduction
Software engineering has always oscillated between human creativity and machine automation. The advent of large language models (LLMs) accelerated this tension. Rather than a single breakthrough, the discipline has unfolded in distinct phases:
- Auto complete — AI as a micro-productivity booster.
- Prompt Engineering — AI as a dialogue partner.
- Context Engineering — AI as a system component.
- Spec-Driven Development — AI as an executor bound by specification.
2. Auto-Complete: The Productivity Baseline
Characteristics
- Tools: IntelliSense, GitHub Copilot (early versions).
- Scope: Token-level prediction; local scope only.
- Benefit: Reduced typing overhead, suggested APIs.
Example
# Typing 'import nump' → auto-complete suggests 'import numpy as np' import numpy as np arr = np.array([1, 2, 3]) # AI predicts correct method signatures
This phase improved velocity but not architecture. Code quality remained dependent on human oversight.
3. Prompt Engineering: Negotiating with the Model
Characteristics
- Tools: GPT-3, GPT-4, Claude.
- Scope: Natural language prompts guiding generation.
- Benefit: Expressive, flexible; opened non-technical users to programming.
- Risk: Ambiguity, prompt-fragility, reproducibility issues.
Example
Prompt:"Write a Python function to scrape product names and prices from an e-commerce site."
Output (simplified):import requests from bs4 import BeautifulSoup def scrape_products(url): page = requests.get(url) soup = BeautifulSoup(page.text, 'html.parser') products = [(p.text, price.text) for p, price in zip(soup.find_all('h2'), soup.find_all('span', class_='price'))] return products
The quality of output depends heavily on how specific the prompt is. Prompt engineering emerged as an ad-hoc discipline to manage this variance.
4. Context Engineering: Systems of Grounding
Characteristics
- Tools: RAG pipelines, vector databases (Pinecone, Weaviate, Redis), orchestration frameworks (LangChain, LlamaIndex).
- Scope: Injecting relevant documents, APIs, or state into the model.
- Benefit: Reduces hallucinations, increases grounding, supports enterprise integration.
Example: RAG for Legal Cases
# Step 1: Embed case law documents into a vector DB embedding = model.embed(text) db.upsert({"id": case_id, "embedding": embedding})
# Step 2: Retrieve context for a query query_embedding = model.embed("What is precedent for capital gains tax in Australia?") matches = db.similarity_search(query_embedding)
# Step 3: Inject context into LLM llm_prompt = f""" Answer using the following sources: {matches} """ response = llm.generate(llm_prompt)
Amazon Kiro (kiro.dev) embodies this phase. It integrates with Model Context Protocol (MCP) servers to supply codebases, APIs, and rules into the IDE. Context is no longer hand-stitched but orchestrated.
5. Spec-Driven Development: Alignment over Improvisation
Origins of the Spec
The idea of a specification isn’t new. In traditional engineering disciplines, the spec is the binding contract between design intent and implementation. Civil engineers don’t improvise bridges in steel; they work from blueprints. Software engineers, however, often drifted into “code first, document later” practices. Specs were sidelined into lightweight user stories or JIRA tickets. AI exposed how brittle this was: without a clear spec, model outputs wandered, hallucinated, or contradicted architecture constraints.
What Spec-Driven Development Is
Spec-Driven Development (SDD) re-centres the specification as the source of truth. Instead of generating code directly from loose prompts, the process is structured:
1 Specify — Capture what must be built, why it matters, and non-negotiable constraints (security, scale, compliance).
2 Plan — Derive candidate architectures, technology choices, and sequencing from the spec.
3 Tasks — Break down into unit-sized steps that AI agents and developers can own.
4 Implement — Code is generated to fulfil tasks, reviewed against the original spec, and iterated.
The defining principle: code serves the spec, not the other way around.
Characteristics
- Tools: GitHub Spec Kit, Cursor IDE, Windsurf, Amazon Kiro.
- Scope: Specification → Plan → Tasks → Implementation.
- Benefit: Discipline, reproducibility, alignment with constraints.
Example Workflow with GitHub Spec Kit
- Specify: Produce a structured spec file (YAML/Markdown) that captures intent.
specify init "E-commerce cart service must support 10k concurrent users, REST + GraphQL APIs, and PCI compliance." - Plan: AI suggests architecture diagrams and stack recommendations.
- Node.js service on AWS Lambda
- PostgreSQL with read replicas
- Integration with Stripe SDK for payments
- Tasks: Expands the plan into granular backlog items: database schema, endpoints, tests.
/tasksfor example creates this plan - Implement: Code is generated per task, reviewed against the original spec. Developer reviews at each stage.
Why Spec Driven Development is a good step forward
With spec driven development
1. Discipline restored — architecture decisions are upfront, not buried in code.
2. AI alignment — large models follow specs better than vague prompts.
3. Reproducibility — changes can be diffed against spec updates, not only code.
4. Scalability — large teams (and AI agents) can coordinate around one structured artifact.
In this sense, SDD is not a radical invention but a return: it borrows the rigor of traditional engineering and fuses it with AI’s generative capacity. It represents a maturation of AI-led engineering — from improvisation to intentionality.
The key difference: the code serves the spec, not the other way around (spec-driven.md).
6. Implications for Engineering Practice
- Cultural Shift: Docs and specs reclaim primacy.
- Governance: Security, compliance, and performance constraints encoded upfront.
- Collaboration: Engineers act as spec authors and reviewers; AI handles implementation.
- Limitations: Requires rigorous spec writing; adds overhead but improves long-term stability.
7. Conclusion
The arc is instructive:
– Auto-complete was speed.
– Prompt engineering was expression.
– Context engineering was grounding.
– Spec-driven development is alignment.
AI-led engineering is converging on a principle: clarity of intent drives scale. The engineer of the future may spend less time typing code and more time crafting specifications precise enough for intelligent systems to implement.
References
- GitHub Blog — Spec-Driven Development with Spec Kit
- GitHub Spec Kit — Open Source Toolkit
- GitHub Spec-Driven Manifesto — spec-driven.md
- Amazon Kiro Blog — Unlock productivity with Kiro and MCP
- LogRocket — Hands-on with Amazon Kiro
Alok Mishra 