# Generic Markdown Stage Format for Spec-Driven AI Agent Pipelines Suggested filename: `STAGE.md` --- ## Goal One markdown file = one self-sufficient pipeline stage. Each stage must: - describe what it is - declare its input dependencies - declare its output contract - link to previous and next stages - be human-readable - be parseable by a minimal Python agent - work for any pipeline topology, not only `requirements → design → tasks → scaffold` --- ## Core Idea Each stage file is markdown with a YAML frontmatter header. The frontmatter carries routing, validation, and execution metadata. The body carries the actual stage content. Minimal required frontmatter: ```yaml --- stage_id: design stage_type: design pipeline_id: feature status: draft prev: [] next: [] inputs: [] outputs: [] --- ``` Full recommended frontmatter: ```yaml --- stage_id: design stage_type: design pipeline_id: feature pipeline_version: 1 status: draft title: Feature Design owner: agent created_at: 2026-04-27T00:00:00Z updated_at: 2026-04-27T00:00:00Z prev: - requirements next: - tasks inputs: - stage_id: requirements path: requirements.md required: true contract: requirements.v1 outputs: - stage_id: tasks path: tasks.md required: true contract: tasks.v1 depends_on: - requirements produces: - tasks agent: role: software_architect model_policy: default mode: deterministic allowed_actions: - read - write - edit forbidden_actions: - deploy - delete quality_gate: required_sections: - Objective - Inputs - Context - Decisions - Open Questions - Risks - Output Contract must_be_self_sufficient: true must_reference_inputs: true must_define_next_stage: true trace: source_request: null parent_stage: requirements generated_by: agent --- ``` --- ## Canonical Stage Body Template ````markdown # ## Objective What this stage must accomplish. ## Inputs | Stage | Path | Required | Purpose | | ------------ | --------------- | :------: | --------------------------------------- | | requirements | requirements.md | yes | Defines target behavior and constraints | ## Context Self-contained background needed to understand this stage without reading the whole project. ## Current State What is already known, built, broken, missing, or assumed. ## Decisions Concrete decisions made in this stage. ```yaml decisions: - id: D001 decision: Use markdown stage files with YAML frontmatter. reason: Human-readable and easy to parse from Python. impact: Enables simple spec-driven pipelines without a database. ``` ```` ## Main content. The heading varies by `stage_type`: | stage_type | heading | | ------------ | --------------- | | requirements | Requirements | | design | Architecture | | bugfix | Bug Analysis | | tasks | Task Breakdown | | scaffold | Scaffold Plan | | review | Review Findings | ## Open Questions ```yaml open_questions: - id: Q001 question: Should scaffold generation create tests by default? blocks_next_stage: false ``` ## Risks ```yaml risks: - id: R001 risk: Stage output may be too vague for implementation. mitigation: Require an explicit output contract and acceptance criteria. ``` ## Output Contract What the next stage can safely consume. ```yaml contract: stage_id: design produces: tasks guarantees: - Architecture decisions are explicit. - Interfaces are described. - Known risks are listed. - Next stage has enough information to create implementation tasks. ``` ## Next Stage Instructions ```yaml next_stage: stage_id: tasks goal: Convert design decisions into ordered implementation tasks. must_include: - task id - description - files touched - acceptance criteria - dependencies ``` ```` --- ## Stage Type Registry Suggested filename: `stage_types.yaml` ```yaml stage_types: requirements: purpose: Define what must be built and how success is verified. required_sections: - Objective - Context - Functional Requirements - Non-Functional Requirements - Acceptance Criteria - Output Contract design: purpose: Convert requirements or bug analysis into technical decisions. required_sections: - Objective - Inputs - Architecture - Decisions - Risks - Output Contract bugfix: purpose: Analyze a defect and define the expected correction. required_sections: - Objective - Bug Summary - Reproduction - Expected Behavior - Actual Behavior - Suspected Cause - Output Contract tasks: purpose: Convert upstream stage into ordered implementation tasks. required_sections: - Objective - Inputs - Tasks - Acceptance Criteria - Output Contract scaffold: purpose: Generate initial file layout and code skeleton. required_sections: - Objective - Inputs - Files - Commands - Verification - Output Contract ```` --- ## Pipeline Definitions Suggested filename: `pipelines.yaml` ```yaml pipelines: feature: stages: - requirements - design - tasks - scaffold design-first: stages: - design - requirements - tasks - scaffold bugfix: stages: - bugfix - design - tasks - scaffold research-first: stages: - research - requirements - design - tasks - scaffold refactor: stages: - assessment - design - tasks - implementation - review ``` --- ## Examples ### `requirements.md` ````markdown --- stage_id: requirements stage_type: requirements pipeline_id: feature pipeline_version: 1 status: ready prev: [] next: - design inputs: [] outputs: - stage_id: design path: design.md required: true contract: design.v1 depends_on: [] produces: - design quality_gate: required_sections: - Objective - Context - Functional Requirements - Non-Functional Requirements - Acceptance Criteria - Output Contract must_be_self_sufficient: true must_reference_inputs: false must_define_next_stage: true --- # Requirements ## Objective Define what the feature must do, why it exists, and how success will be verified. ## Context The system uses markdown files as pipeline stages. A Python agent reads one stage, validates it, and generates the next stage. ## Functional Requirements ```yaml functional_requirements: - id: FR001 requirement: The agent must parse markdown files with YAML frontmatter. priority: must - id: FR002 requirement: The agent must identify previous and next stages from metadata. priority: must - id: FR003 requirement: The stage body must be understandable without hidden state. priority: must ``` ```` ## Non-Functional Requirements ```yaml non_functional_requirements: - id: NFR001 requirement: Implementation must stay minimal. priority: must - id: NFR002 requirement: Format must be generic across feature, bugfix, refactor, and research pipelines. priority: must ``` ## Acceptance Criteria ```yaml acceptance_criteria: - id: AC001 criterion: A stage file declares its previous and next stages. - id: AC002 criterion: A stage file declares input and output contracts. - id: AC003 criterion: A minimal Python script can parse and validate the file. ``` ## Output Contract ```yaml contract: stage_id: requirements produces: design guarantees: - Requirements are explicit. - Acceptance criteria are present. - The next stage can produce a design without extra hidden context. ``` ## Next Stage Instructions ```yaml next_stage: stage_id: design goal: Convert requirements into concrete architecture and design decisions. must_include: - system boundaries - data model - stage lifecycle - validation rules - risks ``` ```` --- ### `design.md` ```markdown --- stage_id: design stage_type: design pipeline_id: feature pipeline_version: 1 status: ready prev: - requirements next: - tasks inputs: - stage_id: requirements path: requirements.md required: true contract: requirements.v1 outputs: - stage_id: tasks path: tasks.md required: true contract: tasks.v1 depends_on: - requirements produces: - tasks quality_gate: required_sections: - Objective - Inputs - Architecture - Stage Model - Validation Rules - Risks - Output Contract must_be_self_sufficient: true must_reference_inputs: true must_define_next_stage: true --- # Design ## Objective Design a generic markdown-based stage format for a spec-driven AI agent pipeline. ## Inputs | Stage | Path | Required | Purpose | | ------------ | --------------- | :------: | ------------------------------------- | | requirements | requirements.md | yes | Defines required behavior and constraints | ## Architecture ```text pipeline definition ↓ stage markdown files ↓ minimal Python runner ```` The pipeline definition decides stage order. Each stage file contains: - YAML frontmatter — machine routing and validation metadata - markdown body — human and model context - output contract — guarantees for the next stage - next stage instructions — explicit prompt for the following agent call ## Stage Model ```yaml stage: identity: - stage_id - stage_type - pipeline_id routing: - prev - next - inputs - outputs execution: - status - agent - quality_gate content: - markdown body - structured yaml blocks - next stage instructions ``` ## Validation Rules ```yaml validation_rules: - id: V001 rule: stage_id must be present. - id: V002 rule: stage_type must be present. - id: V003 rule: prev and next must be lists. - id: V004 rule: required_sections must exist in the markdown body. - id: V005 rule: outputs must describe the next produced stage. ``` ## Risks ```yaml risks: - id: R001 risk: Too much schema makes the format heavy to hand-edit. mitigation: Keep only routing metadata mandatory. - id: R002 risk: Too little structure makes agent output unreliable. mitigation: Require output contracts and quality gates on every stage. ``` ## Output Contract ```yaml contract: stage_id: design produces: tasks guarantees: - Stage metadata model is defined. - Validation rules are explicit. - The tasks stage can implement parser, validator, pipeline loader, and runner. ``` ## Next Stage Instructions ```yaml next_stage: stage_id: tasks goal: Convert this design into implementation tasks. must_include: - parser task - validator task - pipeline loader task - stage generator task - minimal CLI task ``` ```` --- ### `tasks.md` ```markdown --- stage_id: tasks stage_type: tasks pipeline_id: feature pipeline_version: 1 status: ready prev: - design next: - scaffold inputs: - stage_id: design path: design.md required: true contract: design.v1 outputs: - stage_id: scaffold path: scaffold.md required: true contract: scaffold.v1 depends_on: - design produces: - scaffold quality_gate: required_sections: - Objective - Inputs - Tasks - Acceptance Criteria - Output Contract must_be_self_sufficient: true must_reference_inputs: true must_define_next_stage: true --- # Tasks ## Objective Create a minimal implementation plan for markdown-driven pipeline execution. ## Inputs | Stage | Path | Required | Purpose | | ------ | --------- | :------: | ---------------------------------------- | | design | design.md | yes | Defines architecture and validation rules | ## Tasks ```yaml tasks: - id: T001 title: Implement markdown stage parser files: - stage.py acceptance: - Can read YAML frontmatter. - Can read markdown body. - Returns a Stage dataclass. - id: T002 title: Implement stage validator files: - stage.py acceptance: - Validates required metadata keys. - Validates required body sections. - Returns a list of error strings (empty = valid). - id: T003 title: Implement pipeline definition loader files: - pipeline.py acceptance: - Loads stage order from pipelines.yaml. - Supports arbitrary pipeline names. - Returns previous and next stage for a given stage_id. - id: T004 title: Implement minimal CLI files: - main.py acceptance: - Validates a stage file and prints errors. - Prints previous and next stage links. - Exits non-zero on validation failure. ```` ## Acceptance Criteria ```yaml acceptance_criteria: - id: AC001 criterion: Parses a valid stage markdown file without errors. - id: AC002 criterion: Rejects a stage missing required metadata keys. - id: AC003 criterion: Loads a generic pipeline definition from pipelines.yaml. ``` ## Output Contract ```yaml contract: stage_id: tasks produces: scaffold guarantees: - Implementation tasks are explicit. - Required files are named. - Acceptance criteria are testable. ``` ## Next Stage Instructions ```yaml next_stage: stage_id: scaffold goal: Generate minimal Python files for parser, validator, pipeline loader, and CLI. must_include: - pyproject.toml - stage.py - pipeline.py - main.py ``` ```` --- ## Minimal Python Implementation Dependency: `pip install pyyaml` Suggested layout: ```text spec_agent/ stage.py pipeline.py main.py pipelines.yaml stage_types.yaml requirements.md design.md tasks.md ```` ### `stage.py` ```python from __future__ import annotations from dataclasses import dataclass from pathlib import Path from typing import Any import yaml @dataclass(frozen=True) class Stage: path: Path meta: dict[str, Any] body: str @property def stage_id(self) -> str: return str(self.meta["stage_id"]) @property def stage_type(self) -> str: return str(self.meta["stage_type"]) @property def prev(self) -> list[str]: return list(self.meta.get("prev") or []) @property def next(self) -> list[str]: return list(self.meta.get("next") or []) def load_stage(path: str | Path) -> Stage: stage_path = Path(path) text = stage_path.read_text(encoding="utf-8") if not text.startswith("---\n"): raise ValueError(f"{stage_path}: missing YAML frontmatter") _, raw_meta, body = text.split("---", 2) meta = yaml.safe_load(raw_meta) or {} if not isinstance(meta, dict): raise ValueError(f"{stage_path}: frontmatter must be a YAML mapping") return Stage(path=stage_path, meta=meta, body=body.strip()) _REQUIRED_META = ( "stage_id", "stage_type", "pipeline_id", "status", "prev", "next", "inputs", "outputs", ) _LIST_FIELDS = ("prev", "next", "inputs", "outputs") def validate_stage(stage: Stage) -> list[str]: errors: list[str] = [] for key in _REQUIRED_META: if key not in stage.meta: errors.append(f"missing metadata: {key}") for key in _LIST_FIELDS: if key in stage.meta and not isinstance(stage.meta[key], list): errors.append(f"metadata must be a list: {key}") quality_gate = stage.meta.get("quality_gate") or {} for section in quality_gate.get("required_sections") or []: if f"## {section}" not in stage.body: errors.append(f"missing required section: {section}") return errors ``` ### `pipeline.py` ```python from __future__ import annotations from pathlib import Path import yaml def load_pipelines(path: str | Path) -> dict[str, list[str]]: data = yaml.safe_load(Path(path).read_text(encoding="utf-8")) or {} raw = data.get("pipelines") or {} result: dict[str, list[str]] = {} for name, config in raw.items(): stages = config.get("stages") if isinstance(config, dict) else config if not isinstance(stages, list) or not all(isinstance(s, str) for s in stages): raise ValueError(f"invalid pipeline definition: {name}") result[str(name)] = stages return result def get_neighbors( stages: list[str], stage_id: str ) -> tuple[str | None, str | None]: idx = stages.index(stage_id) prev = stages[idx - 1] if idx > 0 else None nxt = stages[idx + 1] if idx < len(stages) - 1 else None return prev, nxt ``` ### `main.py` ```python from __future__ import annotations import argparse import sys from pipeline import get_neighbors, load_pipelines from stage import load_stage, validate_stage def main() -> int: parser = argparse.ArgumentParser(description="Validate and inspect a pipeline stage.") parser.add_argument("stage_file", help="Path to the stage markdown file") parser.add_argument("--pipelines", default="pipelines.yaml", help="Pipeline definitions YAML") args = parser.parse_args() stage = load_stage(args.stage_file) errors = validate_stage(stage) if errors: for e in errors: print(f"ERROR: {e}", file=sys.stderr) return 1 pipeline_id = str(stage.meta["pipeline_id"]) pipelines = load_pipelines(args.pipelines) if pipeline_id not in pipelines: print(f"ERROR: pipeline not found: {pipeline_id}", file=sys.stderr) return 1 prev_stage, next_stage = get_neighbors(pipelines[pipeline_id], stage.stage_id) print(f"stage: {stage.stage_id}") print(f"type: {stage.stage_type}") print(f"pipeline: {pipeline_id}") print(f"prev: {prev_stage}") print(f"next: {next_stage}") return 0 if __name__ == "__main__": raise SystemExit(main()) ``` --- ## Agent Prompt Contract Suggested filename: `STAGE_GENERATION_PROMPT.md` ```markdown You are executing a spec-driven markdown pipeline. You will receive: 1. Current stage markdown. 2. Optional previous stage markdown. 3. Target next stage id. 4. Pipeline definition. Your job: - Generate only the next stage markdown file. - Preserve YAML frontmatter structure. - Make the stage self-sufficient — no hidden context assumed. - Connect it to previous and next stages via `prev`, `next`, `inputs`, `outputs`. - Include all sections listed in `quality_gate.required_sections`. - Include an explicit `Output Contract`. - Include explicit `Next Stage Instructions`. - Do not skip risks, open questions, or acceptance criteria when relevant. Output must be a valid markdown file with YAML frontmatter and nothing else. ``` --- ## Core Rule **Strict in metadata. Flexible in body.** Only eight fields are mandatory: ```yaml stage_id: stage_type: pipeline_id: status: prev: [] next: [] inputs: [] outputs: [] ``` Everything else is optional and can evolve without breaking existing stages. This gives you: - generic pipelines with arbitrary topology - minimal Python parsing (pyyaml only) - readable, hand-editable markdown - enough structure for reliable agent execution - no database, no framework, no special tooling - clean git diffs and code review