Cycle-Driven Engineering: Building the Context Layer for AI-Assisted Engineering

AI-assisted engineering has made project context more visible as an engineering problem. Teams are adding architecture documents, coding rules, repository instructions, examples and decision records, but having more information available does not mean the right information reaches the right task.

Documentation captures what is known about a project. Context is the subset of that knowledge needed to understand, plan and perform a specific change. A repository can therefore be well documented and still provide poor context to both engineers and AI agents.

The Context Layer is the mechanism that closes that gap. It turns project knowledge into task-specific context through a simple flow:

Knowledge → Classification → Discovery → Selection → Task context

The first step is making sure the necessary knowledge exists in an explicit and reliable form. No amount of indexing, retrieval or context assembly can compensate for architecture, decisions and implementation patterns that remain undocumented or scattered across people’s heads.

Knowledge

The Context Layer starts with a simple requirement: the project/repository must describe itself explicitly. Architecture, important decisions, coding conventions, domain concepts and common implementation patterns cannot remain distributed across people’s memories, old conversations or the source code alone.

In practice, this may translate into a dedicated docs folder inside the repository. That folder becomes the canonical home for project knowledge, while a README.md at its root acts as the entry point. Its purpose is not to summarize the entire system in one large document, but to explain how the documentation is organized and direct engineers or AI agents towards the area relevant to the work at hand.

One area may be a systemOverview folder. It should describe the system at a level that can be understood without first reading the source code: its purpose, boundaries, major components, dependencies and technology stack. It may also introduce the technical concepts used across the system, such as how CQRS is applied, how asynchronous processing works or how services communicate. Each folder and subfolder should include its own README.md, allowing someone to enter the documentation at any level and understand what is available below it.

A separate architecture folder can provide a deeper description of how and why the system has been designed in a particular way. This is where detailed component interactions, data flows, deployment architecture, cross-cutting patterns and significant architectural decisions belong.

Some overlap with the system overview is inevitable, but the distinction remains useful: the overview provides the map, while the architecture documentation explains the structures and decisions behind it. Architecture Decision Records can live here as well, preserving rationale and trade-offs without mixing historical decisions into the description of the current system.

A features folder can organize knowledge vertically around the behaviour of the product. Each feature receives its own subfolder containing a complete domain description of how it is expected to work, together with the relevant technical implementation. This should cover business rules, important scenarios, state transitions, integrations, affected components and feature-specific constraints. An engineer or agent working on a feature should not need to reconstruct its intended behaviour from controllers, database tables, tests and old tickets.

Machine-readable diagrams should be part of each level of this structure. Mermaid or similar text-based formats can be versioned, reviewed, searched and updated alongside the documentation they support. A system overview may contain context and component diagrams, architecture documentation may describe interactions or deployment flows, while a feature folder may include state, sequence or domain-flow diagrams. Keeping diagrams close to the explanation they support reduces the risk of detached, outdated visual documentation.

This structure creates the foundation for the rest of the Context Layer. The root README.md points towards the main areas, each area narrows the scope, and each feature or architectural topic becomes a self-contained source that can later be classified, discovered and selected for a specific task. The repository does not need to load all available knowledge into every agent session. It first needs to make that knowledge predictable enough that the relevant pieces can be found.

For an existing project with little or poor documentation, the goal should not be to describe the entire system upfront. The starting point can be a high-level system map, the most important architectural patterns and the features currently being changed or repeatedly misunderstood. The knowledge base can then grow through normal engineering work, with each task filling a real context gap rather than producing documentation for its own sake.

Classification

Once project knowledge has a predictable structure, each artifact needs to make its meaning explicit. A document should not rely on its title alone to communicate where it applies, how it should be interpreted or whether it still represents the current system.

The folder structure already provides a first level of classification. Knowledge under systemOverview is broad and descriptive. Documentation under architecture explains deeper technical structures and decisions. Material under features is scoped to a particular area of product behaviour and its implementation. As the hierarchy becomes more specific, so does the context it can provide.

Location, however, is not always enough. Important documents should state their scope, purpose and status explicitly. A repository-wide convention should be distinguishable from a rule that applies only to one component. An accepted architectural decision should not look the same as a proposal, and a historical document should not be mistaken for guidance that is still active.

In practice, this may be achieved through a small amount of consistent metadata or a standard introductory section. Depending on the type of document, this can identify:

• the system, component or feature it applies to;
• whether it is descriptive, normative or historical;
• whether it represents a mandatory rule, an accepted decision, a recommended pattern or an example;
• whether it is active, deprecated or superseded;
• who owns it and which document replaces it when applicable.

This classification becomes especially important when AI agents consume project knowledge. Humans can often infer from experience that one document is outdated or that a particular example should not be copied. An agent is more likely to treat all retrieved text as equally valid unless those distinctions are made explicit.

Classification therefore prepares knowledge for reliable discovery and selection. Before the correct information can be assembled for a task, the system must first be able to determine what each artifact means and where its authority begins and ends.

Discovery

Once knowledge is structured and classified, it must be possible to find it without already knowing the repository by heart. This is where the hierarchy of README.md files becomes important. The root docs/README.md acts as the main entry point, while each folder-level README.md explains what is available within that area and links towards more specific sources.

This creates a navigable path through the documentation. A task involving a feature may begin in features, then lead towards the relevant domain description, technical implementation and architectural decisions. A task affecting a shared technical concern may instead move from the system overview into architecture, patterns or ADRs. Cross-links between related areas are important because project knowledge rarely fits perfectly inside a single branch of the structure.

Discovery should also be supported by predictable naming and lightweight indexing. Engineers and agents should be able to search for a feature, component, pattern or technical concept and reach the canonical source rather than several competing documents. Tools such as ctxkit can help generate or maintain this index and expose it to agent skills, rules or repository-level instructions.

The goal is not to load the entire documentation set into every session. The initial context only needs to provide a map of what exists and how to reach it. More detailed knowledge can then be loaded progressively as the task becomes clearer.

Discovery identifies the sources that may be relevant. It does not yet decide which of them belong in the final working context. That is the responsibility of selection.

Selection

Discovery makes potentially relevant knowledge accessible, but not all of it should enter the context of every task. The purpose of selection is to reduce that wider knowledge set to the smallest collection of instructions, documentation and examples needed for the work at hand.

AI agents can perform part of this selection themselves, but they should not be expected to search the entire repository and decide from scratch what matters. The project should first define the routes through which context can be selected. Repository and path-level rules provide the mandatory baseline, while domain-specific sub-agents and granular skills narrow the available knowledge according to the feature, component or technical concern involved.

A task related to onboarding, for example, may first be routed to a sub-agent responsible for that area of the system. That sub-agent can then activate more focused skills covering data ingestion, domain behaviour or implementation patterns. Each skill acts as a controlled entry point into the documentation, pointing towards the relevant feature descriptions, architectural decisions, examples and code references without loading everything known about onboarding.

The descriptions attached to sub-agents and skills are therefore part of the selection mechanism. They should clearly state the concepts, terminology and types of work for which that context is relevant. Broad or vague descriptions make context selection unreliable, while overly large skills recreate the same problem as monolithic instruction files. Context units need to be specific enough that the agent can distinguish between them and load only what the task requires.

Selection should also account for the phase of work. Investigation may require system and feature documentation. Planning may need architectural decisions, constraints and acceptance criteria. Execution may additionally require coding conventions, implementation patterns and canonical examples. Loading implementation details too early can distract planning, while omitting them during execution leaves the agent to infer local conventions from the code.

The final choice remains partly probabilistic, but it takes place inside boundaries defined by the project. The repository determines which rules always apply, how work is routed, which skills are available and where their authoritative sources live. The agent selects among those predefined context units rather than treating every document and code example as equally relevant.

Once that selection has been made, the chosen knowledge can be assembled into the task context used for planning and execution.

Task Context

The output of the Context Layer is not the documentation itself, but the task-specific context assembled from it. This is the subset of project knowledge that a human or AI agent needs in order to understand the change, plan it correctly and execute it without reconstructing the system from scratch.

A complete task context may include the relevant feature description, applicable architectural decisions, repository and path-level rules, implementation patterns, affected components, important code references, constraints and expected validation steps. The exact composition depends on the task. A small change may require only a few focused sources, while a broader or riskier change may need context from several parts of the repository.

This context should also preserve the distinction between what has already been decided and what still needs to be resolved. Existing architecture, business rules and mandatory constraints belong in the task context as settled information. Open questions, implementation options and trade-offs belong in planning. Mixing the two makes it harder for both engineers and agents to understand where they are expected to follow an existing decision and where they are expected to make a new one.

Task context should therefore be specific enough to answer a few practical questions before planning begins:

• What part of the system is being changed?
• Which feature or domain behaviour is involved?
• Which architectural boundaries and previous decisions apply?
• Which rules, patterns and examples should guide the implementation?
• Which constraints must not be violated?
• How will the result be validated?

When those answers are available, planning can focus on the change itself rather than rediscovering how the project works. This is the point where project knowledge has completed its transformation into usable context.

The Context Layer Must Change with the System

The Context Layer cannot be maintained independently of the system it describes. When a change invalidates a feature description, architectural decision, implementation pattern or repository instruction, the relevant context should be updated within the same delivery workflow.

This does not mean rewriting every historical artifact. Project context needs to distinguish clearly between the current state of the system and its archaeology. ADRs, completed plans and older specifications may remain valuable because they explain how the system arrived at its present shape and might be useful during discoveries and planning sessions. They become harmful when engineers or agents cannot tell whether they describe what exists today, what was once proposed or what has already been replaced.

This is one of the risks of spec-driven approaches that continuously accumulate specifications without maintaining a clear representation of the current system. Over time, the repository may contain a detailed history of previous intentions while offering no reliable answer to how the system is expected to behave now. The amount of available context increases, but its usefulness decreases.

Historical material should therefore remain explicitly historical, while active documentation should provide the canonical description of the current architecture, feature behaviour and implementation expectations. When a decision changes, the previous record may be superseded rather than erased, but every path through the documentation should lead clearly towards the active source.

The Context Layer should also improve through the work performed against it. When an engineer or agent repeatedly reconstructs the same knowledge, selects an obsolete source or misses an important constraint, the failure should be traced back to the appropriate stage of the model. The problem may be missing knowledge, poor classification, weak discovery or incorrect selection. Adding another global instruction is rarely the only possible response.

The Context Layer should evolve through the same cycles as the system itself. Otherwise, it gradually stops being a source of guidance and becomes a source of incorrect authority.

Closing Thoughts

The Context Layer is not built by simply adding more documentation to a repository. It emerges when project knowledge is structured, classified, made discoverable and selected deliberately for the work being performed.

Agents can consume far more information than engineers would normally read, but they still need a reliable way to distinguish current guidance from archaeology, global rules from local patterns and useful context from unnecessary noise.

A good Context Layer makes the relevant parts of the repository available when a specific change is being planned or implemented. Once that task context is available, the next challenge is turning it into a concrete and reviewable approach to change. That is the responsibility of the Planning Layer.

FAQ

What is the Context Layer in Cycle-Driven Engineering?

It is the mechanism that turns project knowledge into task-specific context. Documentation captures what is known about a project; context is the subset of that knowledge needed to understand, plan and perform a specific change. The Context Layer closes the gap between the two through a flow of knowledge, classification, discovery, selection and task context.

How is context different from documentation?

Documentation captures everything known about a project. Context is the smaller subset needed for a specific change. A repository can be thoroughly documented and still provide poor context, because having more information available does not mean the right information reaches the right task.

What are the stages of the Context Layer?

Knowledge, classification, discovery, selection and task context. Knowledge must exist explicitly, classification makes each artifact’s scope and status clear, discovery makes it findable without knowing the repository by heart, selection reduces it to the smallest useful set, and task context is the assembled result used for planning and execution.

Why does classification matter more for AI agents than for people?

Humans can often infer from experience that a document is outdated or that a particular example should not be copied. An agent is more likely to treat all retrieved text as equally valid unless scope, purpose and status are made explicit, so classification prevents obsolete or non-authoritative material from being used as if it were current guidance.