The Seven Layers of the Software Factory Stack—and the Comments Went Straight for Permissions and Review Costs

Zach Lloyd spoke at AI Engineer Code Summit SF about the seven-layer software factory stack. A software factory can be understood as an automated pipeline for building software, with agents, permissions, sandboxes, models, observability, and entry points all inside. He split this into seven blocks, 00 through 06.

00 Factory as code. Agent workflows, access permissions, and cloud sandbox configuration should all become version-controlled code that both humans and agents can change.

01 Data and context. The data agents touch should live on your own infrastructure. Internal skills and MCP, agent traces, access logs, and telemetry all need to be kept in your own hands.

02 Compute. Agents need their own computers, with controllable sandbox permissions, support for multiple operating systems, and the ability to self-host.

03 Inference. Any model and any harness should be pluggable, so you can run benchmarks and find the best fit. Don’t lock yourself into a single model vendor.

04 Improvement. You need tools to measure and improve the factory. Observability metrics include cost-per-PR, cycle time, latency, and other DORA metrics. Agents should also be able to score output quality, propose changes to factory configuration, and then re-measure, creating a loop.

05 Orchestration. You need a runtime to launch agents, track work, recover from failures, and handle message passing in multi-agent systems.

06 Access. The factory needs to connect to external systems. The factory itself should have an MCP server and REST API, and it also needs to connect to where teams work, such as Slack, MS Teams, Jira, and Linear. Zach does not recommend tightly coupling the ADE or web interface to the factory.

The comments didn’t stop at applause. @ilmAI asked a specific question: permissions as code should be done first, but can agents change their own access file, or must a human merge it?

@Karthik pressed on what cost-per-PR actually measures: does it count only tokens and sandbox compute, or human review time as well? He thinks review may now be the bigger piece.

@John Rood added a part of 01 that is easy to miss: traces tell you what ran, not why. Which branch the agent took, which branch it abandoned—the reasoning wasn’t written down, and you can’t reconstruct it later. He mentioned that memoryrouter preserves that rationale.

@Ethan Walkrman thought the most compelling point was that agents score their own runs and open PRs against factory configuration. That’s what gives the whole stack a continuous-improvement loop.

@Biscuit said their sandbox rules are still sitting in a document. @星海拾珠 felt most strongly about 01: agent logs and traces really do need to be kept in your own hands.

Once this stack is broken down, the model vendor is just 03. What will actually come up in implementation is who merges the permissions file, whether review time counts as cost, and how to store rationale that wasn’t written down. The seven layers look more like an ownership checklist and a feedback-loop checklist.

Original post and discussion on X: https://x.com/zachlloydtweets/status/2121318753711583254