# The Codexyo Production Standard

**Version 1.0 — July 2026**

This is Codexyo's engineering constitution. It defines what we mean when we say a system is ready for production, the gates every project must pass before we ship it, and the responsibilities we accept after deployment. It is published so that clients can hold us to it, engineers can build against it, and anyone evaluating us can check our claims against something concrete.

This document contains no marketing. If a sentence in it cannot change an engineering decision, it does not belong here.

---

## Why This Document Exists

Most AI systems die between the demo and production. Before writing a standard, we had to be precise about why.

**What separates a demo from a production system?** A demo works in the *presence* of everything: its creator at the keyboard, curated inputs, one user, low stakes, and an audience prepared to forgive. A production system works in the *absence* of those things. The difference is not polish. It is a different engineering problem.

**Why do AI projects fail after deployment?** Because the properties that make AI impressive in a demo are the same properties that make it dangerous under real inputs: flexibility, open-endedness, fluency. Teams ship the model's best behavior and discover its worst behavior in front of customers. Nobody defined what "good enough" meant measurably, so nobody knew it hadn't been reached.

**Why do prototypes rarely survive real users?** Because real users are an absence of cooperation: malformed input, adversarial input, impatient retries, volumes nobody predicted, and questions the happy path never imagined. A prototype encodes the builder's assumptions; production is the systematic removal of the builder's assumptions.

**What disciplines are usually skipped?** The ones with no demo value: failure-path design, evaluation baselines, monitoring, cost ceilings, documentation, rollback, handover. They are skipped because they are invisible in the meeting where the project gets applause. They are decisive in the months after.

**What does production mean beyond uptime?** Uptime measures whether a system is running. Production is whether it is running *correctly, affordably, observably, and without requiring its builders*. A system can have perfect uptime and still be a liability nobody understands, nobody can change, and nobody can afford.

**What responsibility does an engineering partner have after deployment?** To have built the system so that the client's dependence on the partner is a choice, not a trap. Everything in Part 4 follows from that answer.

---

## Part 1 — Our Definition of Production

> **A system is in production when it survives four absences:**
> **the absence of its builders, the absence of well-behaved input, the absence of the happy path, and the absence of attention.**

1. **The absence of its builders.** The system can be operated, deployed, debugged, and changed by a competent engineer who has never met us, using only what we handed over.
2. **The absence of well-behaved input.** The system behaves in a defined way when the input is malformed, hostile, enormous, empty, or simply strange. That includes inputs designed to manipulate an AI component.
3. **The absence of the happy path.** Every dependency the system relies on, from APIs and models to databases and the network, has a defined behavior for the day it fails, and the system can be rolled back from any change.
4. **The absence of attention.** The system reports its own condition (correctness, drift, cost, and failure) loudly enough that problems reach the responsible humans before they reach the client's customers.

A demo survives none of these absences. Most "launched" software survives one or two. We do not call a system production until it survives all four, and every gate, principle, and scorecard in this document traces back to one of them.

This definition is deliberately checkable. Each absence can be tested, and Part 8 defines how.

---

## Part 2 — The Production Principles

Twelve principles. Each exists because of a specific, recurring failure we refuse to repeat.

### 1. Production is a condition, not a milestone.
**Why:** "Launch" language makes production sound like a date. It is a state the system must continuously satisfy. A system can fall out of production without ever going down.
**How it changes decisions:** We plan for the months after deployment inside the original scope: monitoring, cost review, drift checks. "Done" is defined by the four absences, not the calendar.
**Failure prevented:** Launch-and-abandon: the system that worked on day one and decayed silently for six months.

### 2. The demo is a hypothesis, not a small version of the product.
**Why:** A demo proves something is possible. Production proves it is durable. The distance between them is the majority of the engineering, not the remainder.
**How it changes decisions:** Estimates and scopes treat post-demo work as the main body of the project. We never present a working prototype as "80% done," because it is usually the first 20%.
**Failure prevented:** The industry's signature failure: budgets exhausted on the demo, with production quietly reclassified as "phase two" and never funded.

### 3. Every system must survive its builders' absence.
**Why:** A system only its creators can run is not an asset; it is a dependency with our faces on it. Our positioning says "built so you don't need us." This principle is what makes that sentence true.
**How it changes decisions:** Documentation, runbooks, and handover are engineered deliverables with acceptance criteria, built during the project, not after it. Clever code that resists explanation is rewritten.
**Failure prevented:** Key-person risk, vendor lock-in, and the orphaned system whose only manual is an ex-contractor's memory.

### 4. A system that cannot report its own condition is not finished.
**Why:** Between "working" and "known to be working" sits every silent failure that ever reached a customer. For AI systems this gap is wider: an LLM feature can degrade for weeks while returning HTTP 200.
**How it changes decisions:** Observability is scoped as a feature: what the system measures about itself, what it alerts on, and who is woken. For AI components, quality metrics are monitored, not just uptime.
**Failure prevented:** Discovering the outage from the client's customers, and the slow-motion version: quality drift nobody was measuring.

### 5. Failure is a design input, not an incident category.
**Why:** Every dependency will fail: the model API, the database, the third-party service, the network. The only question is whether the system's behavior at that moment was designed or improvised.
**How it changes decisions:** For every external dependency and every model call, the failure behavior is specified before the integration is written: timeout, retry, fallback, degraded mode, honest error.
**Failure prevented:** Undefined behavior at the worst possible time, and the cascade where one dependency's bad day becomes the whole system's outage.

### 6. Cost is an engineering property.
**Why:** AI systems are the first mainstream software whose marginal cost per request can quietly exceed its marginal value. A system that works perfectly and costs unpredictably is a production failure.
**How it changes decisions:** Cost per request is measured in development, bounded in production (budgets, ceilings, alerts), and reported to the client as an explicit system property alongside latency and reliability.
**Failure prevented:** The invoice that kills the project after launch, the most common cause of death for technically successful AI features.

### 7. Nondeterminism is not an excuse.
**Why:** AI outputs vary; engineering standards do not. "The model is unpredictable" explains variance. It does not excuse shipping behavior nobody measured.
**How it changes decisions:** Before an AI feature is built, its acceptable behavior is defined measurably: an evaluation set, scoring criteria, and a baseline. The feature ships when it clears the baseline, and no change ships that regresses it.
**Failure prevented:** Vibes-based AI quality. "It seemed good when we tried it" is how hallucinations reach customers.

### 8. The boring path is the deliverable.
**Why:** Clients pay for outcomes, not for our curiosity. Novelty is a budget, and it must be spent where the problem is genuinely novel, never on infrastructure choices that proven components solve.
**How it changes decisions:** Default to established, well-documented components everywhere the problem is ordinary. Every exotic choice must be justified in the decision log by a problem property, not a preference.
**Failure prevented:** Résumé-driven architecture: the system that is fascinating to its builders and unmaintainable by everyone else.

### 9. Understandable before scalable.
**Why:** Complexity added for imaginary future load is a cost paid every day by everyone who touches the system. Most systems die from being incomprehensible long before they die from being popular.
**How it changes decisions:** We build for the load that exists and the load the client's plan credibly implies, with measured scaling paths identified, not pre-built. Architecture reviews ask "can a stranger follow this?" before "will this scale?"
**Failure prevented:** The distributed system that serves forty users, and the complexity debt that makes every future change slower.

### 10. Security is scope, not a surcharge.
**Why:** Data handling, access control, and secret management are properties of the system, not an optional add-on the client must know to ask for. AI adds new surfaces that traditional checklists miss: prompt injection, and data leakage through model inputs and outputs.
**How it changes decisions:** Every scope includes a security review appropriate to the system's stakes; every AI input path is treated as untrusted; secrets live in the client's controlled store from day one, never in ours.
**Failure prevented:** The breach that was nobody's line item, and the AI feature that leaked what it was fed.

### 11. A change we cannot undo is a change we are not ready to make.
**Why:** Confidence in deployment comes from reversibility, not from optimism. This applies to code, infrastructure, data migrations, and, often forgotten, prompts and model versions.
**How it changes decisions:** Every deployment has a rollback path that has actually been exercised. Prompt and model changes are versioned and reversible like code. Destructive migrations get rehearsals and backups whose restore has been tested.
**Failure prevented:** The one-way door: an irreversible deployment discovered to be wrong, with no path back and customers watching.

### 12. We do not ship what we would not operate.
**Why:** The final quality gate is personal: would we accept being on call for this system, tonight? If the answer is no, the system is not done, whatever the checklist says.
**How it changes decisions:** This is the tiebreaker for every judgment call in this document. It is why the gates in Part 3 exist and why Part 6 exists.
**Failure prevented:** Shipping a problem and calling it a product.

---

## Part 3 — The Production Gates

Nine gates, grouped by the absence they defend. A project ships when every gate holds. Each gate names its evidence, the artifact that proves it passed, because a gate without evidence is an opinion.

### Defending against the absence of the builders

**Gate 1 — Documentation & Runbooks.**
*Why it matters:* Knowledge that lives only in our heads is a liability the client paid for.
*Evidence:* Architecture documentation with a current diagram; a runbook covering routine operations and the known failure modes with responses; setup-to-deploy instructions executed successfully by someone who didn't write them.
*If it fails:* The system does not ship. Documentation is a deliverable with acceptance criteria, not a courtesy.

**Gate 2 — Ownership & Handover.**
*Why it matters:* Ownership ambiguity discovered at the end of a project is a hostage situation. We refuse to be the kind of vendor that benefits from one.
*Evidence:* Repositories, infrastructure, and credentials verifiably under client control (per Part 4); a completed handover session, recorded; the Stranger Test (Part 4) passed.
*If it fails:* The project is not complete, whatever else works.

### Defending against the absence of well-behaved input

**Gate 3 — Code Quality & Testing.**
*Why it matters:* Tests are the only scalable defense against inputs and states nobody imagined in the demo.
*Evidence:* Automated tests covering the system's critical paths and its ugliest inputs (malformed, empty, oversized, concurrent); tests run in CI and gate every deployment; a written statement of what is *not* covered and why.
*If it fails:* Untested critical paths are either tested or explicitly accepted in writing by the client as a known risk. Silence is not acceptance.

**Gate 4 — Security & Data Handling.**
*Why it matters:* The client's data and their customers' trust transit the system; both are unrecoverable once lost.
*Evidence:* A security review scaled to the system's stakes; untrusted-input handling on every external path, including prompt-injection handling on every AI path; secrets in a client-controlled store; a written account of what data the system stores, sends to third parties (including model providers), and for how long.
*If it fails:* Launch blocks until resolved. Security findings do not ride along as post-launch tickets.

**Gate 5 — AI Evaluation** (applies to every system with a model in it; detailed standard in Part 5).
*Why it matters:* An AI feature without an evaluation baseline has no definition of working.
*Evidence:* An evaluation set representing real and hostile usage; a scored baseline the client has seen; eval runs gating prompt, model, and logic changes; documented known failure modes with rates.
*If it fails:* The AI component does not ship on anecdotes. No baseline, no launch.

### Defending against the absence of the happy path

**Gate 6 — Failure & Recovery.**
*Why it matters:* Principle 5: failure behavior is designed, not improvised.
*Evidence:* A dependency map with a defined failure behavior for each dependency; degraded modes specified for the failures that matter; backups whose restore has actually been performed, not assumed.
*If it fails:* Any dependency with undefined failure behavior gets one before ship, even if that behavior is an honest, designed error state.

**Gate 7 — Deployment & Rollback.**
*Why it matters:* Principle 11: reversibility is where deployment confidence comes from.
*Evidence:* Repeatable, documented deployments (no artisanal servers); CI running tests and evals on every change; a rollback procedure that has been exercised at least once, including for prompts and model versions.
*If it fails:* No irreversible deployment ships. A rollback that has never been rehearsed is a rumor.

### Defending against the absence of attention

**Gate 8 — Observability.**
*Why it matters:* Principle 4: unknown-state systems fail in front of customers.
*Evidence:* Health, error, and latency monitoring with alerts routed to named humans; for AI components, quality and drift signals monitored, not just uptime; logs sufficient to reconstruct what happened when something goes wrong.
*If it fails:* A system nobody will hear failing does not ship.

**Gate 9 — Cost & Performance.**
*Why it matters:* Principle 6: an unaffordable system is a failed system with good uptime.
*Evidence:* Measured cost per request/user/period under realistic load; configured ceilings or alerts on runaway cost, especially model spend; latency measured against targets the client agreed to.
*If it fails:* Unbounded cost exposure blocks launch. "We'll watch the bill" is not a ceiling.

---

## Part 4 — The Client Ownership Standard

"Built so you don't need us," made operational. These are defaults on every engagement, stated in the contract, not favors negotiated at the end.

- **Repositories** live in the client's organization from day one. We are contributors to their repos, never landlords of ours. Version history is theirs from the first commit.
- **Infrastructure** runs in the client's accounts: their cloud, their billing, their control plane. Where we operate it, we do so with access they grant and can revoke; we never own the account a client's system lives in.
- **Credentials and secrets** live in a client-controlled secret store. We hold access, not custody. On exit, revoking us is a settings change, not a negotiation.
- **Documentation** lives in the client's systems, current as of handover, and written for the next engineer, not as a monument to the project.
- **Runbooks** cover routine operations, deployments, the known failure modes and their responses, and where to look first when something is wrong. A runbook that has never been followed by a stranger is a draft.
- **Deployment instructions** are complete enough that a competent engineer can deploy without contacting us. If a step lives only in our shell history, the gate is not passed.
- **Architecture diagrams** are handover artifacts kept current through the project. The diagram matches the system on the day we leave, not the day we started.
- **Knowledge transfer** is a scheduled, recorded working session (or more than one, for larger systems): architecture, decisions and their reasons, operations, and known sharp edges, with the client's team driving, not watching.
- **The decision log** is handed over with the system: every significant technical decision, its alternatives, and why. The next engineer inherits our reasoning, not just our conclusions.

**Exit readiness — the Stranger Test.** A system passes when a competent engineer who has never met us can, using only the handover materials: understand the architecture, run the system locally or in staging, deploy a small change, and respond to a simulated failure using the runbook. That is the standard "you could survive without us" is measured against. We build to pass it on every project, and we say plainly that passing it is what makes clients comfortable staying.

**What we ask in return:** ownership is bilateral. The client names an owner on their side, someone who accepts the handover, holds the credentials, and can make decisions. We can build a system that doesn't need us; we cannot make someone own it.

---

## Part 5 — The AI Standard

What "production" additionally requires when the system contains a model. These are standards, not implementations; the techniques change, the obligations don't.

1. **Evaluation before construction.** Acceptable behavior is defined measurably, with an evaluation set and scoring criteria, before the feature is built. If we cannot define what good output looks like, we are not ready to build the feature, and we say so.
2. **Evals gate every change.** Prompt edits, model swaps, retrieval changes, logic changes: anything that can alter model-facing behavior runs the evaluation suite before it ships. A change that regresses the baseline does not deploy on the promise that it "seems better."
3. **Prompts are code.** Versioned, reviewed, tested, and rollbackable. A prompt edited live in a dashboard is an unversioned production change, and we do not make unversioned production changes.
4. **Every model call has a defined failure behavior.** Timeout, refusal, malformed output, provider outage. Each has a designed response: retry, fallback, degraded mode, or an honest error. The user never sees the raw absence of a plan.
5. **The system is allowed to say "I don't know."** Hallucination mitigation starts with permission to abstain. Grounding in real data, constrained output formats, and honest uncertainty in the interface are design requirements. A confident wrong answer is a worse failure than a visible refusal.
6. **Human review where stakes exceed model reliability.** Where an output can cause irreversible harm (money moved, records changed, commitments made to customers), a human is in the loop until the measured error rate justifies removing them, with the client making that call on evidence, not on our enthusiasm.
7. **Quality is monitored, not assumed.** Uptime monitoring cannot see a model getting worse. Production AI systems carry quality signals with defined thresholds for concern: sampled scoring, drift indicators, user-feedback loops.
8. **Model upgrades are migrations.** Provider model changes and deprecations are planned for: the system is built so the model can be swapped, and every swap runs the full evaluation baseline before and after. No silent upgrades.
9. **Guardrails face both directions.** Inbound: untrusted input handling and prompt-injection defense on every path where user content meets a model. Outbound: output checked against the harms that matter for this system before it reaches a user or triggers an action.
10. **Cost per request is a design constraint.** Known during development, bounded in production, alerted on deviation. Token spend is engineering data, not a surprise on the invoice.
11. **Known failure modes are disclosed.** Every AI system ships with a written, plain-language account of what the model gets wrong, roughly how often, and what the system does about it. Clients get the real error profile. The alternative is that they discover it themselves, in front of their customers, having been told it "just works."

---

## Part 6 — What We Refuse to Ship

Launch blockers. When one of these is true, we recommend delaying. If the date cannot move, we cut scope rather than cut the standard. A smaller system that holds beats a complete system that doesn't.

We do not ship a system with:

1. **No rollback path**, or one that exists on paper and has never been exercised.
2. **No monitoring**: a system nobody will hear failing.
3. **An unresolved critical security finding**: security findings do not become post-launch tickets.
4. **No evaluation baseline for an AI component**: anecdotes are not a launch criterion.
5. **No runbook or handover-grade documentation**: shipping it would manufacture dependence on us, which our own standard forbids.
6. **Ambiguous ownership**: if it is unclear who owns the repos, the infrastructure, or the credentials, that is resolved before launch, not after.
7. **Untested restore**: a backup that has never been restored is a hope, not a backup.
8. **Unbounded cost exposure**: no ceilings or alerts on spend that can run away, especially model spend.
9. **Undefined success metrics**: if nobody has said what the system must do to be judged working, "working" will be decided by whoever is angriest later.
10. **Critical-path behavior nobody tested**: untested paths either get tests or get the client's explicit, written acceptance of the risk.

**How a delay is handled:** it is raised the moment we see it coming, not the week of launch, with a specific statement of what is missing, what it risks, and the options: move the date, cut the scope, or accept the named risk in writing. The client decides with full information. What we do not offer is the fourth option most of the industry sells: shipping it quietly and hoping.

---

## Part 7 — The Client Experience

The operating model a client can expect. This is how the Standard feels from the other side of the table, described plainly. It is a description of process, not a pitch.

- **Diagnosis before construction.** Every engagement starts with discovery: the business problem, the constraints, what already exists, and whether the thing being asked for should be built at all. "Don't build this," including "you don't need AI for this," is a standing possible outcome, stated at the start.
- **Bounded scope, agreed in writing.** What we are building, what we are explicitly not building, what it costs, and what "done" means, before work begins. Changes are negotiated in the open, never absorbed silently or invoiced ambiguously.
- **Working software on a regular cadence.** Progress is demonstrated with the system running, weekly in most engagements, not narrated in status reports. If a week produced nothing demonstrable, we say that, and why.
- **Decisions are shared and logged.** Significant technical decisions come with the trade-offs attached: what we recommend, what it costs, what the alternatives were. The decision log is the client's from day one.
- **Risks are named unprompted.** Anything that threatens the date, the budget, or the system's behavior is raised when we see it, with options. Bad news travels fastest; the client is never the last to know.
- **Ownership from the first commit.** Repos, infrastructure, and credentials are set up under client control at the start (Part 4), not transferred at the end. There is no moment in the engagement when the client does not own their system.
- **The gates are visible.** Clients see the Production Gates and the scorecard against their own project. "Production-ready" arrives as evidence, not as an announcement.
- **Handover is a scheduled event, not a farewell.** The knowledge-transfer sessions, the Stranger Test, and the runbook walk-through are in the plan from the first week.

**What we ask of clients:** a named decision-maker, honest access to the people who understand the problem, and responses on the decisions that gate the work. The cadence above is a two-party system; we hold up our side and say so when the other side slips.

---

## Part 8 — The Production Scorecard

Every project is scored against the four absences before ship, and the scorecard is shared with the client. Three grades per absence. No arbitrary numbers, because a "7.5" tells nobody what to fix.

**Grades:** **Holds** (the standard) · **Conditional** (ships only with the gaps named and accepted in writing) · **Demo-grade** (does not ship).

### Absence 1 — Survives the builders' absence
- **Holds:** Passes the Stranger Test. Documentation, runbooks, decision log, and handover complete; ownership fully under client control; a competent stranger could operate and change the system tomorrow.
- **Conditional:** A stranger could operate the system but specific gaps are named (a subsystem documented thinly, a transfer session pending), each with an owner and a date.
- **Demo-grade:** Operating the system requires us. Undocumented knowledge, our accounts, our credentials, our memory.

### Absence 2 — Survives ill-behaved input
- **Holds:** Critical paths tested including hostile and malformed input; every AI input path treats content as untrusted; security review complete; the eval set includes adversarial cases.
- **Conditional:** Core paths tested; specific untested or unhardened paths are named and accepted in writing as known risks.
- **Demo-grade:** The system has only ever seen the inputs its builders gave it.

### Absence 3 — Survives the loss of the happy path
- **Holds:** Every dependency has designed failure behavior; degraded modes work; rollback exercised; restore tested; model-provider failure has a plan.
- **Conditional:** The failures most likely to occur are handled; named dependencies carry accepted, documented risk.
- **Demo-grade:** Failure behavior is whatever happens. Rollback is a theory. The backup has never been restored.

### Absence 4 — Survives inattention
- **Holds:** Monitoring and alerting route to named humans; AI quality and drift are watched, not assumed; cost is measured, bounded, and alerted; logs can reconstruct an incident.
- **Conditional:** Availability and errors are monitored; named gaps (e.g., drift signals pending) have owners and dates.
- **Demo-grade:** The system's condition is unknown between the moments someone happens to look. The first alarm is a customer.

**The shipping rule:** nothing ships below Conditional on any absence, and Conditional always means *named, written, client-accepted* gaps, never silent ones. The gap list is a work list, not a shrug.

**Second use:** the scorecard applies to systems we didn't build. It is the assessment instrument for existing systems and stalled pilots: the honest answer to "how far from production are we?"

---

## Part 9 — Red Team

We attacked this document before publishing it. The attacks and their answers stay in, because a standard that hasn't been argued with is a poster.

**"Another agency could copy this in an afternoon."**
The words, yes. The practice is the moat, and the document is built to make faking it expensive: nearly every commitment produces a client-visible artifact. Repos in the client's org on day one, a shared scorecard, an exercised rollback, a recorded handover, a passed Stranger Test. An agency that copies the text inherits the obligations, and several of them (client-owned everything, exit readiness as a deliverable, disclosed error profiles) run directly against the dependence-based revenue model most agencies live on. Copying our standard while keeping their business model produces a visible contradiction on the first engagement. We accept the residual risk that good firms adopt these practices honestly; if this document nudges the industry's baseline upward, that is the reputation working.

**"Is any of this just a generic checklist wearing a nicer coat?"**
The gates overlap with good engineering practice everywhere. They must, or they'd be wrong. What is ours: the four-absences definition that organizes everything, the Stranger Test as a concrete exit-readiness bar, the scorecard's three-grade/no-numbers design, the shipping rule that Conditional means written client acceptance, and the AI standard's insistence that evals precede construction. Generic checklists list virtues; this document names the evidence for each and what blocks when it's missing.

**"Is anything here unverifiable?"**
Each gate names its artifact precisely so a client can ask for it. Where a claim resisted evidence, we cut it or converted it. That is why this document promises exercised rollbacks rather than "reliability," and disclosed error profiles rather than "accuracy."

**"Is this realistic for two people?"**
This was the sharpest attack, and it forced three honest positions into the document. First: the Standard is enforced by *scope selection*, not heroics. It is precisely why we take bounded engagements and refuse work we cannot bring through the gates; a firm that takes everything cannot have a standard. Second: we do not pretend to capabilities headcount provides. There is no 24/7 operations center here, and support terms in every engagement say exactly what we provide and what we don't, rather than implying a bench that doesn't exist. Third: the two-person reality is why Part 4 exists at full strength. A large vendor can *afford* to make clients dependent; we cannot and should not, so we engineered the standard where our absence is survivable. The constraint built the discipline.

**"Ten years from now, will this embarrass you?"**
The techniques will age. Today's eval tooling and model-provider quirks will read like period detail. The four absences will not, because they describe what production has always meant, including before AI: systems that outlive their builders' presence, their inputs' goodwill, their dependencies' uptime, and their operators' attention. Version 1.0 will be revised; the definition is built to hold.

---

*The Codexyo Production Standard, version 1.0. Maintained by the founders; revisions are versioned and public. Clients are invited to hold us to any line of it.*

*Production is not a deployment event. It is a standard of engineering responsibility.*