> ## Documentation Index
> Fetch the complete documentation index at: https://hubify.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Reproducibility Levels

> The 0-5 reproducibility ladder every lab.yaml reports against, and what evidence each level actually requires.

# Reproducibility Levels

Reproducibility is layered, not binary. A view is not a reproduction. A cloned environment is not a verified result. A partial reproduction is not a full one. Every [lab manifest](/docs/concepts/lab-manifest) reports exactly how far it's gotten, level by level, in `reproducibility.levels`.

<Note>
  Levels are honestly reported, not automatically computed. There is no scoring function that inspects a lab's artifacts and assigns a level — a captain (or an agent under their direction) sets each level's `status` in `lab.yaml` based on real, checkable evidence.
</Note>

## The six levels

| Level | Name                      | What it means                                     |
| ----- | ------------------------- | ------------------------------------------------- |
| 0     | Archived                  | Code and data are preserved and browsable.        |
| 1     | Environment reconstructed | The environment builds and runs.                  |
| 2     | Artifacts verified        | Committed outputs match their manifest.           |
| 3     | Lightweight reproduction  | A reduced run reproduces the shape of the result. |
| 4     | Partial computational     | A real subset of the full computation reproduces. |
| 5     | Full reproduction         | The full computation reproduces end to end.       |

Each level's `status` is one of three values:

| Status        | Meaning                                                           |
| ------------- | ----------------------------------------------------------------- |
| `complete`    | This level has been done and checked.                             |
| `in_progress` | Tooling or process for this level exists and has partial results. |
| `not_started` | Nothing has happened at this level yet.                           |

## Levels are independent, not cumulative

The headline `reproducibility.level` number and the per-level `status` map are both hand-authored, and they are allowed to disagree with a naive "ladder" reading. The `bigbounce` manifest is the concrete example: it reports headline `level: 2`, with Level 2 (`artifacts_verified`) `in_progress` — but Level 1 (`environment_reconstructed`) is still `not_started`:

```yaml theme={null}
0: { name: archived,                status: complete,    note: "Complete — 740 publication-critical files SHA-256 hashed at source commit 130cc061." }
1: { name: environment_reconstructed, status: not_started, note: "Not started as an independently verified step." }
2: { name: artifacts_verified,      status: in_progress, note: "Tooling proven — 740/740 files verified OK against the manifest, checked against source." }
```

That is intentional, not an authoring error. Level 2's artifact-hash verification was proven against the source repo directly; a from-scratch environment rebuild (Level 1) is simply a separate, not-yet-attempted verification step. Reporting it as `not_started` rather than quietly implying it via a higher headline level is the entire point of the honesty model — see the `note` field below.

## Evidence, by example

The registry manifests show what "evidence" concretely looks like at each level today:

<Steps>
  <Step title="Level 0 — Archived">
    `bigbounce` marks this `complete`: 740 publication-critical files SHA-256 hashed at a specific source commit (`130cc061`), recorded against `manifests.level0` (a JSON file-hash manifest). Evidence = a hash manifest checked into the repo, not a claim.
  </Step>

  <Step title="Level 1 — Environment reconstructed">
    Not yet attempted as an independently verified step on any lab in the registry today (`not_started` on `bigbounce`; `not_started` by definition on the spec-only `eti` and `hubify-self-improving` labs, which have no repo to reconstruct).
  </Step>

  <Step title="Level 2 — Artifacts verified">
    `bigbounce` marks this `in_progress`: the verification tooling (`tools/verify_artifacts.py`, invoked from the Reproduce sidepeek's Path A) is proven — 740/740 files verified OK against the Level 0 manifest, checked against source. "In progress" here means the tooling works and has been run successfully, not that every possible artifact category has been covered.
  </Step>

  <Step title="Levels 3-5 — Lightweight through full reproduction">
    `not_started` everywhere in the registry today. Level 5 (full computational reproduction) carries an explicit warning in every manifest: it requires explicit cost authorization before any compute runs, since reproducing the full computation means spending real GPU budget.
  </Step>
</Steps>

## The `note` field

`note` on each level entry is a Hubify-side extension to spec v0.1 — additive, and safe for any spec-v0.1-only consumer to ignore. It's free text, and today it drives the per-row copy on the site's reproducibility board (`ReproducibilityLevels` component). Use it to say *why* a status is what it is — "Not started" alone says nothing; "Not started — no source repo exists to snapshot" (the `eti` manifest's Level 0 note) tells a reader exactly what's missing.

## Where this is enforced

The [reproduce surface](/docs/concepts/reproduce) enforces one hard rule tied to this ladder: a freshly reproduced lab **always** starts every level at `not_started` and headline `level: 0`, regardless of what level the source lab had reached. Reproducibility describes what a specific copy has independently verified — it is never inherited.