<!--
SPDX-FileCopyrightText: © 2026 Tenstorrent USA, Inc.
SPDX-License-Identifier: Apache-2.0
-->

# Bringup op-level tracing and parity tests

This directory contains a two-phase workflow for capturing per-op golden tensors
from a PyTorch model and replaying them against TTNN to check parity (PCC).

- **Phase 1** (`phase1_record_ops.py`): run the model, hook the interesting leaf modules, and record a `manifest.json` plus a directory of `.pt` tensor artifacts.
- **Phase 2** (`phase2_generate_tests.py` + `tracer_test_harness.py`): read the manifest, replay each op on a TT device, and compare against the recorded golden tensors. The generated pytest file imports the harness from this directory (via `sys.path`) rather than copying it, so there is a single source of truth for the harness.
- `tracer_op_specs.py`: a dependency-light (stdlib-only) registry of the op kinds the tracer records, their required `params`, and which kinds the harness can replay. Both the validator and the harness import it so they agree on op metadata.
- `validate_trace_manifest.py`: a fail-fast preflight that validates a manifest (schema, artifact existence, shape consistency) before launching a test run. The CLI is a thin wrapper; the validation core lives in `trace_manifest_validation.py`.

## Artifact path contract

All artifact paths in a manifest (`in_path`, `out_path`, `w_path`, `b_path`) are **resolved relative to the directory that contains the manifest file**.

- Relative paths (the default, e.g. `tensors/00000_conv_in.pt`) are resolved as `<manifest_dir>/<path>`.
- Absolute paths are used verbatim. They are supported as an unambiguous escape hatch, but manifests produced by Phase 1 always use relative paths so an artifact directory can be moved or copied alongside its manifest.

## Manifest schema

```jsonc
{
  "input_shape": [B, C, H, W],   // input to the traced forward pass
  "num_records": N,              // must equal len(records)
  "records": [
    {
      "idx": 0,                  // must match the record's position in the list
      "name": "encoder.block1.conv1",
      "kind": "Conv2d",          // module type recorded by the tracer
      "params": { /* op-specific args, e.g. Conv2d kernel/stride/... */ },
      "in_shape":  [n, c, h, w], // 4D NCHW
      "out_shape": [n, c, h, w], // 4D NCHW
      "in_path":  "tensors/00000_encoder.block1.conv1_in.pt",
      "out_path": "tensors/00000_encoder.block1.conv1_out.pt",
      "w_path":   "tensors/00000_encoder.block1.conv1_w.pt",  // or null
      "b_path":   "tensors/00000_encoder.block1.conv1_b.pt"   // or null
    }
  ]
}
```

## Canonical example manifest

The following `manifest.json` sits next to a `tensors/` directory. All artifact paths are manifest-relative:

```
bringup/artifacts/phase1/
├── manifest.json
└── tensors/
    ├── 00000_conv_in.pt
    ├── 00000_conv_out.pt
    ├── 00000_conv_w.pt
    ├── 00000_conv_b.pt
    ├── 00001_relu_in.pt
    └── 00001_relu_out.pt
```

```json
{
    "input_shape": [1, 3, 256, 256],
    "num_records": 109,
    "records"    : [
        {
            "idx"      : 0,
            "name"     : "input_proj",
            "kind"     : "Conv2d",
            "params"   : {
                "in_channels" : 3,
                "out_channels": 64,
                "kernel_size" : [1, 1],
                "stride"      : [1, 1],
                "padding"     : [0, 0],
                "dilation"    : [1, 1],
                "groups"      : 1,
                "bias"        : false
            },
            "in_shape" : [1, 3, 256, 256],
            "out_shape": [1, 64, 256, 256],
            "in_path"  : "tensors/00000_input_proj_in.pt",
            "out_path" : "tensors/00000_input_proj_out.pt",
            "w_path"   : "tensors/00000_input_proj_w.pt",
            "b_path"   : null
        }
    ]
}

```

## Usage

```bash
# Phase 1: record ops + tensors (manifest written to bringup/artifacts/phase1/manifest.json)
python tools/bringup/phase1_record_ops.py --input-shape 1 3 256 256

# Validate the manifest before running tests
python tools/bringup/validate_trace_manifest.py --manifest bringup/artifacts/phase1/manifest.json

# Phase 2: generate a pytest file from the manifest. The generated test imports
# tracer_test_harness from tools/bringup via sys.path (no files are copied).
python tools/bringup/phase2_generate_tests.py \
    --manifest bringup/artifacts/phase1/manifest.json \
    --out-test bringup/artifacts/phase2/test_automatic_execution.py

# Phase 2: execute tests
python -m pytest --capture=fd --log-level=DEBUG --tb=short bringup/artifacts/phase2/test_automatic_execution.py
```
