pystamps/docs/native-cli.html at main · ESA-PhiLab/pystamps · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>Native Rust CLI - pySTAMPS Docs</title>
  <link rel="stylesheet" href="assets/styles.css" />
  <script defer src="assets/scripts.js"></script>
</head>
<body>
<div class="site">
  <aside class="sidebar">
    <div class="logo"><img class="brand-logo" src="assets/pystamps-logo.svg" alt="pySTAMPS logo" /><h1>pySTAMPS Docs</h1></div>
    <small>Rust execution and validation commands.</small>
    <nav>
      <div class="nav-group"><h2>Guide</h2><ul class="nav-list"><li><a href="index.html">Introduction</a></li><li><a href="stages.html">Stages and Code Paths</a></li><li><a href="pipeline-science-guide.html">Pipeline Guide</a></li><li><a href="getting-started.html">Getting Started</a></li><li><a href="installation.html">Installation</a></li><li><a href="quickstart.html">Quick Start</a></li><li><a class="active" href="native-cli.html">Native Rust CLI</a></li><li><a href="architecture.html">Architecture</a></li><li><a href="configuration.html">Configuration</a></li><li><a href="usage.html">Usage</a></li><li><a href="testing.html">Testing</a></li><li><a href="verification.html">Verification</a></li></ul></div>
      <div class="nav-group"><h2>Reference</h2><ul class="nav-list"><li><a href="function-reference.html">Function Reference</a></li><li><a href="api/pystamps.html">Package API</a></li><li><a href="api/cli.html">cli</a></li></ul></div>
    </nav>
  </aside>
  <main class="content">
    <div class="breadcrumb">Home / Native Rust CLI</div>

    <section class="hero">
      <span class="badge">Native execution</span>
      <h1>Run the Rust pipeline directly when you want the fastest native path.</h1>
      <p>The Python CLI remains the general entrypoint. The standalone <span class="code-inline">pystamps-native</span> executable is the direct Rust runner for native-only stages, coverage checks, and full-chain validation.</p>
      <div class="hero-actions">
        <a class="button-link" href="#full-chain">Run full native chain</a>
        <a class="button-link secondary" href="#stage-debug">Debug one stage</a>
        <a class="button-link secondary" href="stages.html">Read stage details</a>
      </div>
    </section>

    <section class="card">
      <h2>Build the native binary</h2>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">cargo build --release -p pystamps-core --bin pystamps-native</code></pre>
      <p>Wheel installs can also expose <span class="code-inline">pystamps-native</span> as a console command. Source checkouts can run <span class="code-inline">target/release/pystamps-native</span> after the build above.</p>
    </section>

    <section class="diagram-card">
      <h2>Native Stage Coverage</h2>
      <p>The native runner covers patch and merged scopes directly. Stage 5 appears twice because it has both patch promotion and merged aggregation behavior.</p>
      <img class="doc-figure" src="assets/pystamps-stage-map.svg" alt="Native pySTAMPS stage coverage map" />
    </section>

    <section id="full-chain" class="card">
      <h2>Run native stages 1 through 8</h2>
      <p>Always run on a writable copy. The command writes stage artifacts into the dataset tree.</p>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">export SOURCE_DATASET=/path/to/original_dataset
export RUN_DATASET=/path/to/native_run_dataset

rm -rf "$RUN_DATASET"
cp -a "$SOURCE_DATASET" "$RUN_DATASET"

target/release/pystamps-native run \
  --native-only \
  --dataset "$RUN_DATASET" \
  --start-step 1 \
  --end-step 8 \
  --backend native \
  --stage2-kernel-backend native \
  --cpu-workers 0 \
  --stage2-native-threads 0</code></pre>
      <ul>
        <li><span class="code-inline">--native-only</span> requires <span class="code-inline">--backend native</span> and <span class="code-inline">--stage2-kernel-backend native</span>.</li>
        <li><span class="code-inline">--cpu-workers 0</span> and <span class="code-inline">--stage2-native-threads 0</span> use all CPUs visible to the process.</li>
        <li>Set positive worker values when you need reproducible resource limits.</li>
      </ul>
    </section>

    <section class="card">
      <h2>Preview or inspect coverage</h2>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">target/release/pystamps-native run \
  --dataset "$RUN_DATASET" \
  --start-step 1 \
  --end-step 8 \
  --dry-run</code></pre>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">target/release/pystamps-native coverage --start-step 1 --end-step 8</code></pre>
      <p>Both commands print JSON. Dry-run reports planned stage work; coverage reports which stage scopes are Rust-driven, native-executable, and parity-certified.</p>
    </section>

    <section id="stage-debug" class="card">
      <h2>Debug one stage</h2>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">target/release/pystamps-native stage 1 --patch "$RUN_DATASET/PATCH_1"
target/release/pystamps-native stage 2 --patch "$RUN_DATASET/PATCH_1"
target/release/pystamps-native stage 5 --patch "$RUN_DATASET/PATCH_1"
target/release/pystamps-native stage 5 --dataset "$RUN_DATASET"
target/release/pystamps-native stage 8 --dataset "$RUN_DATASET"</code></pre>
      <p>Stages 1 through 5 can operate on patch data where appropriate. Stage 5 also has a merged dataset command, and stages 6 through 8 run at the dataset root.</p>
      <div class="table-wrap">
        <table>
          <thead><tr><th>Stage</th><th>Native target</th><th>Command shape</th></tr></thead>
          <tbody>
            <tr><td>1-4</td><td>Patch</td><td><code>stage N --patch "$RUN_DATASET/PATCH_1"</code></td></tr>
            <tr><td>5</td><td>Patch or merged</td><td><code>stage 5 --patch ...</code> or <code>stage 5 --dataset ...</code></td></tr>
            <tr><td>6-8</td><td>Merged</td><td><code>stage N --dataset "$RUN_DATASET"</code></td></tr>
          </tbody>
        </table>
      </div>
    </section>

    <section class="card">
      <h2>Full native validation gate</h2>
      <p>This is the release-style local gate for native execution. It builds the release binary, prepares a clean run copy, executes the chain, compares outputs, and writes reports.</p>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">make native-full-chain-verify</code></pre>
      <pre><button class="copy-btn">Copy</button><code class="language-bash">make native-full-chain-verify \
  DATASET=/path/to/source_dataset \
  GOLDEN=/path/to/reference_dataset \
  RUN=/path/to/clean_run_dataset \
  THREADS=8</code></pre>
      <p>Reports are written under <span class="code-inline">RUN/_native_gate_reports/</span>, including coverage, native run output, timings, and verification results.</p>
    </section>
  </main>
</div>
</body>
</html>