Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
6d990d2
plan(fuzz): open the fuzz/oracle completeness remediation plan (sourc…
Nucs Jul 7, 2026
f11cf0c
fuzz(gate): B8 — verify invert(non-integer) crash guard, pin 5 regres…
Nucs Jul 7, 2026
6fc3be0
docs(fuzz): Phase 1 gate-semantics truth pass (D7 + D3a)
Nucs Jul 7, 2026
cb52e42
fuzz(oracle): close clip-complex + round dtype false-premise exclusio…
Nucs Jul 7, 2026
1bb5551
fuzz(oracle): matmul bool semiring + negstride/k0 edges + strided tra…
Nucs Jul 7, 2026
7418a26
fuzz(oracle): non-bool where cond + revive iscomplex/isreal wiring (G…
Nucs Jul 7, 2026
1c7c7e7
fix(math): np.convolve accumulates per NumPy *_dot dtype contract (co…
Nucs Jul 7, 2026
a4e79cd
fuzz(gate): B1-B7 — scope every over-broad MisalignedRegistry excuse …
Nucs Jul 7, 2026
07042d4
fuzz(oracle): reduce offset-view layouts + fix all/any Half+Complex o…
Nucs Jul 7, 2026
ab08600
fuzz(oracle): NaN-contract + strided/negstride sort/argsort coverage …
Nucs Jul 7, 2026
73d86df
fuzz(gate): B9 — per-corpus minimum case-count floors in RunCorpus
Nucs Jul 7, 2026
103cdf2
plan(fuzz): WS-BUGS status B1-B9 DONE + ledger L5/L6/L7 + L4 review-ack
Nucs Jul 7, 2026
04e62f2
fuzz(oracle): weave Char into where/logic/matmul/rounding/copyto + fi…
Nucs Jul 7, 2026
3af77d7
fuzz(oracle): revive dead E03 empty-base index recipe; V0/E03 join th…
Nucs Jul 7, 2026
be9177b
fuzz(gate): registry-tightness self-tests + truthful header + ledger …
Nucs Jul 7, 2026
9004f32
fuzz(oracle): decimal negative power + bounded decimal widening (G7, G8)
Nucs Jul 7, 2026
8e44c37
fuzz(oracle): widen the random fuzzer envelope — 13 dtypes, +reduce/a…
Nucs Jul 7, 2026
b730bcc
fuzz(oracle): six new error-parity specs incl. the un-crashed invert(…
Nucs Jul 7, 2026
a41a7a4
fuzz(oracle): metamorphic invariants widened to Half/Complex/Decimal/…
Nucs Jul 7, 2026
4f91e83
fuzz(oracle): G15 cross-dtype index setters + plan status reconciliation
Nucs Jul 8, 2026
f0c2a47
docs(fuzz): Phase 2 truth pass — counts, ledger rewrite, coverage-gap…
Nucs Jul 8, 2026
22e29ba
plan(fuzz): close the remediation — B10 sweep, resolution map, final …
Nucs Jul 8, 2026
bf86887
docs(oracle): add the differential-fuzz oracle skill (SKILL.md + 4 re…
Nucs Jul 19, 2026
e1e24de
docs(benchmark): add the NumPy-vs-NumSharp benchmark skill (SKILL.md …
Nucs Jul 19, 2026
7335eb9
fix(indexing): R3 — OOB heap write in the boolean-mask gather/scatter…
Nucs Jul 19, 2026
0dd29e0
test(io): isolate tofile/fromfile temp files per process — the two TF…
Nucs Jul 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -639,9 +639,9 @@ Proves every NDIter-backed op is **bit-identical** to NumPy 2.4.2 across the inp

```
test/oracle/ corpus generators (NumPy 2.4.2 — run by hand / nightly soak)
layout_catalog.py memory-layout builders (the "44 variations": 26 single + 9 pair + 5 where)
gen_oracle.py deterministic op matrices (astype/binary/unary/reduce/where/… — ~90 ops);
per-mode dtype axes widened to ALL_DTYPES; Char woven into every tier
layout_catalog.py memory-layout builders (the 40 variations: 26 single + 9 pair + 5 where)
gen_oracle.py deterministic op matrices (astype/binary/unary/reduce/where/… — ~159 ops);
per-mode dtype axes widened to ALL_DTYPES; Char woven into 18 tier files
via the uint16 proxy (char_tier, relabelled uint16->char)
gen_decimal_oracle.cs INDEPENDENT C# oracle for Decimal (no NumPy analog): naive scalar
System.Decimal math -> decimal_{unary,binary,reduce,scan,power,
Expand All @@ -661,7 +661,7 @@ test/NumSharp.UnitTest/Fuzz/ C# replay harness (no Python)
IndexOracleTests.cs index get/set differential gate (curated + dtype + seeded-random tiers)
MetamorphicTests.cs oracle-free invariants (round-trips / involutions / identities — no NumPy)
HarnessSelfTests.cs proves the harness has teeth (BitDiff detects value/NaN/-0 diffs; non-vacuous)
corpus/*.jsonl committed corpus (~68K cases / 40 tiers; op corpus ~53K incl. 3.7K Char woven + 579 Decimal), copied to test output via the csproj glob
corpus/*.jsonl committed corpus (~76K cases / 43 tiers; op corpus ~63K incl. 4.4K Char woven + 695 Decimal), copied to test output via the csproj glob
test/NumSharp.UnitTest/IO/ .npy/.npz format gate (no Python)
NpyOracleCorpus.cs opens npy_oracle.zip, rebuilds arrays from the manifest
NpyOracleTests.cs one [NpyOracle] test per claim: read / byte-exact write / header-only
Expand All @@ -679,9 +679,9 @@ test/NumSharp.UnitTest/IO/ .npy/.npz format gate (no Python)
```

- **Generators live in `test/oracle/`** and write the corpus into `test/NumSharp.UnitTest/Fuzz/corpus/` (path resolved relative to `test/oracle/`). CI replays the committed corpus, never the generators.
- **Three `FuzzMatrix` gates**: `FuzzCorpusTests` (the op corpus — ~53K cases across the tiers, checking dtype + shape + bytes + error parity; Char woven into every tier, 12 `Decimal*` tiers: unary/binary/reduce/scan/power/varstd/matmul/astype/stat/where/sort/manip), `IndexOracleTests` (the index oracle — `index_curated` 2,265 + `index_dtype` 104 + `index_random` 10,000; the advanced-indexing parity gate), and `MetamorphicTests` (12 NumPy-free invariants). A failing op case auto-shrinks to a 1-element repro.
- **Dtype coverage**: per-mode dtype axes widened toward `ALL_DTYPES`. **Char** (no NumPy dtype) is woven into every tier via the uint16 proxy (`gen_oracle.char_tier`, relabelled uint16→char). **Decimal** (no NumPy analog) rides an independent C# oracle (`gen_decimal_oracle.cs`, naive scalar `System.Decimal`). Verified Char/clip-bool bugs are carved from the green corpus and reproduced under `[OpenBugs]` (`OpenBugs.Char.cs`, `OpenBugs.DtypeCoverage.cs`) — NOT excused in `MisalignedRegistry`.
- **Regenerate** (deterministic; needs `numpy==2.4.2`): `python test/oracle/gen_oracle.py <mode>` (modes: `smoke astype_full binary divmod_power comparison unary reduce where place matmul bitwise unary_extra nanreduce scan stat logic modf manip sort tail params aliasing copyto errors`) + `python test/oracle/gen_index_oracle.py` (the `index_*` tiers) + `python test/oracle/fuzz_random.py 1234 2000 random_smoke.jsonl` + `dotnet run test/oracle/gen_decimal_oracle.cs` (the `decimal_*` tiers), then `dotnet build` (copies the corpus to test output).
- **Three `FuzzMatrix` gates**: `FuzzCorpusTests` (the op corpus — ~63K cases across the tiers, checking dtype + shape + bytes + error parity + per-file minimum-count floors; Char woven into 18 tier files, 12 `Decimal*` tiers: unary/binary/reduce/scan/power/varstd/matmul/astype/stat/where/sort/manip), `IndexOracleTests` (the index oracle — `index_curated` 2,273 + `index_dtype` 104 + `index_setter_dtype` 10 (cross-dtype cast-on-set) + `index_random` 10,000; the advanced-indexing parity gate), and `MetamorphicTests` (12 NumPy-free invariants incl. Half/Complex/Decimal/bool/char + strided views). A failing op case auto-shrinks to a 1-element repro.
- **Dtype coverage**: per-mode dtype axes widened toward `ALL_DTYPES`. **Char** (no NumPy dtype) is woven into 18 tier files via the uint16 proxy (`gen_oracle.char_tier`, relabelled uint16→char). **Decimal** (no NumPy analog) rides an independent C# oracle (`gen_decimal_oracle.cs`, naive scalar `System.Decimal`; incl. axis reductions, empty, negative int powers). Verified Char/clip-bool/round/dot bugs are carved from the green corpus and reproduced under `[OpenBugs]` (`OpenBugs.Char.cs`, `OpenBugs.DtypeCoverage.cs`, `OpenBugs.FuzzGaps.cs`) — NOT excused in `MisalignedRegistry`.
- **Regenerate** (deterministic; needs `numpy==2.4.2`): `python test/oracle/gen_oracle.py <mode>` (modes: `smoke astype_full binary divmod_power comparison unary reduce where place matmul rounding bitwise unary_extra nanreduce scan stat logic modf manip sort tail params aliasing copyto errors groupa`) + `python test/oracle/gen_index_oracle.py` (the `index_*` tiers) + `python test/oracle/fuzz_random.py 1234 2000 random_smoke.jsonl` + `dotnet run test/oracle/gen_decimal_oracle.cs` (the `decimal_*` tiers), then `dotnet build` (copies the corpus to test output).
- **Run the gate**: `dotnet test --filter "TestCategory=FuzzMatrix"`. Each case is bit-exact (pass), a documented difference in `MisalignedRegistry` (excused, never silent), or a failure (red). Full divergence ledger: `test/NumSharp.UnitTest/Fuzz/README.md`.

### The `.npy`/`.npz` format oracle (same philosophy, separate corpus)
Expand Down
94 changes: 94 additions & 0 deletions .claude/skills/benchmark/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
---
name: benchmark
description: >-
NumSharp's NumPy-vs-NumSharp performance harness — the op/dtype/N matrix (C# BenchmarkDotNet vs a
warm NumPy process) plus five appended subsystems (nditer, layout, operand, cast, fusion), all in
the NPY/NS convention. Use this whenever you add a benchmark for an np.* op, wire a C# benchmark to
its NumPy twin, run the official suite or a subset, read/interpret the ratio matrix or history
snapshots, add a whole subsystem, or debug a suspicious measurement (the Debug-taint 2x pitfall,
the InProcessEmit toolchain). Trigger on: "benchmark", "add a benchmark", "how fast is <op> vs
numpy", "run_benchmark.py", "BenchmarkDotNet", "NPY/NS ratio", "perf comparison", "benchmark
<op>", "benchmark-report", "history snapshot", "why is my timing 2x slow". Reach for it before
quoting any NumSharp-vs-NumPy speed number.
---

# NumSharp Benchmark Harness

The exhaustive guide is **`benchmark/CLAUDE.md`** (architecture, every suite, config, troubleshooting). This skill
is the distilled map + the actionable playbooks. Read `benchmark/CLAUDE.md` when you need depth beyond this.

## THE convention: NPY/NS (memorize this)

> **ratio = NumPy_ms / NumSharp_ms.** `>1` = NumSharp **faster**, `<1` = slower, `=1` = parity. **Higher is better.**

Used everywhere — matrices, geomeans, commit messages, every `*_sheet.py`. Icons: ✅ `≥1.0` · 🟡 `≥0.5` · 🟠 `≥0.2`
· 🔴 `<0.2`. (The legacy `run-benchmarks.ps1` prints the INVERSE NS/NPY — prefer NPY/NS for anything new.)

## THE pitfall: Debug taints timings ~2×

Ad-hoc `dotnet run file.cs` / `dotnet_run` (file-based apps) compile **both the script AND any `#:project`
NumSharp.Core in Debug** (`DebuggableAttribute(DisableOptimizations)`), which the JIT honors even over
`[MethodImpl(AggressiveOptimization)]`. Hand-written C# hot loops run ~2× slow; IL-emitted kernels look normal.
**Every timing script MUST run `dotnet run -c Release - < script.cs`.** `#:property Optimize=true` fixes only the
script assembly, not Core. The BenchmarkDotNet projects are exempt (they mandate `-c Release`).

## Structure — two sides, joined on (op, dtype, N)

| Side | Where | What |
|------|-------|------|
| C# | `benchmark/NumSharp.Benchmark.CSharp/Benchmarks/<Category>/*.cs` | BenchmarkDotNet classes; `[Benchmark(Description="np.foo(a)")]` methods. |
| NumPy | `benchmark/NumSharp.Benchmark.Python/numpy_benchmark.py` | `run_<suite>_benchmarks(...)` emitting `BenchmarkResult` rows. |
| Merge | `benchmark/scripts/merge-results.py` | Joins on `(normalize_op_name(name), dtype, N)`. |
| Orchestrator | `benchmark/run_benchmark.py` | Builds C#, runs each suite (BDN) + warm NumPy across 1K/100K/10M, merges, snapshots. |

The join is by **normalized op name**: `normalize_op_name` strips the dtype tag, `[annotations]`, and
identifier-only arg lists — so C# `"np.foo(a)"` and NumPy `"np.foo"` both collapse to `np.foo` and join. Get the
names to normalize identically or the row shows as "C# not run" / "NumPy only".

## The matrix + subsystems

- **Op matrix** — 14 comparison suites, each a C# namespace filter in `run_benchmark.py`'s `SUITES` map
(`arithmetic, unary, reduction, broadcast, creation, manipulation, slicing, comparison, bitwise, logic,
statistics, sorting, linalg, selection`). Swept over 1K/100K/10M × the 15 dtypes.
- **Five appended subsystems** (own result models, appended not merged): `nditer` (iterator machinery),
`layout` (op × 8 memory layouts × dtype), `operand` (1-D/scalar/mixed/broadcast), `cast` (astype 15×15 × layout),
`fusion` (`np.evaluate`). Each is a `*_bench.{cs,py}` pair + a `*_sheet.py` renderer.

## Playbook — add a benchmark for a new op

The most common task. Full worked example in **`references/add-benchmark.md`**. In brief:

1. **C# side** — add a `[Benchmark(Description = "np.foo(a)")]` method to the class in
`Benchmarks/<Category>/` that fits (or a new class in that namespace so the suite's `*Benchmarks.<Category>.*`
filter auto-includes it). Use `BenchmarkBase` (single-dtype, float64) like the manipulation classes, or
`TypedBenchmarkBase` (dtype-swept) like arithmetic. `[Params(Medium, Large)]` for size.
2. **NumPy twin** — append to the matching `run_<suite>_benchmarks(...)` in `numpy_benchmark.py`, setting
`r.name, r.category, r.suite, r.dtype`. Make `.name` normalize to the C# Description (`"np.foo"` ↔ `"np.foo(a)"`).
3. **Smoke it** (this is usually the right scope — a full measured run is the post-release CI job):
`dotnet build -c Release`; `dotnet run -c Release --no-build -f net10.0 -- --list flat | grep <Class>` to confirm
BenchmarkDotNet discovers it; `python numpy_benchmark.py --suite <suite> --quick` to confirm the NumPy rows emit.
4. **Full numbers** come from `python run_benchmark.py` (or the `benchmark.yml` post-release workflow).

## Other tasks → where to go

- **Run the suite (official / subset), interpret the report, snapshots** → `references/run-and-report.md`.
- **Add or edit a matrix subsystem (nditer/layout/operand/cast/fusion)** → `references/subsystems.md`.
- **Everything else (all suites, config internals, troubleshooting, type map)** → `benchmark/CLAUDE.md`.

## Gotchas

- **BenchmarkDotNet's out-of-process toolchain fails here** ("project names need to be unique") because sibling
`.claude/worktrees/` checkouts hold same-named benchmark projects. The official run uses **InProcessEmit**
(`OfficialBenchmarkConfig`). For a smoke check use `--list flat` (reflection only, no toolchain) rather than
trying to run BDN ad-hoc.
- **What we commit is `benchmark/history/<date>_<sha>/`**, not the gitignored `benchmark/results/<ts>/` scratch.
Reference `benchmark/history/latest/benchmark-report.md`.
- **These are mostly view ops → sub-µs.** flip/rot90/transpose-aliases are O(1) views; their benchmark tracks
allocation/dispatch overhead, not throughput. Ops doing real work (trim_zeros, reductions) are where ratios
are meaningful.

## References

- `references/add-benchmark.md` — the detailed add-a-benchmark playbook (C# + NumPy twin + join-key rules + smoke).
- `references/run-and-report.md` — running the official run / subsets, the report + history snapshots, InProcessEmit.
- `references/subsystems.md` — the five subsystems and how to add one (`*_bench.{cs,py}` + `*_sheet.py`).
133 changes: 133 additions & 0 deletions .claude/skills/benchmark/references/add-benchmark.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Adding a benchmark for a new op

Goal: the official op matrix measures `np.<foo>` every run, with the C# result JOINED to its NumPy twin so the
report shows an NPY/NS ratio. Two edits (C# class + NumPy twin), then a smoke check. No `run_benchmark.py` change
if you place the C# class in an existing suite namespace.

## 1. C# side — a `[Benchmark]` method

Pick the class in `benchmark/NumSharp.Benchmark.CSharp/Benchmarks/<Category>/` that fits your op, or add a new
class in that **namespace** (`NumSharp.Benchmark.CSharp.Benchmarks.<Category>`) so the suite filter
`*Benchmarks.<Category>.*` auto-includes it — no orchestrator change.

Two base classes:
- **`BenchmarkBase`** — single dtype (float64 via `np.random.rand`). Use for dtype-agnostic ops (manipulation,
slicing). The merge defaults such rows to `dtype=float64`.
- **`TypedBenchmarkBase`** — sweeps `[ParamsSource(nameof(Types))] NPTypeCode DType`. Use for arithmetic/unary/etc.

Worked example (`Benchmarks/Manipulation/FlipRotBenchmarks.cs`, the flip/rot90/transpose-alias/trim_zeros twin):

```csharp
using BenchmarkDotNet.Attributes;
using NumSharp;
using NumSharp.Benchmark.CSharp.Infrastructure;

namespace NumSharp.Benchmark.CSharp.Benchmarks.Manipulation;

[BenchmarkCategory("Manipulation", "FlipRot")]
public class FlipRotBenchmarks : BenchmarkBase
{
private NDArray _arr2D = null!;

[Params(ArraySizeSource.Medium, ArraySizeSource.Large)] // 100K, 10M — match the sibling manip classes
public override int N { get; set; }

[GlobalSetup]
public void Setup()
{
np.random.seed(Seed);
var rows = (int)Math.Sqrt(N);
_arr2D = np.random.rand(rows, N / rows) * 100;
}

[GlobalCleanup]
public void Cleanup() { _arr2D = null!; GC.Collect(); }

[Benchmark(Description = "np.flip(a)")] // Description is the JOIN KEY (normalized)
[BenchmarkCategory("Flip")]
public NDArray Flip() => np.flip(_arr2D);

[Benchmark(Description = "np.trim_zeros(a)")]
[BenchmarkCategory("TrimZeros")]
public NDArray TrimZeros() => np.trim_zeros(_arr2D, "fb");
// ...fliplr / flipud / rot90 / permute_dims / matrix_transpose the same way
}
```

## 2. NumPy twin — a row in `numpy_benchmark.py`

Append to the matching `run_<suite>_benchmarks(...)` (e.g. `run_manipulation_benchmarks`). Each benchmark: define a
closure, time it with `benchmark(fn, n, iterations=iterations)`, set the four fields. The suite function is already
wired into `main()`'s dispatch and `run_benchmark.py`, so appending is all it takes.

```python
def np_flip(): return np.flip(arr_2d)
r = benchmark(np_flip, n, iterations=iterations)
r.name, r.category, r.suite, r.dtype = "np.flip", "Flip", "Manipulation", dtype_name
results.append(r)

def np_trim_zeros(): return np.trim_zeros(arr_2d, 'fb')
r = benchmark(np_trim_zeros, n, iterations=iterations)
r.name, r.category, r.suite, r.dtype = "np.trim_zeros", "TrimZeros", "Manipulation", dtype_name
results.append(r)
```

## 3. The join — make the names normalize identically

The merge key is `(normalize_op_name(name), dtype, N)`. `normalize_op_name` (in `merge-results.py`) applied to BOTH
sides: strips a trailing dtype tag `(float64)`, strips `[annotations]`, folds `(a, axis=k) -> axis=k`, and strips
**identifier-only** arg lists (`(a)`, `(a, b)`) but KEEPS numeric args (`(a, 50)`). So:

- C# `"np.flip(a)"` → strip `(a)` → `np.flip`. NumPy `"np.flip"` → `np.flip`. **Join.** ✓
- C# `"np.percentile(a, 50)"` → `np.percentile(a, 50)` (numeric arg kept). NumPy must match that literally.

Rule of thumb: name the C# Description `"np.<op>(a)"` and the NumPy `.name` `"np.<op>"`, and they join. If a row
shows as "C# not run" or "NumPy only" in the report, the names didn't normalize to the same string — that's the
first thing to check.

Dtype/N join: `BenchmarkBase` (no `DType`) → merge defaults dtype to `float64`, so the NumPy twin must set
`dtype_name = 'float64'`. `[Params(Medium, Large)]` joins the NumPy 100K/10M rows; NumPy's 1K row shows C#-missing
(fine — the sibling manip classes do the same).

## 4. Smoke test (the usual scope)

A full measured run is expensive and normally the post-release `benchmark.yml` job. To confirm the wiring:

```bash
cd benchmark/NumSharp.Benchmark.CSharp
dotnet build -c Release -v q --nologo
dotnet run -c Release --no-build -f net10.0 -- --list flat | grep FlipRot # BDN discovers the 7 methods
cd ../NumSharp.Benchmark.Python
python numpy_benchmark.py --suite manipulation --quick --size medium | grep -iE "flip|trim_zeros" # NumPy rows emit
```

`--list flat` is reflection-only (no toolchain), so it works despite the out-of-process-toolchain limitation.
A direct-call `dotnet run -c Release` script (mirroring each benchmark body) confirms the C# calls execute.

## 5. Full run (optional)

`python run_benchmark.py --suites manipulation` measures just your suite; `python run_benchmark.py` runs everything
and writes a `benchmark/history/<date>_<sha>/` snapshot. See `references/run-and-report.md`.

## Adding a whole new SUITE (rare — most ops fit an existing one)

Only when your ops form a new category with no existing home:

1. **C#** — create `Benchmarks/<NewCategory>/<Foo>Benchmarks.cs` in namespace
`NumSharp.Benchmark.CSharp.Benchmarks.<NewCategory>`.
2. **NumPy** — add `def run_<newsuite>_benchmarks(n, dtype_name, iterations): ...` in `numpy_benchmark.py` and
dispatch it in `main()` (`if suite in ["<newsuite>", "all"]: results_all.extend(run_<newsuite>_benchmarks(...))`).
3. **Orchestrator** — add `"<newsuite>": "*Benchmarks.<NewCategory>.*"` to the `SUITES` map in `run_benchmark.py`.
4. (legacy) optionally add the filter to `run-benchmarks.ps1`'s `ValidateSet` and Program.cs menu.

Then the new suite runs in `python run_benchmark.py --suites <newsuite>` (and in the full run), joined and reported
exactly like the built-in 14. Keep the C# `[Benchmark(Description)]` labels normalizing onto the NumPy `.name`s.

## Choosing base class & size (quick reference)

- **Dtype-agnostic op** (shape/view/copy) → `BenchmarkBase`, NumPy twin `dtype_name='float64'` (merge default).
- **Numeric op you want swept over dtypes** → `TypedBenchmarkBase` (override `Types` for the relevant set), and give
the NumPy twin a `dtype_name` loop in `main()` so both sides sweep the same dtypes.
- **Size**: `[Params(Medium, Large)]` (100K/10M) is the manipulation/reshape norm; add `Small` (1K) for ops where
per-element overhead matters. The NumPy side sweeps all three via `--cache-sizes`; unmatched sizes just show as
C#-missing cells.
Loading
Loading