Skip to content

Publish the layer catalog as a validated, dependency-graphed dataset#85

Draft
slavazeph-coder wants to merge 1 commit into
mainfrom
claude/backend-layer-architecture-vb55g9
Draft

Publish the layer catalog as a validated, dependency-graphed dataset#85
slavazeph-coder wants to merge 1 commit into
mainfrom
claude/backend-layer-architecture-vb55g9

Conversation

@slavazeph-coder

Copy link
Copy Markdown
Owner

Summary

Applies the patterns from Marble's os-taxonomy — a schema-validated knowledge graph with a prerequisite/dependency graph, a versioned manifest, and a validator — to BrainSNN's 103-layer catalog. Turns a hand-maintained JS array into a validated, published dataset.

Type of change

  • feature
  • docs / infra

What changed

  • Dependency graph (src/lib/layerCatalog.js) — LAYER_DEPENDENCIES, a DAG mirroring runLayerRouter's real data flow (e.g. L103 soliton ← L4 firewall + L29 affect; L3 TRIBE ← L4 + L29; L46 receipt ← L4 + L103), each edge with a reason.
  • Versioned manifestgetCatalogManifest(): schema/version, total, per-group counts, core ids, and a deterministic content hash (no timestamps, so a scan receipt could cite it).
  • ValidatorvalidateCatalog(): unique/contiguous ids, valid groups, resolvable CORE_LAYER_IDS, referential + acyclicity checks on the graph, non-empty blurbs. Exposed as scripts/validate-catalog.mjs (npm run validate:catalog) and gated in CI.
  • Published datasetscripts/export-catalog.mjsdata/layer-catalog.json (pure data, no JS import needed) with schema/layer-catalog.schema.json. A test keeps the committed JSON in sync via the manifest hash.
  • APIGET /api/layers now returns manifest + dependencies.
  • Docs — README section on the dataset; new PROVENANCE.md recording sources/licenses and the os-taxonomy (ODbL/CC-BY-SA, inspiration only — no data copied) and Brain2Qwerty (CC-BY-NC, network boundary only) lines.

Validation

  • npm run validate:catalog — 103 layers, 6 edges, hash smjcw2
  • npm test — 73/73 (4 new catalog tests incl. dataset-freshness)
  • npm run lint (tsc --noEmit) — clean
  • npm run build — green
  • GET /api/layers returns manifest + dependencies (verified via curl)

Notes

No data was copied from os-taxonomy — only the approach. The dependency edges are original and reflect BrainSNN's own engine flow. CI (from #82) will run validate:catalog + tests + build + MCP smoke on this PR.

🤖 Generated with Claude Code


Generated by Claude Code

Inspired by Marble's os-taxonomy (a schema-validated knowledge graph with a
prerequisite graph, versioned manifest and validator), applied to the 103-layer
catalog:

- layerCatalog.js: LAYER_DEPENDENCIES (a DAG mirroring runLayerRouter's real
  data flow — e.g. L103 soliton ← L4 firewall + L29 affect), a deterministic
  getCatalogManifest() (version + per-group counts + content hash), and
  validateCatalog() (unique/contiguous ids, valid groups, resolvable core ids,
  referential + acyclic dependency checks, non-empty blurbs).
- scripts/validate-catalog.mjs + scripts/export-catalog.mjs; npm run
  validate:catalog / export:catalog. Published data/layer-catalog.json +
  schema/layer-catalog.schema.json (pure data, no runtime import needed).
- GET /api/layers now returns the manifest + dependencies.
- src/lib/layerCatalog.test.js: validation, manifest self-consistency, acyclic
  graph, and a freshness check that the committed JSON matches the live catalog.
- CI runs validate:catalog; README documents the dataset; PROVENANCE.md records
  sources/licenses and the os-taxonomy + Brain2Qwerty boundaries.

73 tests pass, tsc clean, build green, catalog valid.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014aeNbHUmmm6bRDt3eiZ9GV

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request publishes the 103-layer engine catalog as a validated, schema-backed dataset with a data-flow dependency graph, complete with export/validation scripts, server integration, and tests. The reviewer feedback highlights several key areas for improvement: ensuring metadata changes (blurbs and reasons) are captured in the deterministic content hash, adding missing edge-case validations (such as empty fields, array checks, and core-layer enforcements) to the catalog validator, improving cycle detection error reporting to trace the exact path, and tightening the JSON schema with enums for groups and regex patterns for hex colors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +123 to +127
const payload = JSON.stringify({
layers: LAYER_CATALOG.map((layer) => [layer.id, layer.name, layer.group]),
core: CORE_LAYER_IDS,
deps: LAYER_DEPENDENCIES.map((edge) => [edge.id, edge.dependsOn]),
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The catalogHash() function currently excludes layer.blurb and edge.reason from the hash payload. This means any changes to layer blurbs or dependency reasons in layerCatalog.js will not update the content hash. Consequently, the synchronization test keeps the published data/layer-catalog.json in sync will pass even if the committed JSON file is out of sync with the JS source file. Including these fields in the hash payload ensures metadata changes are correctly tracked and synchronized.

Suggested change
const payload = JSON.stringify({
layers: LAYER_CATALOG.map((layer) => [layer.id, layer.name, layer.group]),
core: CORE_LAYER_IDS,
deps: LAYER_DEPENDENCIES.map((edge) => [edge.id, edge.dependsOn]),
});
const payload = JSON.stringify({
layers: LAYER_CATALOG.map((layer) => [layer.id, layer.name, layer.group, layer.blurb]),
core: CORE_LAYER_IDS,
deps: LAYER_DEPENDENCIES.map((edge) => [edge.id, edge.dependsOn, edge.reason]),
});

Comment on lines +155 to +185
export function validateCatalog() {
const errors = [];
const ids = LAYER_CATALOG.map((layer) => layer.id);
const idSet = new Set(ids);

if (idSet.size !== ids.length) errors.push('Duplicate layer ids.');
ids.forEach((id, index) => {
if (id !== index + 1) errors.push(`Layer ids must be contiguous from 1; got ${id} at position ${index + 1}.`);
});

LAYER_CATALOG.forEach((layer) => {
if (!LAYER_GROUPS[layer.group]) errors.push(`Layer ${layer.id} has unknown group "${layer.group}".`);
if (!layer.blurb || !layer.blurb.trim()) errors.push(`Layer ${layer.id} has an empty blurb.`);
});

new Set(CORE_LAYER_IDS).forEach((id) => {
if (!idSet.has(id)) errors.push(`CORE_LAYER_IDS references missing layer ${id}.`);
});
if (new Set(CORE_LAYER_IDS).size !== CORE_LAYER_IDS.length) errors.push('Duplicate ids in CORE_LAYER_IDS.');

const seenEdges = new Set();
LAYER_DEPENDENCIES.forEach((edge) => {
if (!idSet.has(edge.id)) errors.push(`Dependency references missing layer ${edge.id}.`);
if (seenEdges.has(edge.id)) errors.push(`Duplicate dependency entry for layer ${edge.id}.`);
seenEdges.add(edge.id);
(edge.dependsOn || []).forEach((dep) => {
if (!idSet.has(dep)) errors.push(`Layer ${edge.id} depends on missing layer ${dep}.`);
if (dep === edge.id) errors.push(`Layer ${edge.id} depends on itself.`);
});
if (!edge.reason || !edge.reason.trim()) errors.push(`Dependency ${edge.id} has no reason.`);
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Several validation checks are missing or can be optimized in validateCatalog():

  1. Empty Catalog Check: It does not verify if LAYER_CATALOG is empty, which would violate the JSON schema requirement (totalLayers >= 1).
  2. Empty Name Check: It does not verify if layer.name is missing or empty.
  3. Array Check for Dependencies: It does not verify if edge.dependsOn is missing or not an array.
  4. Duplicate Dependencies Check: It does not check for duplicate dependencies within the same edge.
  5. Core Layer Enforcements: Since dependencies represent real-time data flow in runLayerRouter, only core layers should have dependencies or be depended upon. Enforcing that both edge.id and dependsOn elements are in CORE_LAYER_IDS strengthens referential integrity.
  6. Set Reuse: Reuses a single coreSet to avoid creating new Set(CORE_LAYER_IDS) twice.
export function validateCatalog() {
  const errors = [];
  if (LAYER_CATALOG.length === 0) {
    errors.push('LAYER_CATALOG cannot be empty.');
  }
  const ids = LAYER_CATALOG.map((layer) => layer.id);
  const idSet = new Set(ids);

  if (idSet.size !== ids.length) errors.push('Duplicate layer ids.');
  ids.forEach((id, index) => {
    if (id !== index + 1) errors.push('Layer ids must be contiguous from 1; got ' + id + ' at position ' + (index + 1) + '.');
  });

  LAYER_CATALOG.forEach((layer) => {
    if (!layer.name || !layer.name.trim()) errors.push('Layer ' + layer.id + ' has an empty name.');
    if (!LAYER_GROUPS[layer.group]) errors.push('Layer ' + layer.id + ' has unknown group "' + layer.group + '".');
    if (!layer.blurb || !layer.blurb.trim()) errors.push('Layer ' + layer.id + ' has an empty blurb.');
  });

  const coreSet = new Set(CORE_LAYER_IDS);
  coreSet.forEach((id) => {
    if (!idSet.has(id)) errors.push('CORE_LAYER_IDS references missing layer ' + id + '.');
  });
  if (coreSet.size !== CORE_LAYER_IDS.length) errors.push('Duplicate ids in CORE_LAYER_IDS.');

  const seenEdges = new Set();
  LAYER_DEPENDENCIES.forEach((edge) => {
    if (!idSet.has(edge.id)) errors.push('Dependency references missing layer ' + edge.id + '.');
    if (!coreSet.has(edge.id)) errors.push('Dependency source layer ' + edge.id + ' is not in CORE_LAYER_IDS.');
    if (seenEdges.has(edge.id)) errors.push('Duplicate dependency entry for layer ' + edge.id + '.');
    seenEdges.add(edge.id);

    if (!Array.isArray(edge.dependsOn)) {
      errors.push('Dependency ' + edge.id + ' is missing a valid \'dependsOn\' array.');
    } else {
      const seenDeps = new Set();
      edge.dependsOn.forEach((dep) => {
        if (!idSet.has(dep)) errors.push('Layer ' + edge.id + ' depends on missing layer ' + dep + '.');
        if (!coreSet.has(dep)) errors.push('Dependency target layer ' + dep + ' (depended on by layer ' + edge.id + ') is not in CORE_LAYER_IDS.');
        if (dep === edge.id) errors.push('Layer ' + edge.id + ' depends on itself.');
        if (seenDeps.has(dep)) errors.push('Layer ' + edge.id + ' has duplicate dependency on layer ' + dep + '.');
        seenDeps.add(dep);
      });
    }
    if (!edge.reason || !edge.reason.trim()) errors.push('Dependency ' + edge.id + ' has no reason.');
  });

Comment on lines +187 to +205
// Cycle detection (DFS over the dependency edges).
const adjacency = new Map(LAYER_DEPENDENCIES.map((edge) => [edge.id, edge.dependsOn || []]));
const state = new Map();
const visit = (node) => {
if (state.get(node) === 'done') return false;
if (state.get(node) === 'active') return true;
state.set(node, 'active');
for (const next of adjacency.get(node) || []) {
if (visit(next)) return true;
}
state.set(node, 'done');
return false;
};
for (const edge of LAYER_DEPENDENCIES) {
if (visit(edge.id)) {
errors.push(`Dependency cycle detected at layer ${edge.id}.`);
break;
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current cycle detection logic reports "Dependency cycle detected at layer ${edge.id}." even if edge.id is not part of the cycle but merely depends on a cycle (e.g., A -> B -> C -> B would report a cycle at A). We can improve this by tracing and reporting the exact cycle path (e.g., B -> C -> B), which makes debugging much easier.

  // Cycle detection (DFS over the dependency edges).
  const adjacency = new Map(LAYER_DEPENDENCIES.map((edge) => [edge.id, edge.dependsOn || []]));
  const state = new Map();
  const visit = (node, path = []) => {
    if (state.get(node) === 'done') return false;
    if (state.get(node) === 'active') {
      const cycleStartIdx = path.indexOf(node);
      const cyclePath = path.slice(cycleStartIdx).concat(node).join(' -> ');
      errors.push('Dependency cycle detected: ' + cyclePath);
      return true;
    }
    state.set(node, 'active');
    for (const next of adjacency.get(node) || []) {
      if (visit(next, [...path, node])) return true;
    }
    state.set(node, 'done');
    return false;
  };
  for (const edge of LAYER_DEPENDENCIES) {
    if (visit(edge.id)) {
      break;
    }
  }

Comment on lines +45 to +46
"name": { "type": "string" },
"group": { "type": "string" },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

In the JSON schema, we can make validation stronger and self-documenting by:

  1. Restricting group to the allowed enum values ("view", "firewall", "share", "data", "backend", "progression").
Suggested change
"name": { "type": "string" },
"group": { "type": "string" },
"name": { "type": "string", "minLength": 1 },
"group": { "type": "string", "enum": ["view", "firewall", "share", "data", "backend", "progression"] },

Comment on lines +31 to +32
"label": { "type": "string" },
"color": { "type": "string" }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

We can add a pattern check for the color property in the group definitions to ensure it is a valid 6-digit hex color starting with #.

Suggested change
"label": { "type": "string" },
"color": { "type": "string" }
"label": { "type": "string", "minLength": 1 },
"color": { "type": "string", "pattern": "^#[0-9a-fA-F]{6}$" }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants