trace-schemas: add namespace-driven schema generation and validation tooling by jutaro · Pull Request #6527 · IntersectMBO/cardano-node

jutaro · 2026-04-13T10:06:20Z

Description

Summary

This PR introduces a namespace-driven trace schema workflow and the tooling around it.

It adds support for generating schema files from traced namespaces, validates generated schemas and trace logs, documents how overrides work, and wires the schema scripts into a standalone Cabal package so they can be run consistently from the dev environment. It also updates trace documentation generation to work with optional namespace lists and refreshes the generated trace-schema artifacts.

On the tracing side, it extends the source discovery and inference logic used by schema generation, improves handling of external trace-dispatcher definitions, and tightens warning reporting so schema generation only reports real unresolved cases instead of false positives.

Why this is hard

This cannot be solved by deriving JSON Schema directly from Haskell types, because the trace JSON is not determined by the types alone. The emitted payloads are shaped by forMachine implementations, helper functions, pattern matches, namespace mapping, verbosity/detail levels, and hand-written rendering logic across multiple packages. In practice, fields may be renamed, omitted, flattened, synthesized, or rendered as strings, so the runtime JSON contract often differs from the source type definition.

Main changes

Add bench/trace-schemas as the home for generated schemas, metadata, documentation, presentations, and helper docs.
Add schema-generation tooling:
- GhciSchemaGen.hs
- ValidateTraceSchemas.hs
- ValidateTraceLog.hs
- ApplySchemaOverrides.hs
- CheckOverrideCoverage.hs
- RegenerateTraceSchemas.sh
Add trace-schema-gen.cabal and wire it into cabal.project.
Add override documentation and examples for human-maintained schema patches.
Update cardano-node trace documentation generation to support optional namespace selection.
Improve namespace/source resolution and schema inference across cardano-node, trace-dispatcher, and related tracing modules.
Regenerate newNamespaces.txt, trace-documentation.md, and schema artifacts from the new pipeline.
Guard destructive override application by default.
Tighten schema-generation warnings so only genuinely empty/unresolved final schemas are reported.

Testing

Ran schema generation through nix develop.
Ran override application and check scripts.
Verified the updated generator completes with Schema generation problems: 0.

mgmeier

Please condense / clean-up commit history for this PR.

mgmeier

Testing feedback: You've already made the various scripts executables in bench/trace-schemas/scripts/schema-gen/trace-schema-gen.cabal. This means, they're part of the nix flake.

So instead of building a full dev shell with all dependencies, and running the script via bash / runghc, you should switch the new Make targets to something like nix run .#schema-gen.

Remove unnecessary artefacts

mgmeier · 2026-04-23T09:49:32Z

+readJsonFile :: FilePath -> IO A.Value
+readJsonFile fp = do
+  bs <- BL.readFile fp
+  case A.eitherDecode bs of


Please use A.eitherDecodeFileStrict'

mgmeier · 2026-04-23T09:52:07Z

+
+parseArgs :: Config -> [String] -> IO Config
+parseArgs = go
+ where


I'm aware it's a standalone script... but it still could use optparse-applicative for CLI parsing.
Only a suggestion - the approach taken here is sufficient for the scope.

mgmeier · 2026-04-23T09:56:34Z

+
+splitPath :: FilePath -> [FilePath]
+splitPath path = filter (not . null) (go path)
+ where


Please use splitPath or splitDirectories from System.FilePath.

mgmeier · 2026-04-23T09:57:19Z

+      (a, []) -> [a]
+      (a, _:rest) -> a : go rest
+
+joinPath :: [FilePath] -> FilePath


joinPath is defined in System.FilePath.

mgmeier · 2026-04-23T09:58:04Z

+joinPath [] = ""
+joinPath (x:xs) = foldl (</>) x xs
+
+replaceFileName :: FilePath -> FilePath -> FilePath


replaceFileName is defined in System.FilePath.

mgmeier · 2026-04-23T10:14:00Z

+
+-- Read file for quick text parsing; tolerate missing files.
+readFileSafe :: FilePath -> String
+readFileSafe fp = unsafePerformIO (readFile fp `catch` (\(_e :: IOException) -> pure ""))


readFileSafe is only ever called inside IO.
There is no need for it to be a pure function using unsafePerformIO - and then naming it "safe" ;)

mgmeier · 2026-04-23T10:15:58Z

+collectTargets :: [FilePath] -> IO [FilePath]
+collectTargets roots = do
+  files <- concat <$> mapM listHsFiles roots
+  catMaybes <$> mapM (\fp -> do


Can you use filterM?

mgmeier · 2026-04-23T10:23:01Z

+  (x:_) -> x
+
+isInfix :: String -> String -> Bool
+isInfix needle hay = T.isInfixOf (T.pack needle) (T.pack hay)


Why not just use Data.List.isInfixOf ?

mgmeier · 2026-04-23T10:23:44Z

+        then pure v
+        else do
+          bs <- BL.readFile histOut
+          case A.decode bs of


-> A.decodeFile'

I first tried A.decodeFile' to match the suggestion, but this repo’s Aeson version does not export it, so A.decodeFileStrict' is the compatible file-based replacement here.

mgmeier · 2026-04-23T10:26:55Z

+      [] -> s''
+
+stripPrefix :: String -> String -> Maybe String
+stripPrefix pre s = if pre `isPrefixOf` s then Just (drop (length pre) s) else Nothing


Data.List.stripPrefix ?

jutaro self-assigned this Apr 13, 2026

jutaro force-pushed the jutaro/namespace_generation branch from 9b37275 to 8913d02 Compare April 20, 2026 11:32

jutaro marked this pull request as ready for review April 21, 2026 09:36

jutaro requested review from a team as code owners April 21, 2026 09:36

jutaro requested review from Icelandjack and removed request for a team April 21, 2026 09:36

mgmeier requested changes Apr 21, 2026

View reviewed changes

jutaro added 5 commits April 22, 2026 11:36

trace-dispatcher: namespace generation

dc4485f

cardano-node: trace docu with optional namespaces

c5d545f

Json schema generation for trace messages

3eaa9f2

Patching CI warnings

957ae2a

Review changes 1

93fa1f5

Remove unnecessary artefacts

jutaro force-pushed the jutaro/namespace_generation branch from 6dd06b1 to 93fa1f5 Compare April 22, 2026 10:18

mgmeier requested changes Apr 23, 2026

View reviewed changes

jutaro added 6 commits April 23, 2026 17:35

Review changes 2

1e620b4

Add testcases with cabal test

a595238

Use optparse-applicative for CL parsing

1e15488

Fix

7541779

trace-schemas: run schema tools via flake executables

23034d9

Docu fix

3d2061f

Conversation

jutaro commented Apr 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Why this is hard

Main changes

Testing

Uh oh!

mgmeier left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mgmeier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jutaro commented Apr 13, 2026 •

edited

Loading