Add lint-package debug command for engines, runtimes and models by mrmara · Pull Request #394 · canonical/inference-snaps-cli

mrmara · 2026-07-08T10:25:19Z

lint-package command checks and validate manifests file for engines, runtimes and models

Example usage:
go run ./cmd/modelctl debug lint-package <snap-directory>

Test:

Succesful

go run ./cmd/modelctl debug lint-package ../qwen3.5
✅ ../qwen3.5/engines/cpu/engine.yaml
✅ ../qwen3.5/engines/nvidia-gpu/engine.yaml
✅ ../qwen3.5/models/4b-q4-k-xl-gguf/model.yaml
✅ ../qwen3.5/models/9b-q4-k-m-gguf/model.yaml
✅ ../qwen3.5/runtimes/llamacpp/runtime.yaml
✅ ../qwen3.5/runtimes/llamacpp-cuda/runtime.yaml

Fail since models, runtimes nor engines directories are found:

go run ./cmd/modelctl debug lint-package .
❌ engines: no manifests found
❌ models: no manifests found
❌ runtimes: no manifests found
Error: not all manifests are valid
exit status 1

Fail since models directory is not found:

go run ./cmd/modelctl debug lint-package ../qwen3.5-another/
✅ ../qwen3.5-another/engines/cpu/engine.yaml
✅ ../qwen3.5-another/engines/nvidia-gpu/engine.yaml
❌ ../qwen3.5-another/models: no manifests found
✅ ../qwen3.5-another/runtimes/llamacpp/runtime.yaml
✅ ../qwen3.5-another/runtimes/llamacpp-cuda/runtime.yaml
Error: not all manifests are valid
exit status 1

Fail because of validation error on a manifest:

go run ./cmd/modelctl debug lint-package ../qwen3.5-another/
✅ ../qwen3.5-another/engines/cpu/engine.yaml
✅ ../qwen3.5-another/engines/nvidia-gpu/engine.yaml
❌ ../qwen3.5-another/models/qwen-3-5-4b-q4-k-xm/model.yaml: model directory name should match name in manifest: qwen-3-5-4b-q4-k-xm != Qwen 3.5 4B
✅ ../qwen3.5-another/models/qwen-3-5-9b-q4-k-m/model.yaml
✅ ../qwen3.5-another/runtimes/llamacpp/runtime.yaml
✅ ../qwen3.5-another/runtimes/llamacpp-cuda/runtime.yaml
Error: not all manifests are valid
exit status 1

Fail because of webui unsupported model capabilities:

go run ./cmd/modelctl debug lint-package ../qwen3.5
✅ ../qwen3.5/engines/cpu/engine.yaml
✅ ../qwen3.5/engines/nvidia-gpu/engine.yaml
❌ ../qwen3.5/models/4b-q4-k-xl-gguf/model.yaml: unsupported capability: "thinking"
❌ ../qwen3.5/models/9b-q4-k-m-gguf/model.yaml: unsupported capability: "thinking"
✅ ../qwen3.5/runtimes/llamacpp/runtime.yaml
✅ ../qwen3.5/runtimes/llamacpp-cuda/runtime.yaml
Error: not all manifests are valid
exit status 1

Suggested change in workflows:

~~go run . debug validate-engines ../../../engines/**/*.yaml~~
go run . debug lint-package ../../../

Copilot

Pull request overview

Adds a modelctl debug lint-package <snap-directory> command to lint/validate manifest YAMLs across an entire snap package layout (engines/models/runtimes), and introduces dedicated validators + tests for model and runtime manifests.

Changes:

Add YAML manifest validators for pkg/models and pkg/runtimes (unknown-field detection, required-field checks, directory-name consistency checks).
Add test suites to validate real manifests under test_data/ and validate key error cases (empty YAML, unknown fields, missing required fields).
Update cmd/modelctl debug command from validate-engines to lint-package, scanning a snap directory for manifests and reporting per-file results.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
cmd/modelctl/commands/debug/validate.go	Implements `lint-package` command to discover and validate manifests under engines/models/runtimes.
pkg/models/validate.go	Adds model manifest YAML validation and directory-name matching logic.
pkg/models/validate_test.go	Adds tests for model manifest validation and required fields.
pkg/runtimes/validate.go	Adds runtime manifest YAML validation and directory-name matching logic.
pkg/runtimes/validate_test.go	Adds tests for runtime manifest validation and required fields.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

farshidtz

Great to have the validation coverage gap reduced.

farshidtz · 2026-07-10T13:24:24Z

+	for _, v := range validators {
+		pattern := filepath.Join(snapDir, v.subdir, "*", v.manifestFilename)
+		manifestPaths, err := filepath.Glob(pattern)
 		if err != nil {
+			return fmt.Errorf("invalid glob pattern %s: %w", pattern, err)
+		}
+
+		if len(manifestPaths) == 0 {
 			allManifestsValid = false
-			fmt.Printf("❌ %s: %s\n", manifestPath, err)
-		} else {
-			fmt.Printf("✅ %s\n", manifestPath)
+			fmt.Printf("❌ %s: no manifests found\n", filepath.Join(snapDir, v.subdir))
+			continue
+		}


I would consider both runtimes and models optional. There could be an engine without a runtime, delivering just optimized weights or an engine with a combined runtime+model, using llamafiles. I see runtimes and models treated as an expected fields for engines elsewhere in the code so there has been an architectural regression. In this case, let's leave this for now.

farshidtz · 2026-07-10T13:29:30Z

+	if manifest.ModelCardUrl == "" {
+		return fmt.Errorf("required field is not set: model-card-url")
+	}


Validate URL. (url.Parse)

farshidtz · 2026-07-10T13:30:06Z

+
+	if manifest.DiskSize == "" {
+		return fmt.Errorf("required field is not set: disk-size")
+	}


Validate input format / suffix.

farshidtz · 2026-07-10T13:31:03Z

+		return fmt.Errorf("required field is not set: capabilities")
+	} else {
+		for _, cap := range manifest.Capabilities {
+			if !slices.Contains(webui.SupportedCapabilities(), cap) {


Make SupportedCapabilities() a private method of the model manifest. This is now used beyond webui

farshidtz · 2026-07-10T13:33:00Z

+		return fmt.Errorf("required field is not set: environment")
+	}
+
+	return nil


Validate Layout

farshidtz · 2026-07-10T13:34:53Z

+		return fmt.Errorf("required field is not set: components")
+	}
+
+	return nil


Validate Layout

farshidtz · 2026-07-10T13:37:19Z

+	if err == nil {
+		t.Fatal("Empty yaml should fail")
+	}
+	t.Log(err)


Won't this just produce noise in verbose test logs. It should only print if there is something unexpected.

Same in several tests, also in models package.

farshidtz · 2026-07-10T13:40:59Z

 	}

 	if !allManifestsValid {
 		return fmt.Errorf("not all manifests are valid")


DE013 / CLI Copy and Tone of Voice:

Suggested change

return fmt.Errorf("some manifests are invalid")

add lint-package for engines, runtimes and models

cb14593

Copilot AI review requested due to automatic review settings July 8, 2026 10:25

Copilot started reviewing on behalf of mrmara July 8, 2026 10:25 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread pkg/models/validate.go

Comment thread pkg/models/validate.go

Comment thread pkg/models/validate.go

Comment thread cmd/modelctl/commands/debug/validate.go Outdated

Comment thread cmd/modelctl/commands/debug/validate.go

mrmara added 3 commits July 8, 2026 12:44

PR review

4d37797

add capablity check

2bd80ef

fix tests

22ff171

mrmara mentioned this pull request Jul 8, 2026

Get model capabilities from metadata #395

Open

farshidtz requested changes Jul 10, 2026

View reviewed changes

Uh oh!

Conversation

mrmara commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

farshidtz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

mrmara commented Jul 8, 2026 •

edited

Loading