Skip to content

Add inference-engine package command to generate Dockerfile, requirements.txt, and deployment.json #26

@AK11105

Description

@AK11105

What problem does this solve?

After deploying a model, there is no way to produce a portable, self-contained artifact that can be shipped to another environment. Users must manually write Dockerfiles, pin dependencies, and assemble metadata — the same boilerplate every time.

Proposed solution

New command: inference-engine package models/sentiment/v1/

Generates in the model directory:

models/sentiment/v1/
├── definition.py       (existing)
├── Dockerfile          (generated from template)
├── requirements.txt    (generated — pinned from current venv)
└── deployment.json     (metadata: name, version, framework, load_format, device, created_at, sample_input)

Dockerfile is template-based, not LLM-generated. Parameterised by Python version (sys.version_info), device (cpupython:3.x-slim, gpunvidia/cuda:12.x-runtime), and port (default 8000, --port to override).

requirements.txt pins framework-relevant packages from the current environment using importlib.metadata.version(). No LLM involved. Always includes fastapi, uvicorn, inference-engine.

Framework Packages pinned
sklearn scikit-learn, joblib, numpy
pytorch torch
transformers transformers, torch, tokenizers
xgboost xgboost, numpy
lightgbm lightgbm, numpy
catboost catboost, numpy
onnx onnxruntime, numpy
sentence_transformers sentence-transformers, torch

deployment.json includes sample_input from DeployAnswers so downstream commands (benchmark, snippets, metadata API) can use it without requiring the user to re-specify it.

Alternatives considered

Manually writing Dockerfiles and requirements files. Error-prone and not reproducible across environments.

Area

CLI (deploy / fix)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions