SCM 1.0 embedded agent runtime

[saishshinde15]

What changed

SCM now has an embedded Python runtime with no SCM server, localhost URL, hosted account, or base URL in the primary path. SCM(user_id=...) exposes the five lifecycle operations directly, and scm.langchain adds LangChain 1.x middleware with automatic recall, durable human-message capture, bounded untrusted memory context, identity-safe tools, and retry idempotency.

The release also adds lifecycle snapshot v2 with atomic local storage, checksums, optional encryption, prior-generation recovery, process locks, PostgreSQL revision/history recovery, per-user advisory locks, and sleep leases. REST, MCP, SCMEngine, and JavaScript remain compatible secondary surfaces.

Product surface

• SCM and SCMContext are public imports under scm.
• create_scm_agent and SCMMiddleware target LangChain 1.x.
• Package metadata and extras target 1.0.0; README, Getting Started, architecture, product, publishing, and LangChain docs now lead with embedded usage.
• Primary examples are server-free; JavaScript is documented honestly as a remote REST client.

Verification

• Full source qualification: compile, front-door invariant, Ruff critical checks, Bandit, 50 production tests, 92 full regression tests, JS tests/package checks.
• Clean wheel/sdist build, Twine check, install into a temporary external venv, SCM smoke, LangChain smoke, CLI doctor/demo, REST restart durability, JS live smoke, and dependency audit all passed.
• Real PostgreSQL tests passed for lifecycle restart, concurrent multi-process writes, sleep lease exclusivity/release, and corrupt-current recovery from history.
• Final report: /tmp/scm-v1-final-qualification.json with ready_for_release=true.

Release note

This is a draft PR for hosted CI and review. Do not tag or publish 1.0.0 until trusted publishing and the hosted matrix are green.

[Copilot]

Pull request overview

This PR promotes SCM to a server-free embedded Python runtime (SCM(user_id=...)) and adds LangChain 1.x middleware, while upgrading durability with snapshot v2 (atomic local storage, optional encryption, recovery) and adding a PostgreSQL-backed state store for multi-worker coordination.

Changes:

• Introduces the embedded SCM/SCMContext public surface and LangChain 1.x middleware (scm.langchain).
• Adds snapshot v2 persistence (checksums, backups, optional encryption, locks) plus a new PostgreSQL state store.
• Updates docs, examples, qualification scripts, and CI/release workflows to lead with embedded usage and test the new surfaces.

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_langchain_embedded.py	New regression tests for embedded LangChain middleware semantics and invariants.
tests/test_embedded_scm.py	New tests for the embedded SCM public API, isolation, idempotency, and recovery.
tests/production/test_recovery.py	Updates snapshot version assertions and adds v1-compat + recovery cases.
tests/production/test_postgres_embedded.py	Adds production tests for PostgreSQL coordination/recovery paths.
tests/production/test_performance.py	Adds embedded warm-path p95 budget tests for add/search.
tests/production/test_concurrency.py	Adds local multi-process concurrency tests for embedded persistence.
src/version.py	Bumps Python package runtime version to 1.0.0.
src/runtime_factory.py	Adds injectable llm parameter to lifecycle engine factory for embedded/middleware reuse.
src/integrations/user_state_store.py	Implements snapshot v2: atomic commit, backup recovery, encryption, file/process locks.
src/integrations/tools.py	Routes add-memory through ingest_memory() when available to avoid response generation.
src/integrations/postgres_state_store.py	New PostgreSQL-backed lifecycle state store with revision/history/quarantine tables.
src/integrations/mcp_server.py	Adds runtime-state persistence, idempotency handling, refresh-on-operation, sleep leases, and Postgres store selection.
src/core/encoder.py	Threading import cleanup and conditional hierarchical extraction behavior for LangChain models.
src/chat/engine.py	Adds ingest_memory(), attaches ingest metadata, strict storage option, and exports/imports runtime state.
sdk/js/package.json	Bumps JS package version to 1.0.0.
scripts/run_product_qualification.py	Adds embedded front-door check + installed LangChain smoke; installs wheel with [langchain].
scripts/installed_wheel_smoke.py	Switches installed-wheel smoke to use embedded SCM instead of direct engine.
scripts/installed_langchain_smoke.py	New installed-wheel LangChain smoke test using embedded middleware.
scripts/check_embedded_front_door.py	New guard script preventing server/URL-first docs/examples in primary path.
scm/langchain.py	New LangChain 1.x middleware, model adapter, and create_scm_agent() helper.
scm/embedded.py	New embedded runtime (SCM, SCMUser) built on UserEnginePool + state stores.
scm/context.py	New framework-neutral SCMContext identity carrier.
scm/init.py	Exposes SCM, SCMContext, and aliases; keeps legacy exports available.
research/REPRODUCIBILITY_MANIFEST_2026_07_09.json	Adds commit/release URL and DOI metadata updates.
README.md	Rewrites primary README to lead with embedded Python + LangChain usage.
pyproject.toml	Bumps version/description; adds filelock; updates LangChain extras to >=1,<2 ranges.
examples/04_langchain_agent.py	New example showing embedded middleware attachment to LangChain 1.x.
examples/03_agent_integration.py	Updates provider-neutral agent integration example to use embedded SCM.
examples/01_quickstart.py	Updates quickstart to use embedded SCM and context manager style.
docs/README.md	Adds adoption plan link and updates docs index for embedded-first messaging.
docs/QUALITY_GATES.md	Updates gate descriptions to include embedded + LangChain and perf budgets.
docs/PUBLISH.md	Updates publishing checklist for 1.0.0 and embedded-first verification.
docs/PRODUCT.md	Updates first-run principle and five-tool contract wording for embedded-first.
docs/PRODUCT_ADOPTION_PLAN.md	New phased product adoption plan document.
docs/LANGCHAIN_GUIDE.md	Replaces server-based guide with embedded LangChain 1.x middleware guide.
docs/INTEGRATIONS.md	Updates roadmap wording for managed cloud + JS client positioning.
docs/GETTING_STARTED.md	Updates getting started for embedded Python/LangChain and Postgres env config.
docs/ARTIFACTS.md	Adds DOI and updates tracked paper bytes/sha.
docs/ARCHITECTURE.md	Updates architecture doc to reflect embedded runtime + snapshot v2.
Dockerfile	Updates container defaults to hash embeddings and includes scm/ package path.
CHANGELOG.md	Adds v1.0.0 release notes and updates historical note references.
.github/workflows/release.yml	Adds build + checksums upload to GitHub release assets.
.github/workflows/ci.yml	Adds LangChain extra to CI install and a dedicated Postgres embedded job; updates wheel install checks.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.