feat(python-sdk): python SDK scaffold

[czi-fsisenda]

Summary

Jira:

Scaffold for evaluators for the Python sdk

• package setup
• abstract base class for all evaluators and supporting types
• using langchain to call LLMs
• abstracts: input validation, logging, metadata, etc
• unit tests

Test Plan

• Wrote automated tests
• Manually tested my changes, and here are the details:

[Copilot]

Pull request overview

Introduces an initial scaffold for the learning-commons-evaluators Python SDK, including core schema/config/error types, a base evaluator abstraction, LangChain-backed provider integration, settings loading utilities, and a pytest-based test suite with CI wiring.

Changes:

• Add core SDK modules (schemas, base evaluator, logging utilities, provider factory, settings loader) and package-level re-exports.
• Add unit tests for public imports, config/errors/schemas, logging, and LangChain provider behavior.
• Add Python SDK packaging + developer tooling (pyproject, Makefile, README) and a GitHub Actions workflow to run make verify.

Reviewed changes

Copilot reviewed 40 out of 41 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
sdks/python/tests/test_package_imports.py	Smoke tests for package-level import surface.
sdks/python/tests/test_logger.py	Unit tests for SDK logging helpers and formatting.
sdks/python/tests/settings/init.py	Test package marker for settings tests.
sdks/python/tests/schemas/test_text_complexity.py	Tests for text complexity schema types.
sdks/python/tests/schemas/test_metadata.py	Tests for metadata models and helpers.
sdks/python/tests/schemas/test_input_specs.py	Tests for input spec registry + discriminated union behavior.
sdks/python/tests/schemas/test_errors.py	Tests for error hierarchy and provider-error wrapping.
sdks/python/tests/schemas/test_config.py	Tests for config dataclasses and factory functions.
sdks/python/tests/schemas/test_common_inputs.py	Tests for common input fields and validation.
sdks/python/tests/schemas/init.py	Test package marker for schemas tests.
sdks/python/tests/providers/test_langchain.py	Tests for LangChain provider factory and token usage extraction.
sdks/python/tests/providers/init.py	Test package marker for providers tests.
sdks/python/tests/evaluators/init.py	Test package marker for evaluators tests.
sdks/python/tests/conftest.py	Shared pytest fixtures for schemas and config.
sdks/python/tests/init.py	Test package marker for SDK tests.
sdks/python/src/learning_commons_evaluators/settings/load_settings.py	TOML settings loading/parsing utilities and result wrapper.
sdks/python/src/learning_commons_evaluators/settings/init.py	Settings package documentation and purpose.
sdks/python/src/learning_commons_evaluators/schemas/text_complexity.py	Shared schemas for text complexity-related evaluators.
sdks/python/src/learning_commons_evaluators/schemas/metadata.py	Evaluation metadata models, enums, token usage, step metadata.
sdks/python/src/learning_commons_evaluators/schemas/input_specs.py	Input spec models + registry + discriminated union.
sdks/python/src/learning_commons_evaluators/schemas/evaluator.py	Core evaluator input/output schema abstractions and validation helpers.
sdks/python/src/learning_commons_evaluators/schemas/errors.py	SDK error hierarchy and provider-error mapping.
sdks/python/src/learning_commons_evaluators/schemas/config.py	Provider configs, evaluator config, prompt settings, and factories.
sdks/python/src/learning_commons_evaluators/schemas/common_inputs.py	Concrete input field types (text, grade) with validation + safe metadata.
sdks/python/src/learning_commons_evaluators/schemas/init.py	Schemas subpackage exports.
sdks/python/src/learning_commons_evaluators/py.typed	Marks package as typed for type checkers.
sdks/python/src/learning_commons_evaluators/providers/langchain.py	LangChain chat model factory + token usage extraction.
sdks/python/src/learning_commons_evaluators/providers/init.py	Provider subpackage exports.
sdks/python/src/learning_commons_evaluators/logger.py	Library-friendly logging utilities and helpers.
sdks/python/src/learning_commons_evaluators/evaluators/base.py	Base evaluator implementation (validation, logging, step execution, prompt chain steps).
sdks/python/src/learning_commons_evaluators/evaluators/init.py	Evaluators subpackage exports.
sdks/python/src/learning_commons_evaluators/errors.py	Package-level error re-exports.
sdks/python/src/learning_commons_evaluators/config.py	Package-level config re-exports.
sdks/python/src/learning_commons_evaluators/_version.py	Centralized version/description lookup.
sdks/python/src/learning_commons_evaluators/init.py	Top-level public API exports for the SDK.
sdks/python/pyproject.toml	Packaging metadata, deps, Ruff/Mypy config, pytest settings.
sdks/python/README.md	SDK documentation, usage examples, dev workflow notes.
sdks/python/Makefile	Local/CI commands for lint/format/typecheck/tests.
sdks/python/.gitignore	Python SDK local ignores (venv, build, cache).
.github/workflows/test-sdk-python.yml	CI workflow to run Python SDK verification.
.gitattributes	Marks generated settings/contract files as generated for GitHub diff UX.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 44 out of 45 changed files in this pull request and generated 5 comments.

[adnanrhussain]

lgtm. thank you for iterating