AI::LLM module

[ziflex]

This pull request adds the new ai/llm module to the project, making advanced LLM (Large Language Model) features available to Ferret queries. It updates the workspace configuration to include the new module and provides comprehensive documentation on its usage, capabilities, and limitations.

Module Addition and Workspace Updates:

• Added the ai/llm module to the project, exposing text generation, chat, summarization, structured extraction, and classification features under the AI::LLM namespace for Ferret.
• Updated go.work to include ./modules/ai/llm in the workspace and added a replace directive for local development. [1] [2]
• Updated go.work.sum to reflect new and changed dependencies required by the ai/llm module. [1] [2]

Documentation:

• Updated the main README.md to list the new ai/llm module and link to its documentation.
• Added a detailed README.md for modules/ai/llm covering:
  • Registration and provider configuration
  • Function signatures and options
  • Security and credential handling
  • Query API integration and usage patterns
  • Session management and limitations

[Copilot]

Pull request overview

This pull request introduces a new modules/ai/llm contrib module that exposes provider-backed LLM capabilities (generation/chat/summarize/extract/classify + local sessions) to Ferret under the AI::LLM namespace, wires it into the workspace, and documents its public contract.

Changes:

• Added the ai/llm module with provider-neutral core contracts, Ferret runtime bindings, and an OpenAI Responses API provider implementation.
• Added module-level, core, lib, and provider tests covering option validation, query integration, session lifecycle/transactionality, and error sanitization.
• Updated workspace/runtime wiring and documentation to include and describe the new module.

Reviewed changes

Copilot reviewed 71 out of 72 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/runtime/main.go	Registers the new ai/llm module in the runtime test harness.
tests/runtime/go.mod	Adds the modules/ai/llm dependency for the runtime test runner.
README.md	Lists the new ai/llm module in the repository module index.
go.work	Adds the module to the workspace and a local replace for development.
go.work.sum	Updates workspace dependency checksums after adding the new module.
modules/ai/llm/README.md	Adds end-user documentation for AI::LLM (API, options, sessions, limitations).
modules/ai/llm/doc.go	Package documentation for the new module.
modules/ai/llm/options.go	Module options for injecting/replacing provider factories.
modules/ai/llm/llm.go	Module wiring: provider registry, Ferret namespace registration, session-scope hooks.
modules/ai/llm/llm_test.go	End-to-end module behavior tests via Ferret engine runs.
modules/ai/llm/fake_backend_test.go	Test backend used by module-level tests.
modules/ai/llm/fake_provider_factory_test.go	Fake provider factory used to validate override behavior.
modules/ai/llm/go.mod	Declares module dependencies (Ferret, OpenAI SDK, JSON Schema compiler, common pkg).
modules/ai/llm/go.sum	Adds dependency checksums for the new module.
modules/ai/llm/lib/doc.go	Documents the lib bridge package.
modules/ai/llm/lib/lib.go	Registers AI::LLM Ferret functions and arities.
modules/ai/llm/lib/lib_test.go	Verifies namespace function registration.
modules/ai/llm/lib/model.go	Implements MODEL function via provider registry, supports {session:true} shortcut.
modules/ai/llm/lib/session.go	Implements SESSION, RESET, and FORK Ferret functions.
modules/ai/llm/lib/history.go	Implements HISTORY Ferret function (visible history projection).
modules/ai/llm/lib/operation.go	Implements GENERATE, CHAT, SUMMARIZE, EXTRACT, CLASSIFY dispatch to core.
modules/ai/llm/lib/operation_test.go	Tests lib arity validation, value decoding, and returned Ferret values.
modules/ai/llm/lib/test_backend_test.go	Stub backend for lib-level tests.
modules/ai/llm/core/doc.go	Documents provider-neutral core package.
modules/ai/llm/core/contracts.go	Defines the provider-neutral contracts, error codes, and operation types.
modules/ai/llm/core/errors.go	Typed error type with stable code + optional operation annotation.
modules/ai/llm/core/error_helpers.go	Helpers for extracting codes and sanitizing/wrapping operation errors.
modules/ai/llm/core/errors_test.go	Validates stability/sanitization behavior of error handling.
modules/ai/llm/core/option_values.go	Shared option decoding primitives + unknown-key rejection.
modules/ai/llm/core/model_options.go	Decodes MODEL options (explicit apiKey/model, no env fallbacks).
modules/ai/llm/core/execution_options.go	Decodes common execution options (temperature/tokens/timeout).
modules/ai/llm/core/semantic_options.go	Decodes operation semantic options and combined function options.
modules/ai/llm/core/messages.go	Decodes strict v1 chat message arrays.
modules/ai/llm/core/labels.go	Decodes and validates label sets for classification.
modules/ai/llm/core/schema_decode.go	Decodes and compiles JSON Schema from Ferret values.
modules/ai/llm/core/schema_helpers.go	Helpers for external $ref detection and decoder trailing-data detection.
modules/ai/llm/core/schema.go	Compiles and validates schema + converts validated output to Ferret values.
modules/ai/llm/core/query.go	Implements Queryable integration (QUERY, QUERY ONE, WITH/OPTIONS separation).
modules/ai/llm/core/request_helpers.go	Shared helpers (instruction joining, message copying, label validation).
modules/ai/llm/core/request_builder.go	Deterministically builds generation and structured requests for providers.
modules/ai/llm/core/execute.go	Central dispatcher for all ops and query execution, including timeout context.
modules/ai/llm/core/stateless_model.go	Implements stateless model handle (opaque display, queryable, hashing).
modules/ai/llm/core/local_session.go	Implements local session semantics (serialized calls, transactional history).
modules/ai/llm/core/session_context.go	Context plumbing for per-run session scope lifecycle.
modules/ai/llm/core/session_scope.go	Tracks and closes all sessions created during one execution.
modules/ai/llm/core/session_options.go	Decodes and validates session options and v1 context policy.
modules/ai/llm/core/session_helpers.go	Normalizes programmatic session options for local sessions.
modules/ai/llm/core/resource.go	Resource ID generation for model/session identity.
modules/ai/llm/core/fake_executor_test.go	Core fake executor used by core-level tests.
modules/ai/llm/core/fake_factory_test.go	Core fake factory used by registry tests.
modules/ai/llm/core/test_helpers_test.go	Core test helpers for decoding and assertions.
modules/ai/llm/core/model_test.go	Tests model opacity, concurrency, and query behavior.
modules/ai/llm/core/options_test.go	Tests decoding/validation of model/execution/session options.
modules/ai/llm/core/schema_test.go	Tests schema compilation rules and numeric preservation behavior.
modules/ai/llm/core/query_test.go	Tests supported USING modes and WITH/OPTIONS separation.
modules/ai/llm/core/request_builder_test.go	Tests deterministic request construction and instruction ordering.
modules/ai/llm/core/registry.go	Provider registry supporting normalization and replacement.
modules/ai/llm/core/registry_helpers.go	Factory validation + typed-nil detection.
modules/ai/llm/core/registry_test.go	Tests provider normalization, replacement, and error cases.
modules/ai/llm/core/session_test.go	Tests session opacity, transactional commits, scope cleanup, and serialization.
modules/ai/llm/providers/openai/doc.go	Documents the OpenAI provider adapter package.
modules/ai/llm/providers/openai/config.go	Defines the fixed OpenAI base URL.
modules/ai/llm/providers/openai/factory.go	OpenAI provider factory implementing explicit credential requirements.
modules/ai/llm/providers/openai/factory_test.go	Tests factory validation and opaque model handling.
modules/ai/llm/providers/openai/http_client.go	Bridges OpenAI SDK HTTP calls through Ferret’s network policy client.
modules/ai/llm/providers/openai/request.go	Maps core requests to OpenAI Responses API request params.
modules/ai/llm/providers/openai/response.go	Normalizes OpenAI Responses API responses into core response shape.
modules/ai/llm/providers/openai/errors.go	Sanitizes SDK/transport errors into stable core error codes.
modules/ai/llm/providers/openai/fake_http_client_test.go	Fake HTTP client for provider tests.
modules/ai/llm/providers/openai/test_helpers_test.go	Provider test helpers for response payloads and typed error checks.
modules/ai/llm/providers/openai/executor.go	Executes provider requests via OpenAI SDK with Ferret network policy and no retries.
modules/ai/llm/providers/openai/executor_test.go	Tests request mapping, structured format, refusal/error normalization, and policy enforcement.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 71 out of 72 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 73 out of 74 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 73 out of 74 changed files in this pull request and generated 1 comment.