feat(server): embedded secret delivery via TaskDef.runtimeMetadata + AgentspanSecretsDAO#312
Merged
Merged
Conversation
…AgentspanSecretsDAO Server-side of host-delivered worker secrets when embedded (agentspan.embedded=true): - Declare each worker tool's credential names on TaskDef.runtimeMetadata at registration (AgentService + AgentCompiler.collectToolCredentials/collectAgentCredentials), extended to the non-worker user-code task kinds (guardrail/callback/stop_when/ gate/instructions/router/graph) via agent-level creds. Gated on EmbeddedMode. - AgentspanSecretsDAO: bridges conductor's SecretsDAO to AgentSpan's encrypted CredentialStoreProvider (scoped to the anonymous/system user), selected by conductor.secrets.type=agentspan; the embedded conductor's RuntimeMetadataResolver resolves declared names at task poll onto the wire-only Task.runtimeMetadata. - Store beans (datasource, master key, schema migrator, store provider) re-gated on conductor.secrets.type=agentspan (defaulted on in application.properties). - Native secret mechanism gated on agentspan.embedded; LLM keys via host AI integration. Tests: WorkerRuntimeMetadataTest, AgentspanSecretsDAOTest, NativeSecretGatingTest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…module It is a standalone-server-only @ControllerAdvice (its only consumers are the server-module masking tests). Co-locating it in conductor-agentspan-server removes it from the embeddable library, where it never applied. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lobal) store Remove userId from the CredentialStoreProvider SPI (get/set/delete/list). The EncryptedDbCredentialStoreProvider keeps its credentials_store schema (the user_id column stays, part of the (user_id, name) PK) but always reads/writes under the fixed 00000000-0000-0000-0000-000000000000 owner, so lookups are effectively by name. - SPI: drop userId from all four methods. - EncryptedDbCredentialStoreProvider: DEFAULT_USER_ID constant in the SQL. - Consumers drop the userId argument: CredentialResolutionService, SecretController (currentUserId() retained for audit logging), CredentialEnvSeeder, AgentspanSecretsDAO. - Per-user isolation is gone; delete the obsolete isolation tests (WorkerCredentialsIntegrationTest, AgentspanAIModelProviderPerUserTest) and update the remaining tests to the no-userId signatures. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follows the CredentialStoreProvider single-scope change: resolve is now resolve(name), and the userId plumbing that fed it is removed. - CredentialResolutionService.resolve(name) — drop userId (logs no longer reference it). - WorkerController / CredentialAwareHttpTask / CredentialAwareMcpService: call resolve(name). HTTP/MCP keep the execution-token presence check as an auth gate, but the header resolution methods no longer take userId (resolveHeadersForUser -> resolveHeaders on HTTP). - AgentspanAIModelProvider: resolveUserApiKey/resolveUserCredential resolve globally; the now-dead extractUserIdFromTaskContext + the ExecutionTokenService dependency (field + constructor param) are removed (ExecutionTokenService itself is unchanged and still used by WorkerController et al.). - Tests updated to the no-userId signatures; drop dead per-user constants. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The execution-token mechanism gated the native out-of-process worker credential pull, but provides no real auth/tenancy in agentspan today (the credential store is global, userId already removed). Remove it; real security will be added separately. Credential delivery relies on host-resolved TaskDef.runtimeMetadata. DELETE: - ExecutionTokenService, WorkerController (/api/workers/secrets), ResolveRequest - Token-only tests: ExecutionTokenServiceTest, AgentEventListenerTokenRevocationTest, WorkerCredentialsTest MODIFY: - AgentService: drop the mint block + extractDeclaredCredentials/collectCredentialsRecursive; keep preallocatedExecutionId/setWorkflowId (host time-based IDs). The Lombok @requiredargsconstructor now serves tests too. - AgentEventListener: drop the token-revocation path. - AgentExceptionHandler: drop the 401 token handler. - Remove __agentspan_ctx__ end-to-end (only carried the token): AgentService writer, the forwarders in AgentCompiler/MultiAgentCompiler/ToolCompiler/JavaScriptBuilder/ PlanAndCompileTask, and the strips in AgentChatCompleteTaskMapper/PlanAndCompileTask. - Update NativeSecretGatingTest, AgentServiceTokenTest, MultiAgentCompilerTest, PlanAndCompileTaskTest. Survives: CredentialResolutionService (+ CredentialNotFoundException), CredentialAware Http/Mcp (resolve directly), credentialMasterKey/MasterKeyConfig. Consequence (server-only now): no worker credential-pull path remains; delivery is via TaskDef.runtimeMetadata. Coordinated follow-up in SDK PR #313 must remove the native token-pull fetchers and rely on the runtimeMetadata read-path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
….secrets.NAME}
Rewrite credential placeholders in HTTP/MCP/planner-context headers from ${NAME} to
conductor's native ${workflow.secrets.NAME}, resolved by ParametersUtils.substituteSecrets
wire-only at task hand-off (poll / async+sync system tasks) against the configured
SecretsDAO — AgentspanSecretsDAO by default. Plaintext is never persisted.
- ToolCompiler.rewriteCredentialPlaceholders + MultiAgentCompiler planner-context: always
emit ${workflow.secrets.NAME} (drop the embedded/#{ split; remove EmbeddedMode usage).
- Delete the now-redundant #{NAME} resolvers: CredentialAwareHttpTask +
CredentialAwareHttpTaskConfig (reverts to conductor's built-in @component("HTTP") HttpTask)
and CredentialAwareMcpService (reverts to conductor-ai's base @component MCPService). Their
tests are removed.
- CredentialResolutionService stays (still used by AgentspanAIModelProvider for LLM keys).
- Bonus: planner-context secrets now resolve in standalone too (previously only embedded),
since AgentspanSecretsDAO is the default SecretsDAO.
Tests updated: ToolCompilerTest, MultiAgentCompilerTest, EnrichToolsScriptTest assert the
${workflow.secrets.NAME} form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (plaintext leak)
The ${workflow.secrets.NAME} rewrite baked the reference into the enrich INLINE task's
'expression' input; conductor's substituteSecrets resolves any such reference in an INLINE
task's input to PLAINTEXT at that task's hand-off, and the plaintext then persisted via the
script's dynamicTasks output into the forked HTTP/MCP tasks' inputParameters — defeating the
wire-only guarantee. A secret containing a quote/backslash also broke the GraalJS script.
Two-form design:
- Transport form #{NAME} (dots allowed for JSONPath): inert to BOTH ParametersUtils passes,
rides safely through every INLINE hop (script text, $.mcpConfig/$.apiConfig data, toolSpecs).
ToolCompiler.rewriteCredentialPlaceholders now emits this everywhere headers are escaped.
- Real reference ${workflow.secrets.NAME}: applied ONLY at final placement into a real task's
inputParameters, where conductor defers it at binding and resolves wire-only at that task's
own hand-off:
* Java secretRefHeaders() for statically built tasks (LIST_MCP_TOOLS inputs, api spec-fetch
HTTP tasks — both variants).
* JS _sec() at dynamic-task emission in both enrich scripts (HTTP, MCP, API headers). The
'$' + '{workflow.secrets.' split keeps the contiguous reference literal out of the script
SOURCE so it can never be early-resolved.
Also widens both CREDENTIAL_PLACEHOLDER patterns to allow dotted names (${GCP_SVC.project_id}
-> ${workflow.secrets.GCP_SVC.project_id}, supported by conductor's resolveWithOptionalJsonPath),
fixing the dotted-JSONPath regression.
Tests (fail-first validated): leak tripwires assert the enrich script source never contains
'${workflow.secrets.' (ToolCompilerTest, EnrichToolsScriptTest); GraalJS emission tests assert
#{NAME} configs emit ${workflow.secrets.NAME} on dynamic HTTP/MCP task inputs, incl. dotted.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…re module They test ToolCompiler and JavaScriptBuilder, which live in conductor-agentspan — tests belong with the module of the class under test. The core module's test classpath already carries GraalJS for the script-execution harness. No content changes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…odule Tests belong with the module of the class under test. Moves the 29 pure unit tests that exercise conductor-agentspan classes (compilers, normalizers, models, services, utils, ai providers, ProviderController) plus SkillNormalizerTest's /skills JSON fixtures (used by it alone). Stays in conductor-agentspan-server: the @SpringBootTest/MockMvc integration tests, everything exercising server-module classes (encrypted store, seeder, migrator, master key, AgentspanSecretsDAO, CredentialMaskingResponseAdvice, skill package store), and ServerSmokeTest. Pure git mv — no content changes. Suites: core 523, server 159, all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ci-server.yml (Server CI): agentspan library + server module tests and the Go CLI tests. Triggers on server/**, cli/**. - ci-sdk.yml (SDK CI): the four SDK unit-test jobs plus all e2e suites — e2e is part of the SDK pipeline. Builds its own server JAR (-x test; server correctness is Server CI's job) so the workflows share no artifacts. Triggers on sdk/**, server/**, cli/** — server changes still prove the SDKs keep working via e2e. Job bodies are unchanged from the old ci.yml; this is a reorganization. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
runtimeMetadata is now the only worker credential-delivery path (the native execution-token pull was removed). Standalone resolves it end-to-end already: the built-in conductor core's RuntimeMetadataResolver + AgentspanSecretsDAO (the default SecretsDAO) resolve declared names at poll from the encrypted store. Gating the declaration on EmbeddedMode left standalone with no credential delivery at all (review finding #3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drops ci-sdk.yml (SDK unit tests + all e2e suites). Server CI (ci-server.yml) remains as the only workflow, covering the agentspan library/server modules and the Go CLI. SDK test coverage moves out of per-PR CI. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Server-side of host-delivered worker secrets when AgentSpan is embedded (
agentspan.embedded=true). Split out from #307 (server half). SDK half: companion PR.TaskDef.runtimeMetadataat registration.AgentServicestamps each worker tool's credential names (viaAgentCompiler.collectToolCredentials), extended to the non-worker user-code task kinds — guardrails, callbacks,stop_when, gates, instructions, routers, graph node/edge — using agent-level creds (collectAgentCredentials). Declarative/control-flow tasks (termination, transfer/handoff) stay empty. Gated onEmbeddedMode.isEmbedded().AgentspanSecretsDAObridges conductor's globalSecretsDAOto AgentSpan's encryptedCredentialStoreProvider, scoped to the anonymous/system user. Selected byconductor.secrets.type=agentspan; the embedded conductor'sRuntimeMetadataResolverresolves the declared names at each SIMPLE task's poll and injects values onto the wire-onlyTask.runtimeMetadata.conductor.secrets.type=agentspan(defaulted on inapplication.properties).agentspan.embedded; LLM keys via the host AI integration (no apiKey stamping). Conductor bumped to3.32.0-rc.5(carriesTaskDef/Task.runtimeMetadata).Tests
WorkerRuntimeMetadataTest,AgentspanSecretsDAOTest,NativeSecretGatingTest. Full:conductor-agentspan-server:testgreen (683/0).Validated live
Embedded host +
conductor.secrets.type=agentspan: a secret stored in the encrypted store resolves onto the polledTask.runtimeMetadata.Note: the design doc still describes the retired interim
__resolved_credentials__path — doc cleanup tracked separately.🤖 Split from #307 with Claude Code