From d9ab1aac5c87e4ed0925fa294e6d25171a0b9dc9 Mon Sep 17 00:00:00 2001 From: soustruh Date: Wed, 3 Jun 2026 08:56:05 +0200 Subject: [PATCH 1/2] docs: document configuration-level runtime.tag with example --- extend/docker-runner/index.md | 3 ++- integrate/jobs/index.md | 30 +++++++++++++++++++++++------- 2 files changed, 25 insertions(+), 8 deletions(-) diff --git a/extend/docker-runner/index.md b/extend/docker-runner/index.md index 53e85ed8..fd7727ea 100644 --- a/extend/docker-runner/index.md +++ b/extend/docker-runner/index.md @@ -86,7 +86,8 @@ all of them are optional: - `storage` --- configuration of [input and output mapping](/extend/common-interface/folders/); specific options correspond to the options of the [unload data](https://keboola.docs.apiary.io/#reference/tables/unload-data-asynchronously) and [load data](https://keboola.docs.apiary.io/#reference/tables/load-data-asynchronously) API calls. -- `runtime` --- configuration for modifying some image parameters at run time +- `runtime` --- [runtime settings](/integrate/jobs/#job-runtime-configuration) (`tag`, `backend`, `parallelism`); most notably `runtime.tag` +pins the Docker image tag that jobs of this configuration run, which is the usual way of testing a development build of a component - `processors` --- configuration of [Processors](/extend/component/processors/) - `authorization` --- OAuth authorization [injected to the configuration](/extend/common-interface/oauth/); not stored in the component configuration - `image_parameters` --- an arbitrary object passed from the [component](/extend/component/); not stored in the component configuration diff --git a/integrate/jobs/index.md b/integrate/jobs/index.md index 7cbf7636..04a01a0d 100644 --- a/integrate/jobs/index.md +++ b/integrate/jobs/index.md @@ -242,19 +242,35 @@ When creating a job, you need to provide `mode`. This can be one of `run`, `forc Use the `forceRun` mode to run a configuration that is disabled. The `debug` can be used during [Component Development & Debugging](https://developers.keboola.com/extend/component/tutorial/debugging/). ### Job Runtime configuration -You may provide runtime settings for a job. Runtime settings do not affect what the job does, it affects how the job does it. Current -available runtime settings are `backend` and `parallelism`. The `backend` parameter defines what Snowflake warehouse is used for the job and currently -affects mostly Snowflake transformations. Available values for backend size come from [Workspace Create API call](https://keboola.docs.apiary.io/#reference/workspaces/workspaces-collection/create-workspace) and are `small`, `medium`, `large`. +You may provide runtime settings for a job. Runtime settings do not affect what the job does, they affect how the job does it. The available runtime settings are: -The `parallelism` allows to run [Configuration Rows](https://help.keboola.com/components/#configuration-rows) (if present in the configuration) in -parallel. Allowed values are integer values and `infinity` which runs all rows in parallel. When parallelism is not specified, the rows are run sequentially. -You may also specify `tag` if you want to run the component with a specific version of code. This is mostly used during component development, testing -and debugging. +- `backend` --- the Snowflake warehouse used for the job; currently affects mostly Snowflake transformations. Available values for backend size come from the [Workspace Create API call](https://keboola.docs.apiary.io/#reference/workspaces/workspaces-collection/create-workspace) and are `small`, `medium`, `large`. +- `parallelism` --- runs [Configuration Rows](https://help.keboola.com/components/#configuration-rows) (if present in the configuration) in parallel. Allowed values are integer values and `infinity`, which runs all rows in parallel. When not specified, the rows are run sequentially. +- `tag` --- runs the component with a specific version of code. This is mostly used during component development, testing and debugging. Runtime parameters can be specified on various levels. The values can be specified in the component configuration. They can also be specified when creating a job, in which case it overrides the configuration. It may also be specified for an orchestration, in which case it overrides what is specified in individual jobs of that orchestration. +When stored in the component configuration, the runtime settings live in a top-level `runtime` node of the configuration JSON --- a sibling of `parameters`, +not inside it. For example, to pin all jobs of a configuration to a specific image tag: + +{% highlight json %} +{ + "parameters": { + "...": "..." + }, + "runtime": { + "tag": "my-branch-3" + } +} +{% endhighlight %} + +Jobs of this configuration then run the given image tag (the job detail shows the resolved value in its `tag` field) until the `runtime.tag` key +is removed from the configuration. This is the usual way of testing a development build of a component in a single project without affecting +other projects --- the tag in the [Developer Portal](/extend/publish/) stays untouched. Do not forget to remove the key when done; a pinned +configuration keeps running the old image even after new versions of the component are released. + ### Job Type Job can be of one of the four types `standard`, `container`, `phaseContainer` and `orchestrationContainer`. The `standard` is something which does actual work. Only standard jobs consume billable time and are counted towards consumption of any resources. Other job types are virtual containers encapsulating standard jobs. From af224857f9a0b3dc4ff98a0749853febc18ea7b3 Mon Sep 17 00:00:00 2001 From: Ondrej Popelka Date: Wed, 3 Jun 2026 14:35:59 +0200 Subject: [PATCH 2/2] clarified backend setting --- integrate/jobs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/integrate/jobs/index.md b/integrate/jobs/index.md index 04a01a0d..bfcafcab 100644 --- a/integrate/jobs/index.md +++ b/integrate/jobs/index.md @@ -244,7 +244,7 @@ Use the `forceRun` mode to run a configuration that is disabled. The `debug` can ### Job Runtime configuration You may provide runtime settings for a job. Runtime settings do not affect what the job does, they affect how the job does it. The available runtime settings are: -- `backend` --- the Snowflake warehouse used for the job; currently affects mostly Snowflake transformations. Available values for backend size come from the [Workspace Create API call](https://keboola.docs.apiary.io/#reference/workspaces/workspaces-collection/create-workspace) and are `small`, `medium`, `large`. +- `backend.type` --- for Snowflake transformations this is the size of the [Snowflake warehouse](https://github.com/keboola/developers-docs/pull/380/changes) used for the job; otherwise it affects the [container size](https://help.keboola.com/transformations/python-plain/#dynamic-backends). Available values for backend type are `small`, `medium`, `large`. - `parallelism` --- runs [Configuration Rows](https://help.keboola.com/components/#configuration-rows) (if present in the configuration) in parallel. Allowed values are integer values and `infinity`, which runs all rows in parallel. When not specified, the rows are run sequentially. - `tag` --- runs the component with a specific version of code. This is mostly used during component development, testing and debugging.