Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 27 additions & 9 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,20 @@

All notable changes to RushTI are documented in this file.

## Unreleased — `feat/issue-156-chore-task-kind`
## [Unreleased]

- **Added: `--config PATH` CLI flag** (closes #164). Overrides the location of
`config.ini` (TM1 connection parameters) for a single invocation, on every
TM1-connecting command (`run`, `build`, `tasks …`, `resume`). Precedence:
`--config` > `RUSHTI_DIR` > legacy CWD > `config/`. The flag relocates only
`config.ini` — `settings.ini` (`--settings`) and `logging_config.ini` keep
their own resolution. A missing path fails fast (exit 1, no traceback). The
resolved path is threaded explicitly into the TM1 connection layer rather
than mutating a global. Enables sharing one read-only `config.ini` with other
tm1py utilities. No behavioural change when the flag is absent. See
`docs/adr/0003-config-ini-location-resolution.md`.

## [2.3.0] - 2026-06-17

- **Added: TM1 chore execution as a first-class task kind** (closes #156).
Mixed process + chore taskfiles are now supported across JSON, TXT, and
Expand All @@ -29,27 +42,32 @@ All notable changes to RushTI are documented in this file.
- Dashboard: the per-task "Process" column is replaced by a unified
"Task target" column with a `[P]` / `[C]` kind indicator so process
and chore rows render side-by-side.

## Unreleased — docs: `--mode` for cube reads (#160)

- **Docs fix:** corrected the long-standing claim that `--mode` is
deprecated/ignored. It is only auto-detected (and ignored) for **file
sources** (`--tasks`). A **cube read** (`--tm1-instance`) cannot infer
the mode — every workflow occupies the same cube measures — so it
deprecated/ignored (#160). It is only auto-detected (and ignored) for
**file sources** (`--tasks`). A **cube read** (`--tm1-instance`) cannot
infer the mode — every workflow occupies the same cube measures — so it
defaults to `norm` and silently drops the `predecessors` measure unless
`--mode opt` is passed. This caused predecessors to disappear from
cube-read execution plans (e.g. `Sample_Optimal_Mode`). Updated the CLI
reference, settings reference, migration guide, TM1 integration guide
(new "Choosing the mode for cube reads" section), getting-started
task-files page, and the `rushti run --help` text.

## Unreleased — `feat/issue-154-v12-load-results`
## [2.2.3] - 2026-06-01

- Fix: `TM1Service` kwarg collision when connection parameters are set in
`config.ini` (#158).

## [2.2.2] - 2026-05-20

- Fix: `rushti build` now installs a TM1-version-aware `}rushti.load.results`
TI (closes #154). On v12 targets the body no longer references the removed
`CubeGetLogChanges` / `CubeSetLogChanges` / `ExecuteCommand` functions;
source-file cleanup uses the TM1-native `ASCIIDelete` instead of shelling
out via `cmd /c del`. The v11 body is unchanged.

## [2.2.1] - 2026-05-20

- **Per-workflow `tm1_instance` setting** for results push and auto-load.
Set it inside a JSON taskfile's `settings` block to override the
`settings.ini` default per workflow. Resolution chain (highest wins):
Expand All @@ -71,7 +89,7 @@ All notable changes to RushTI are documented in this file.
`--tm1-instance` instead. The legacy flag is aliased and continues to
work; a `DEPRECATION:` warning fires on use.

## Unreleased — `feat/issue-146-detailed-results`
## [2.2.0] - 2026-05-18

- Add `--detailed-results` for per-execution cube rows (closes #146).
- Log migration hint at run start when `--detailed-results` is enabled.
Expand Down
29 changes: 28 additions & 1 deletion CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,34 @@ The effective value for a settings-driven knob is resolved in this order

Three knobs are *not* settings-driven and don't follow this chain:
- Per-task `instance` and `process` — taskfile-only, no fallback.
- TM1 connection parameters — `config.ini` only.
- TM1 connection parameters — `config.ini` only (the *values*; the file's
*location* is resolved separately, see below).

---

## config.ini location resolution

`config.ini` holds TM1 connection parameters only; its *location* is resolved
by `resolve_config_path("config.ini", cli_path=…)` (`app_paths.py`) in this
order (highest wins):

1. **`--config` CLI flag** — explicit path to a `config.ini` file. Present on
every TM1-connecting command (`run`, `build`, `tasks …`, `resume`); not on
`stats`/`db`, which touch only local SQLite + settings.ini. A missing path
fails fast with a clean error, no traceback.
2. **`RUSHTI_DIR` env var** — looks in `{RUSHTI_DIR}/config/config.ini`.
3. **Legacy CWD** — `./config.ini` (deprecated, warns once).
4. **`config/config.ini`** — the recommended default location.

`--config` relocates **only `config.ini`** — settings.ini keeps `--settings`,
logging_config.ini keeps its own resolution, and `RUSHTI_DIR` still governs
those siblings. The flag exists so RushTI can share one read-only `config.ini`
with other tm1py utilities (e.g. OptimusPy) instead of duplicating it. When
`--config` is absent, resolution is unchanged from prior behaviour.

The resolved path is **threaded explicitly** as `config_path` into the TM1
connection layer (`connect_to_tm1_instance`, `setup_tm1_services`); the
module-level `CONFIG` global in `cli.py` is now only the no-flag default.

---

Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,11 @@ user = admin
password = apple
```

RushTI looks for `config.ini` under `config/` (or `RUSHTI_DIR`). To point a
single run at a different file — e.g. a `config.ini` shared with other tm1py
utilities — pass `--config PATH` to any TM1-connecting command (`run`, `build`,
`tasks`, `resume`).

**2. Create a task file**

```json
Expand Down
12 changes: 12 additions & 0 deletions docs/advanced/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ rushti --tasks FILE [options] # 'run' is the default command
| `--retries` | `-r` | INT | `0` | Retry count for failed TI executions. Uses exponential backoff. |
| `--result` | `-o` | PATH | *(empty)* | Output CSV path for execution summary. Omit to skip CSV creation. |
| `--settings` | `-s` | PATH | auto | Path to `settings.ini`. Auto-discovered if omitted. |
| `--config` | | PATH | auto | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. Relocates **only** `config.ini`; `settings.ini` and `logging_config.ini` keep their own resolution. A missing path fails fast (exit 1, no traceback). |
| `--mode` | `-m` | CHOICE | `norm` | Execution mode: `norm` or `opt`. **Ignored for file sources** (`--tasks`), where mode is auto-detected from file content. **Required for cube reads** (`--tm1-instance`) when the workflow uses explicit `predecessors` — pass `--mode opt`, otherwise the cube is read in `norm` mode and predecessors are dropped. See [TM1 integration → Choosing the mode for cube reads](../features/tm1-integration.md#choosing-the-mode-for-cube-reads). |
| `--exclusive` | `-x` | FLAG | `false` | Enable exclusive mode. Waits for other RushTI sessions to finish. |
| `--force` | `-f` | FLAG | `false` | Bypass exclusive mode checks and run immediately. |
Expand Down Expand Up @@ -123,6 +124,7 @@ rushti resume --tasks FILE [options] # auto-finds checkpoint
| `--resume-from` | | STR | *(none)* | Resume from a specific task ID, overriding checkpoint state. |
| `--max-workers` | `-w` | INT | *(from settings)* | Maximum parallel workers. |
| `--settings` | `-s` | PATH | auto | Path to `settings.ini`. |
| `--config` | | PATH | auto | Path to `config.ini` (TM1 connection parameters). Forwarded into the resumed run. |
| `--force` | `-f` | FLAG | `false` | Force resume even if checkpoint does not match the current task file. |
| `--log-level` | `-L` | CHOICE | `INFO` | Override log level. |

Expand Down Expand Up @@ -164,6 +166,7 @@ rushti tasks export --tm1-instance tm1srv01 -W DailyETL --output daily.json
| `--tm1-instance` | | STR | Read source from TM1 |
| `--workflow` | `-W` | STR | Workflow in TM1 |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -183,6 +186,7 @@ rushti tasks expand --tasks template.json --output expanded.json
| `--tm1-instance` | | STR | TM1 source instance |
| `--workflow` | `-W` | STR | Workflow in TM1 |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -203,6 +207,7 @@ rushti tasks visualize --tasks daily-etl.json --output dag.html --show-parameter
| `--tm1-instance` | | STR | TM1 source instance |
| `--workflow` | `-W` | STR | Workflow in TM1 |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -224,6 +229,7 @@ rushti tasks validate --tasks daily-etl.json --json > validation.json
| `--tm1-instance` | | STR | TM1 source instance |
| `--workflow` | `-W` | STR | Workflow in TM1 |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -241,6 +247,7 @@ rushti tasks push --tasks daily-etl.json --tm1-instance tm1srv01
| `--tm1-instance` | | STR | TM1 instance to push the taskfile to. Context disambiguates the role — on `tasks push` this is the destination. |
| `--target-tm1-instance` | | STR | **Deprecated alias** for `--tm1-instance`. Still works; emits a `DEPRECATION:` warning when used. |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -265,6 +272,7 @@ rushti stats list tasks --workflow daily-etl --limit 50
| `--workflow` | `-W` | STR | Workflow (*required*) |
| `--limit` | `-n` | INT | Maximum items to show (default: 20) |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -283,6 +291,7 @@ rushti stats export --workflow daily-etl --run-id 20260115_103000 --output run.c
| `--run-id` | `-r` | STR | Specific run ID to export (all runs if omitted) |
| `--output` | `-o` | PATH | Output CSV file path (*required*) |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand All @@ -301,6 +310,7 @@ rushti stats visualize --workflow daily-etl --runs 10 --output dashboard.html
| `--runs` | `-n` | INT | Number of recent runs to display (default: 5) |
| `--output` | `-o` | PATH | Output HTML file path (default: `visualizations/rushti_dashboard_<id>.html`) |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

!!! info "DAG reflects the latest executed run"
Both the dashboard and the DAG are sourced from the stats database — not from the live taskfile or TM1 cube definition. Editing a workflow definition without re-running it will not update either visualization. Re-run the workflow to refresh.
Expand Down Expand Up @@ -328,6 +338,7 @@ rushti stats analyze --workflow daily-etl --report report.json --ewma-alpha 0.5
| `--runs` | `-n` | INT | Number of recent runs to analyze (default: 10) |
| `--ewma-alpha` | | FLOAT | EWMA smoothing factor 0--1 (default: 0.3). Higher = more weight on recent runs. |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |

---

Expand Down Expand Up @@ -374,6 +385,7 @@ rushti build --tm1-instance INSTANCE [options]
| `--tm1-instance` | | STR | TM1 instance name from `config.ini` (*required*) |
| `--force` | `-f` | FLAG | Delete and recreate existing objects |
| `--settings` | `-s` | PATH | Path to `settings.ini` |
| `--config` | | PATH | Path to `config.ini` (TM1 connection parameters). Overrides `RUSHTI_DIR`/default location. |
| `--log-level` | `-L` | CHOICE | Override log level |

### Examples
Expand Down
16 changes: 16 additions & 0 deletions docs/advanced/settings-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,22 @@ RushTI searches for `settings.ini` in the following order:

If no file is found, built-in defaults are used for all settings.

### config.ini location

`config.ini` (TM1 connection parameters) is resolved separately, with its own
precedence (highest wins):

1. `--config` CLI flag (TM1-connecting commands: `run`, `build`, `tasks …`, `resume`)
2. `RUSHTI_DIR` environment variable (looks for `config/config.ini` under this directory)
3. `./config.ini` (current directory -- legacy location, deprecation warning emitted)
4. `./config/config.ini` (recommended location)

`--config` relocates **only** `config.ini`; `settings.ini` keeps `--settings`
and `logging_config.ini` keeps its own resolution. Use it to share a single
read-only `config.ini` with other tm1py utilities. A missing `--config` path
fails fast (exit 1, no traceback). When `--config` is absent, resolution is
unchanged.

---

## Configuration Sections
Expand Down
22 changes: 22 additions & 0 deletions src/rushti/app_paths.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
can be emitted once logging is initialized.
"""

import argparse
import logging
import os

Expand All @@ -15,6 +16,7 @@
"CURRENT_DIRECTORY",
"resolve_config_path",
"log_legacy_path_warnings",
"add_config_arg",
]


Expand Down Expand Up @@ -73,6 +75,26 @@ def resolve_config_path(filename: str, warn_on_legacy: bool = True, cli_path: st
return new_path


def add_config_arg(parser: argparse.ArgumentParser) -> None:
"""Add the ``--config`` argument to a parser.

Long-form only (``-c`` is taken by ``resume --checkpoint``). Relocates
only ``config.ini`` (TM1 connection parameters); ``settings.ini`` and
``logging_config.ini`` keep their own resolution. The resolved value is
fed to :func:`resolve_config_path` as ``cli_path``.
"""
parser.add_argument(
"--config",
dest="config",
default=None,
metavar="FILE",
help=(
"Path to a config.ini file (TM1 connection parameters). "
"Overrides RUSHTI_DIR/default location."
),
)


def log_legacy_path_warnings(logger: logging.Logger) -> None:
"""Log deprecation warnings for any legacy paths that were used.

Expand Down
19 changes: 17 additions & 2 deletions src/rushti/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from TM1py.Utils import integerize_version

from rushti.app_paths import (
add_config_arg,
log_legacy_path_warnings,
resolve_config_path,
)
Expand Down Expand Up @@ -273,6 +274,9 @@ def create_argument_parser() -> argparse.ArgumentParser:
# Add taskfile source arguments
add_taskfile_source_args(parser, required=False, include_settings=True)

# Add --config (relocates config.ini only)
add_config_arg(parser)

parser.add_argument(
"--max-workers",
"-w",
Expand Down Expand Up @@ -426,6 +430,7 @@ def parse_named_arguments(argv: list):
"workflow": workflow,
"log_level": args.log_level,
"detailed_results": args.detailed_results,
"config": args.config,
}

return tasks_file_path, cli_args
Expand Down Expand Up @@ -471,6 +476,7 @@ def parse_arguments(argv: list):
"exclusive": None,
"no_checkpoint": False,
"log_level": None, # Not supported in positional style
"config": None, # Legacy positional style does not support --config
}
return tasks_file, cli_args

Expand Down Expand Up @@ -626,6 +632,14 @@ def main() -> int:
if resume_context:
cli_args.update(resume_context)

# Resolve config.ini location, threading the resolved path explicitly into
# the TM1 connection layer (CLI --config > RUSHTI_DIR > legacy CWD > config/).
# Fail fast with a clean message (no traceback) on an explicit bad path.
try:
config_path = resolve_config_path("config.ini", cli_path=cli_args.get("config"))
except FileNotFoundError:
sys.exit(f"RushTI: --config file not found: {cli_args.get('config')}")

# Apply log level override if specified
apply_log_level(cli_args.get("log_level"))

Expand All @@ -645,7 +659,7 @@ def main() -> int:
try:
from rushti.tm1_integration import read_taskfile_from_tm1, connect_to_tm1_instance

tm1_source = connect_to_tm1_instance(tm1_instance, CONFIG)
tm1_source = connect_to_tm1_instance(tm1_instance, config_path)
try:
# Resolve execution mode for TM1 read (affects predecessor parsing)
tm1_mode = "opt" if cli_args.get("execution_mode") == ExecutionMode.OPT else "norm"
Expand Down Expand Up @@ -752,6 +766,7 @@ def main() -> int:
tasks_file_path=tasks_file_path_for_services if not tm1_taskfile else None,
workflow=workflow,
exclusive=exclusive_mode,
config_path=config_path,
tm1_instances=tm1_instances_needed,
)

Expand Down Expand Up @@ -1106,7 +1121,7 @@ def main() -> int:
)
tm1_upload = connect_to_tm1_instance(
upload_instance,
CONFIG,
config_path,
)
try:
results_df = build_results_dataframe(
Expand Down
Loading