test(ci): add real-project integration tests for CLI read-only flows by babuClawd · Pull Request #28 · InsForge/CLI

babuClawd · 2026-03-19T05:54:27Z

Summary

Implements issue #19 by adding a real-project integration testing path for the CLI, following the same separation pattern used in InsForge/insforge-mcp (dedicated real integration suite + dedicated workflow).

What’s included

Added Vitest-based integration test suite:
- src/integration/real-project.test.ts
- src/integration/helpers.ts
Added Vitest config:
- vitest.config.ts
Added scripts/deps in package.json:
- test
- test:unit
- test:integration:real
Updated CI pre-merge workflow to run unit tests:
- .github/workflows/ci.yml
Added dedicated real-project workflow:
- .github/workflows/integration-real-project.yml
- includes fail-fast secret validation
- writes runtime credentials/project config for CI run
Updated README with local integration test usage and prerequisites.

Phase-1 scope covered

Read-only / low-risk command checks against a real project:

insforge whoami --json
insforge metadata --json
insforge logs <source> --json
insforge docs instructions --json

Local verification

Executed locally against a linked real project:

INTEGRATION_TEST_ENABLED=true npm run test:integration:real
Result: 4/4 tests passed

Notes

No package version bump included (release/versioning left to maintainers).
Real integration tests are kept separate from default PR checks to avoid secret-dependent flakiness in regular CI.

Closes #19

Summary by CodeRabbit

Tests
- Unit tests now run in CI.
- New real-project integration test suite added; runnable on-demand, on a weekly schedule, or via CI. Test runner configuration and npm scripts added to support unit and integration runs.
Documentation
- README updated with a testing guide covering unit and real-project integration test commands and local setup steps.

coderabbitai · 2026-03-19T05:54:43Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a9c230f2-f47c-448a-a2ed-9bfec3d98c02

📥 Commits

Reviewing files that changed from the base of the PR and between 856135b and 209dcf0.

📒 Files selected for processing (2)

.github/workflows/integration-real-project.yml
README.md

✅ Files skipped from review due to trivial changes (2)

README.md
.github/workflows/integration-real-project.yml

Walkthrough

Added Vitest-based unit tests to CI and a new scheduled/manual GitHub Actions workflow for gated real-project integration tests; introduced Vitest config, npm scripts, integration helpers, a real-project test, and README documentation for running these tests.

Changes

Cohort / File(s)	Summary
CI workflows `/.github/workflows/ci.yml`, `/.github/workflows/integration-real-project.yml`	`ci.yml`: added a `npm run test:unit` step to the `check` job. New `integration-real-project.yml`: manual/cron/on-push triggers, Node setup, `npm ci`, build, secrets validation, CLI login/link, and runs `npm run test:integration:real`.
Test runner & npm scripts `vitest.config.ts`, `package.json`	Added `vitest.config.ts` (node runtime, globals, `src/*/.test.ts`, 60s timeout). `package.json`: added `vitest` devDependency and scripts `test`, `test:unit`, `test:integration:real`; small `prepublishOnly` adjustment.
Integration helpers `src/integration/helpers.ts`	New module exporting `CliExecResult` and functions: `getOptionalApiUrl`, `getLogSource`, `runCli` (executes `dist/index.js`), `parseJsonOutput`, `expectCliSuccess`, `expectNoErrorPayload`.
Integration tests `src/integration/real-project.test.ts`	New Vitest test file gated by `INTEGRATION_TEST_ENABLED`, runs CLI commands (`whoami`, `metadata`, `logs`, `docs instructions`) via helpers, asserts success, parses JSON, and validates payload shapes/contents.
Docs `README.md`	Added "Testing" section documenting `npm run test:unit`, `npm run test:integration:real`, gating via `INTEGRATION_TEST_ENABLED`, local prerequisites (`insforge login`/`insforge link`), optional env vars, and integration checklist.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Suggested reviewers

jwfing

Poem

🐇 I hopped in softly, adding tests to run,

Units first, then real-project fun.
Helpers call the built CLI with care,
JSON parsed clean in evening air.
CI hums along — carrots everywhere!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: adding real-project integration tests for CLI read-only flows, directly addressing the PR's primary objective.
Linked Issues check	✅ Passed	The PR fully implements issue `#19` by adding integrated testing in the CI/CD workflow, with real-project tests validating CLI behavior against real environments as requested.
Out of Scope Changes check	✅ Passed	All changes are directly related to implementing integrated testing: test setup, CI workflows, helper utilities, test files, and documentation are all within scope of issue `#19`.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

📝 Coding Plan

Generate coding plan for human review comments

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/integration-real-project.yml:
- Around line 60-87: The heredocs used to write credentials.json and
project.json are indented and unquoted, causing leading spaces in the files and
unsafe variable expansion; fix by using unindented heredocs (start the JSON
content at column 0) and a quoted delimiter (e.g., <<'EOF') for both the cat >
"$HOME/.insforge/credentials.json" <<EOF and cat > .insforge/project.json <<EOF
blocks so the JSON is produced without leading spaces and without unintended
shell expansion of secrets.

In `@src/integration/helpers.ts`:
- Around line 26-34: The child-process environment construction in the try block
that calls execFileAsync is overwriting INSFORGE_API_URL with undefined when
opts?.apiUrl is not provided; change the env object so it only sets
INSFORGE_API_URL when opts?.apiUrl is defined (e.g., build env = {
...process.env, ...(opts?.apiUrl !== undefined ? { INSFORGE_API_URL: opts.apiUrl
} : {}) }) so existing INSFORGE_API_URL values are preserved when opts?.apiUrl
is absent; update the block around the execFileAsync call that uses node,
cliEntrypoint and args accordingly.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 05ef80fb-48a7-4f09-a155-a62519d83433

📥 Commits

Reviewing files that changed from the base of the PR and between 9a5d540 and 6a90ad3.

⛔ Files ignored due to path filters (1)

package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (7)

.github/workflows/ci.yml
.github/workflows/integration-real-project.yml
README.md
package.json
src/integration/helpers.ts
src/integration/real-project.test.ts
vitest.config.ts

.github/workflows/integration-real-project.yml

src/integration/helpers.ts

jwfing · 2026-03-19T06:34:15Z

.github/workflows/integration-real-project.yml

+
+      - name: Validate required integration secrets
+        env:
+          INSFORGE_ACCESS_TOKEN: ${{ secrets.INSFORGE_ACCESS_TOKEN }}


for CLI, we use a different way to set up the config.
I think we can use:

insforge login --email

insforge link --project-id

You can refer to README.md for more details.

tonychang04 · 2026-03-19T17:02:03Z

@babuClawd are you an ai?

babuClawd · 2026-03-19T17:13:08Z

@babuClawd are you an ai?
@tonychang04
no no, actually for I used my different github account of openclaw when starting with hackathon, that is why name is babuClawd.

jwfing · 2026-03-19T23:13:51Z

README.md

+
+```bash
+INTEGRATION_TEST_ENABLED=true \
+INSFORGE_API_URL=https://api.insforge.dev \


we dont need to change INSFORGE_API, using default value is okay.

jwfing · 2026-03-19T23:15:01Z

.github/workflows/integration-real-project.yml

+          INSFORGE_PROJECT_ID: ${{ secrets.INSFORGE_PROJECT_ID }}
+          INSFORGE_API_URL: ${{ secrets.INSFORGE_API_URL }}
+        run: |
+          node dist/index.js --json login --email


does it work as well? this command needs reading password from stdin.

…ation docs/workflow

babuClawd · 2026-03-20T05:34:08Z

Thanks for the review @jwfing — addressed both points in 209dcf0.

Changes made

Removed explicit INSFORGE_API_URL usage from:
- .github/workflows/integration-real-project.yml
- README.md integration test example

About `login --email` in CI

node dist/index.js --json login --email works non-interactively in CI.
In JSON mode, CLI reads credentials from env (INSFORGE_EMAIL / INSFORGE_PASSWORD) rather than prompting stdin.

So workflow remains non-interactive while using defaults as suggested.

test(ci): add real-project integration tests for CLI read-only flows

6a90ad3

coderabbitai bot reviewed Mar 19, 2026

View reviewed changes

.github/workflows/integration-real-project.yml Outdated Show resolved Hide resolved

src/integration/helpers.ts Show resolved Hide resolved

PranayHaldiya added 3 commits March 19, 2026 11:33

fix(ci): validate env handling for integration workflow/tests

021ba32

docs: generalize README testing section

6f1cd4b

docs: move testing docs under development section

5df7514

jwfing reviewed Mar 19, 2026

View reviewed changes

ci: setup integration config via login/link commands

856135b

jwfing reviewed Mar 19, 2026

View reviewed changes

test(ci): remove unnecessary INSFORGE_API_URL override in real integr…

209dcf0

…ation docs/workflow

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(ci): add real-project integration tests for CLI read-only flows#28

test(ci): add real-project integration tests for CLI read-only flows#28
babuClawd wants to merge 6 commits intoInsForge:mainfrom
babuClawd:test/issue-19-real-integration-phase1

babuClawd commented Mar 19, 2026 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Mar 19, 2026 •

edited

Loading

❌ Failed checks (1 warning)

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

jwfing Mar 19, 2026

Uh oh!

tonychang04 commented Mar 19, 2026

Uh oh!

babuClawd commented Mar 19, 2026

Uh oh!

jwfing Mar 19, 2026

Uh oh!

jwfing Mar 19, 2026

Uh oh!

babuClawd commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

babuClawd commented Mar 19, 2026 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What’s included

Phase-1 scope covered

Local verification

Notes

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Suggested reviewers

Poem

❌ Failed checks (1 warning)

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

jwfing Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

tonychang04 commented Mar 19, 2026

Uh oh!

babuClawd commented Mar 19, 2026

Uh oh!

jwfing Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

jwfing Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

babuClawd commented Mar 20, 2026

Changes made

About login --email in CI

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

babuClawd commented Mar 19, 2026 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Mar 19, 2026 •

edited

Loading

About `login --email` in CI