fix(evals): centralize .env creation guidance for all agent runs

[frederikprijck]

What

Moves the .env no-permission-prompt guidance out of the eight individual quickstart PROMPT.md files and into the central AGENT_GUIDANCE string, so it reaches every agent run automatically.

Why

The line "Do not prompt for permissions to create any .env files" was intended only to stop the agent pausing for a permission (ask_user) prompt. It was duplicated across eight quickstart prompts (fastapi, flask, nuxt, angular, express, fastify-api, nextjs, vue), and any future eval needing a .env would have to copy it again.

writeAgentGuidance() already injects AGENT_GUIDANCE into whichever context file the active runner reads (CLAUDE.md / GEMINI.md / AGENTS.md / .github/copilot-instructions.md) on every agent job — the same mechanism the existing "no documentation files" steering uses. Putting the .env intent there makes it DRY and applies it to all runners and all current/future evals.

Change

• Add to AGENT_GUIDANCE (packages/evals-core/src/workspace/workspace.ts):

  If the task requires environment variables (e.g. .env or .env.local files), create those files directly — do not pause to ask for permission to create them.

• Remove the now-redundant per-prompt lines from the eight quickstart PROMPT.md files.

• Add a test asserting the guidance carries the .env no-permission steering.

The credential-in-source concern (agents hardcoding secrets instead of using .env) is intentionally not covered here — that belongs to the skill, and is enforced independently by the L3 notContainsInSource grader.

Notes

No frontmatter schema change, so no docs/ADDING_EVALS.md update required.

[coderabbitai]

📝 Walkthrough

Walkthrough

Updated FastAPI, Flask, and Nuxt quickstart prompts to store and load Auth0 configuration from environment files and create those files without confirmation.

Changes

Quickstart environment configuration

Layer / File(s)	Summary
Environment-file configuration guidance apps/auth0-evals/src/evals/quickstarts/{fastapi,flask,nuxt}/PROMPT.md	Quickstart instructions now specify storing Auth0 credentials in .env or .env.local files, loading them from environment variables, and creating the files directly without confirmation.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: sanchitmehtagit

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly reflects the main change: consolidating .env creation guidance across the eval quickstart prompts.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/quickstart-env-prompt-wording

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/auth0-evals/src/evals/quickstarts/nuxt/PROMPT.md`:
- Line 20: Update the credential-storage instruction in the prompt to require
creating and reading `.env` as the deterministic primary file, without asking
for confirmation; mention `.env.local` only as an explicitly supported fallback
and do not present it as an equivalent default.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 562376fa-d5d4-4689-9ef3-7e28abfa2bc6

📥 Commits

Reviewing files that changed from the base of the PR and between 3e91dfe and f6886a5.

📒 Files selected for processing (3)

• apps/auth0-evals/src/evals/quickstarts/fastapi/PROMPT.md
• apps/auth0-evals/src/evals/quickstarts/flask/PROMPT.md
• apps/auth0-evals/src/evals/quickstarts/nuxt/PROMPT.md