Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,3 +78,69 @@ jobs:
bash scripts/install.sh
"${INSTALL_DIR}/profitctl" --help >/dev/null
"${INSTALL_DIR}/profitctl" validate -f examples/mix_profit.yml >/dev/null

security-scan:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- name: Check out code
uses: actions/checkout@v5

- name: Run scanners
run: |
set -euo pipefail
mkdir -p reports

docker run --rm --entrypoint semgrep -v "$PWD:/src" -w /src semgrep/semgrep:1.154.0 \
scan --config p/default --sarif --sarif-output reports/semgrep.sarif . \
|| echo "semgrep exited non-zero; preserving report if present"
[ -f reports/semgrep.sarif ] || echo '{"version":"2.1.0","runs":[]}' > reports/semgrep.sarif

docker run --rm --entrypoint trivy -v "$PWD:/src" -w /src aquasec/trivy:0.69.3 \
fs --scanners vuln,misconfig,secret --format sarif -o reports/trivy.sarif . \
|| echo "trivy exited non-zero; preserving report if present"
[ -f reports/trivy.sarif ] || echo '{"version":"2.1.0","runs":[]}' > reports/trivy.sarif

docker run --rm --entrypoint gitleaks -v "$PWD:/src" -w /src ghcr.io/gitleaks/gitleaks:v8.30.0 \
git --report-format sarif --report-path reports/gitleaks.sarif . \
|| echo "gitleaks exited non-zero; preserving report if present"
[ -f reports/gitleaks.sarif ] || echo '{"version":"2.1.0","runs":[]}' > reports/gitleaks.sarif

docker run --rm --entrypoint osv-scanner -v "$PWD:/src" -w /src ghcr.io/google/osv-scanner:v2.3.3 \
scan --recursive --format json --output reports/osv.json . \
|| echo "osv-scanner exited non-zero; preserving report if present"
[ -f reports/osv.json ] || echo '{"results":[]}' > reports/osv.json

- name: Upload scan findings
env:
APPSEC_API_URL: ${{ secrets.APPSEC_API_URL }}
APPSEC_API_TOKEN: ${{ secrets.APPSEC_API_TOKEN }}
APPSEC_STRICT_API: ${{ vars.APPSEC_STRICT_API || 'false' }}
CI_PIPELINE_ID: ${{ github.run_id }}-${{ github.run_attempt }}
PR_NUMBER: ${{ github.event.pull_request.number || '' }}
run: |
set -euo pipefail
upload() {
tool="$1"
format="$2"
report="$3"
if [ -n "${PR_NUMBER}" ]; then
sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$GITHUB_REPOSITORY" "$GITHUB_SHA" "$tool" "$format" "$PR_NUMBER" "$report"
else
sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$GITHUB_REPOSITORY" "$GITHUB_SHA" "$tool" "$format" "$report"
fi
}

upload semgrep sarif reports/semgrep.sarif
upload trivy sarif reports/trivy.sarif
upload gitleaks sarif reports/gitleaks.sarif
upload osv json reports/osv.json

- name: Evaluate policy gate
if: github.event_name == 'pull_request'
env:
APPSEC_API_URL: ${{ secrets.APPSEC_API_URL }}
APPSEC_API_TOKEN: ${{ secrets.APPSEC_API_TOKEN }}
APPSEC_STRICT_API: ${{ vars.APPSEC_STRICT_API || 'false' }}
run: |
sh scripts/ci/evaluate_gate.sh "$APPSEC_API_URL" "$GITHUB_REPOSITORY" "${{ github.event.pull_request.number }}"
93 changes: 0 additions & 93 deletions .woodpecker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,96 +58,3 @@ steps:
bash scripts/release/install-tool.sh cosign "$${TOOL_DIR}"
export PATH="$${TOOL_DIR}:$${PATH}"
doppler run --project profitctl --config prd_ci_woodpecker -- bash scripts/release/smoke-published-release.sh "$${TAG}"

---
labels:
pool: shared-kvm

when:
- event: pull_request
branch: [main]

steps:
- name: verify-go-quality
image: golang:1.24
commands:
- go version
- test -z "$(gofmt -l .)"
- go mod tidy
- git diff --exit-code go.mod go.sum
- go vet ./...
- go test ./...

- name: semgrep
image: semgrep/semgrep:1.154.0
commands:
- mkdir -p reports
- semgrep scan --config p/default --sarif --sarif-output reports/semgrep.sarif . || true
- '[ -f reports/semgrep.sarif ] || echo ''{"version":"2.1.0","runs":[]}'' > reports/semgrep.sarif'

- name: trivy-fs
image: aquasec/trivy:0.69.3
commands:
- mkdir -p reports
- trivy fs --scanners vuln,misconfig,secret --format sarif -o reports/trivy.sarif . || echo '{"version":"2.1.0","runs":[]}' > reports/trivy.sarif

- name: gitleaks
image: ghcr.io/gitleaks/gitleaks:v8.30.0
commands:
- mkdir -p reports
- gitleaks git --report-format sarif --report-path reports/gitleaks.sarif . || true
- '[ -f reports/gitleaks.sarif ] || echo ''{"version":"2.1.0","runs":[]}'' > reports/gitleaks.sarif'

- name: osv
image: ghcr.io/google/osv-scanner:v2.3.3
commands:
- mkdir -p reports
- osv-scanner scan --recursive --format json --output reports/osv.json . || true
- '[ -f reports/osv.json ] || echo ''{"results":[]}'' > reports/osv.json'

- name: upload-findings
image: curlimages/curl:8.11.0
environment:
APPSEC_API_URL:
from_secret: APPSEC_API_URL
APPSEC_API_TOKEN:
from_secret: APPSEC_API_TOKEN
APPSEC_STRICT_API:
from_secret: APPSEC_STRICT_API
commands:
- sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$CI_REPO" "$CI_COMMIT_SHA" semgrep sarif "$CI_COMMIT_PULL_REQUEST" reports/semgrep.sarif
- sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$CI_REPO" "$CI_COMMIT_SHA" trivy sarif "$CI_COMMIT_PULL_REQUEST" reports/trivy.sarif
- sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$CI_REPO" "$CI_COMMIT_SHA" gitleaks sarif "$CI_COMMIT_PULL_REQUEST" reports/gitleaks.sarif
- sh scripts/ci/upload_scan.sh "$APPSEC_API_URL" "$CI_REPO" "$CI_COMMIT_SHA" osv json "$CI_COMMIT_PULL_REQUEST" reports/osv.json

- name: evaluate-gate
image: alpine:3.21
environment:
APPSEC_API_URL:
from_secret: APPSEC_API_URL
APPSEC_API_TOKEN:
from_secret: APPSEC_API_TOKEN
APPSEC_STRICT_API:
from_secret: APPSEC_STRICT_API
commands:
- apk add --no-cache curl jq
- sh scripts/ci/evaluate_gate.sh "$APPSEC_API_URL" "$CI_REPO" "$CI_COMMIT_PULL_REQUEST"

---
labels:
pool: shared-kvm

when:
- event: push
branch: [main]

steps:
- name: verify-main
image: golang:1.24
commands:
- go version
- test -z "$(gofmt -l .)"
- go mod tidy
- git diff --exit-code go.mod go.sum
- go vet ./...
- go test ./...
1 change: 0 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

## Tooling Policy

- Greptile is deprecated for this repository.
- Use Codex plus native GitHub checks and review comments for local review loops.
- Do not use paid review bots or usage-based review add-ons unless the user explicitly approves the spend.
- Use native GitHub checks and review comments with `$check-pr` for PR readiness.
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,18 +148,22 @@ See [SECURITY.md](SECURITY.md).

`ProfitCtl` can publish PR security findings into the separate `appsec-mvp` service.

- PR scans run in Woodpecker via `semgrep`, `trivy`, `gitleaks`, and `osv-scanner`
- PR and main scans run in GitHub Actions via `semgrep`, `trivy`, `gitleaks`, and `osv-scanner`
- findings are uploaded to `appsec-mvp`
- GitHub PRs receive:
- commit status `appsec/mvp`
- check run `AppSec MVP Review`
- selective inline comments for high-confidence findings

Required Woodpecker secret:
Required GitHub repository secrets:

- `APPSEC_API_URL`
- `APPSEC_API_TOKEN`

Optional GitHub repository variable:

- `APPSEC_STRICT_API`

Required `appsec-mvp` runtime config:

- `GITHUB_APP_ID`
Expand Down
2 changes: 2 additions & 0 deletions docs/DOCS_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
- [Quick Start](QUICK_START.md)
- [How It Works](HOW_IT_WORKS.md)
- [Architecture](ARCHITECTURE.md)
- [Cost Intelligence System Design](cost-intelligence-system-design.md)
- [Cost Model Standards](cost-model-standards.md)
- [Open-Core Packaging](OPEN_CORE_PACKAGING.md)
- [Open-Core Roadmap](OPEN_CORE_ROADMAP.md)
- [Full Economics Cost Layers](full-economics-cost-layers.md)
Expand Down
75 changes: 75 additions & 0 deletions docs/cost-intelligence-system-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Cost Intelligence System Design

## Objective

ProfitCtl should be the deterministic economics engine for cost-aware development decisions. Agents may gather context and propose scenarios, but ProfitCtl owns the schema, provenance contract, validation rules, and repeatable calculations.

## Ownership Split

| Owner | Responsibility |
| --- | --- |
| Product / ProfitCtl | Cost schema, formulas, scenario validation, provenance labels, provider catalog format, deterministic output, standards judge |
| User | Business assumptions: users, paid seats, pricing, target margin, risk tolerance, expected usage |
| Agent | Repo inspection, provider/service detection, template selection, temp scenario generation, cost-aware explanation |

Agents must not blur facts and assumptions. Every cost or usage line should identify whether it came from a template, user input, repo detection, telemetry, invoice, or provider catalog.

## Data Flow

1. Agent inspects repository context: framework, hosting, database, auth, paid APIs, billing, background work, and telemetry.
2. ProfitCtl or an agent selects the closest scenario template.
3. User supplies or confirms business assumptions that materially change the answer.
4. Provider catalog entries fill planning defaults when no actuals exist.
5. Telemetry and invoices override defaults when connected.
6. ProfitCtl validates and simulates the scenario.
7. The agent returns a recommendation with assumptions, margins, p95 stress, cost per user, covenant status, and confidence.
8. The final scenario and output become a decision artifact that can later be compared against real actuals.

## Internet Access

V1 does not require internet access. Scenario templates and user inputs are enough for local economics checks.

Productized catalog refresh should use controlled sources:

- official provider pricing pages or APIs
- authenticated billing exports
- product telemetry
- runtime cost ledgers

Do not make random web scraping part of the trust path. Scraped values are brittle, hard to cite, and hard to keep current. If web lookup is used, cache the result with source URL, capture date, and confidence.

## System Changes

### ProfitCtl

- Add cost-source provenance fields to fixed and variable costs.
- Add a provider-catalog seed format for planning defaults.
- Add standards judge tooling that validates provenance and scenario quality.
- Keep core simulation math deterministic and local.

Cost source schema lives at `schemas/cost-source.schema.json`. Seed catalog values live under `provider_catalog/` and must be treated as planning defaults until replaced by sourced provider data, telemetry, or invoices.

### AgentOS / Condere

- Continue using the run ledger as the source for agent/model/search costs.
- Normalize runtime cost observations into project-scoped economics signals when integrating later.
- Treat provider-specific pricing estimates as calibration inputs, not hidden model behavior.

### Web App

- Capture project-scoped business assumptions: users, billable seats, runs per user, tokens per run, search calls per run, pricing plan, and target margin.
- Expose those assumptions to agents and ProfitCtl as scenario inputs.
- Avoid dashboard work until scenario quality and decision usefulness are proven.

## Confidence Model

| Source type | Typical confidence | Meaning |
| --- | --- | --- |
| `template` | low to medium | Default planning assumption, useful for first-pass comparison |
| `repo_detected` | low to medium | Inferred from code/config; needs user or telemetry confirmation |
| `user_supplied` | medium to high | Business assumption from operator or customer |
| `provider_catalog` | medium | Provider default with source and capture date |
| `telemetry` | high | Measured usage from runtime/product systems |
| `invoice` | high | Actual billed cost or revenue export |

High confidence should be reserved for actuals or explicit user/business inputs, not generic templates.
56 changes: 56 additions & 0 deletions docs/cost-model-standards.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Cost Model Standards

## Purpose

These standards are the judge criteria for cost-aware agent work. A scenario can be useful while still being approximate, but the output must make assumptions, provenance, and confidence visible.

## Required Scenario Standards

- Every fixed and variable cost should include `source.type` and `source.confidence`.
- `source.type` must be one of `template`, `user_supplied`, `repo_detected`, `telemetry`, `invoice`, or `provider_catalog`.
- `source.confidence` must be one of `low`, `medium`, or `high`.
- Template assumptions must not use `high` confidence.
- `provider_catalog` assumptions should include `url` or `note`.
- `telemetry` and `invoice` assumptions should include `captured_at` or `note`.
- Productization and adoption costs should be modeled separately from delivery costs using `economics_layer`.
- Non-delivery costs should include an `allocation` rule when the allocation is not obvious.
- Scenarios should define covenants for margin and cost per user.
- Agent recommendations should report assumptions, margin, p95 margin, cost per user, and covenant status.

## Accuracy Judge

Run the local judge from the repository root:

```bash
go run scripts/judge_cost_standards.go
go run scripts/judge_cost_standards.go path/to/scenario.yml
go run scripts/judge_cost_standards.go path/to/scenario-directory
```

The standards judge checks scenario quality, not provider-price truth. It should fail when:

- the scenario does not parse or validate
- cost line provenance is missing
- source type or confidence is invalid
- a template claims high confidence
- non-delivery cost lacks allocation
- margin or cost-per-user covenant is missing

The judge should warn when:

- all costs are template-derived
- no actual telemetry or invoice inputs are present
- provider catalog entries lack source URLs

## Calibration Path

Accuracy improves as sources move from estimates to actuals:

1. `template`: first-pass planning
2. `repo_detected`: architecture-aware planning
3. `user_supplied`: business-context planning
4. `provider_catalog`: sourced planning defaults
5. `telemetry`: measured usage
6. `invoice`: actual spend/revenue

The product goal is not perfect prediction. It is disciplined decision quality: explain assumptions, test stress cases, and update scenarios when actuals arrive.
17 changes: 11 additions & 6 deletions docs/deployment/WOODPECKER_HOSTINGER_SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,14 @@ This runbook onboards `IntelIP/ProfitCtl` into your existing Woodpecker infrastr

`/.woodpecker.yml` defines:

- PR workflow (`pool=shared-kvm`): checks for `main` plus AppSec scan upload/gate evaluation
- Main workflow (`pool=shared-kvm`): push checks for `main`
- Tag release workflow (`pool=shared-kvm`): semver tag release + VPS publish

PR and main verification now run in GitHub Actions:

- `verify-go`
- `verify-install-smoke`
- `security-scan` with AppSec upload and PR gate evaluation

## Required Secrets (Doppler: `profitctl` / `prd_ci_woodpecker`)

- `GITHUB_TOKEN_RELEASE`
Expand All @@ -28,13 +32,14 @@ This runbook onboards `IntelIP/ProfitCtl` into your existing Woodpecker infrastr
- Events: include `tag`
- Image filters: leave empty (`[]`) so command steps can consume it
- Repo trust: set `IntelIP/ProfitCtl` as trusted in Woodpecker so `from_secret` works in command steps

## Required Secrets (GitHub Actions)

- Repo secret name: `APPSEC_API_TOKEN`
- Secret value: bearer token used by `appsec-mvp` ingestion and summary endpoints
- Events: include `pull_request`
- Repo secret name: `APPSEC_API_URL`
- Secret value: reachable base URL for the deployed `appsec-mvp` API, for example `http://172.17.0.1:18080`
- Events: include `pull_request`
- Optional repo secret name: `APPSEC_STRICT_API`
- Secret value: reachable base URL for the deployed `appsec-mvp` API
- Optional repo variable name: `APPSEC_STRICT_API`
- Rollout default: leave unset or set to `false` so AppSec transport failures do not block PRs while `/.appsec.yml` is still `report_only`
- Post-rollout: set to `true` when the service is stable and you want CI to fail closed on AppSec API errors

Expand Down
Loading
Loading