Skip to content

feat: add public benchmark language fixtures#483

Merged
mohanagy merged 1 commit into
nextfrom
issue-469-benchmark-suite
Jun 2, 2026
Merged

feat: add public benchmark language fixtures#483
mohanagy merged 1 commit into
nextfrom
issue-469-benchmark-suite

Conversation

@mohanagy
Copy link
Copy Markdown
Owner

@mohanagy mohanagy commented Jun 2, 2026

Summary

  • add concrete Python and Go service fixtures under the public benchmark suite
  • promote both suite repos to ready and wire prompts for explain, implement, review, and impact tasks
  • tighten suite docs and tests around the five-repo matrix while keeping the install-gated readiness contract explicit

Test Plan

  • npm run test:run -- tests/unit/benchmark-suite.test.ts tests/unit/benchmark-suite-docs.test.ts
  • npm run typecheck
  • npm run build
  • CI=1 npm run test:run

Refs #469

Summary by CodeRabbit

Release Notes

  • New Features

    • Added Go service benchmark fixture with login session HTTP server capability
    • Added Python service benchmark fixture with FastAPI-based login session service
  • Documentation

    • Updated benchmark suite README to reflect new Go and Python service fixtures as ready benchmarks
    • Added documentation for both Go and Python service fixtures describing their architecture and flow
  • Tests

    • Added test coverage for Go service session creation and audit recording
    • Added test coverage for Python service session creation and audit recording
  • Chores

    • Updated benchmark suite configuration to include prompts for new Python and Go service fixtures across multiple task categories

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 2, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds two new service fixtures (Python FastAPI and Go net/http) to the benchmark suite that implement matching login-session endpoints with audit logging, updates the suite configuration to mark them as ready, and extends test coverage to validate their wiring and presence.

Changes

Benchmark Suite Fixtures & Configuration

Layer / File(s) Summary
Python FastAPI Service Fixture
docs/benchmarks/suite/fixtures/python-service/README.md, docs/benchmarks/suite/fixtures/python-service/app/audit.py, docs/benchmarks/suite/fixtures/python-service/app/repositories/session_repository.py, docs/benchmarks/suite/fixtures/python-service/app/services/session_service.py, docs/benchmarks/suite/fixtures/python-service/app/routes/session_routes.py, docs/benchmarks/suite/fixtures/python-service/app/main.py, docs/benchmarks/suite/fixtures/python-service/requirements.txt, docs/benchmarks/suite/fixtures/python-service/tests/test_session_service.py
New FastAPI service with AuditLog, in-memory SessionRepository, SessionService with dependency injection, and POST /login/session endpoint returning session_id and audit_event. Unit test verifies deterministic session ID format and single audit event recording.
Go net/http Service Fixture
docs/benchmarks/suite/fixtures/go-service/README.md, docs/benchmarks/suite/fixtures/go-service/internal/audit/login_audit.go, docs/benchmarks/suite/fixtures/go-service/internal/repository/session_repository.go, docs/benchmarks/suite/fixtures/go-service/internal/service/session_service.go, docs/benchmarks/suite/fixtures/go-service/internal/httpapi/session_handler.go, docs/benchmarks/suite/fixtures/go-service/cmd/api/main.go, docs/benchmarks/suite/fixtures/go-service/go.mod, docs/benchmarks/suite/fixtures/go-service/internal/service/session_service_test.go
New Go HTTP service with LoginAudit, in-memory SessionRepository, SessionService orchestrating repository and audit, and SessionHandler wiring POST /login/session endpoint. HTTP server listens on :8080. Unit test verifies session ID composition and audit event recording.
Suite Configuration & Documentation
docs/benchmarks/suite/repos.json, docs/benchmarks/suite/tasks.json, docs/benchmarks/suite/README.md
Mark python-service and go-service as ready (changed from planned). Add repo-specific prompts for both fixtures across four task categories (explain-runtime, implement, review, impact). Update README to confirm fixtures are concrete public workspaces with prompt-wired task rows.
Test Coverage
tests/unit/benchmark-suite.test.ts, tests/unit/benchmark-suite-docs.test.ts
Extend manifest assertions to verify python-service and go-service load as ready, that their prompts are populated for all ready tasks, and add a test iterating all repos and ready tasks to verify paths exist on disk and every repo has a non-empty prompt for every ready task.

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Route as /login/session POST
  participant SessionService
  participant SessionRepository
  participant AuditLog
  Client->>Route: user_id, device_id
  Route->>SessionService: create_login_session
  SessionService->>SessionRepository: create_login_session
  SessionRepository-->>SessionService: SessionRecord (user_id:device_id)
  SessionService->>AuditLog: record_login_session
  AuditLog-->>SessionService: recorded
  SessionService-->>Route: SessionRecord
  Route-->>Client: session_id, audit_event
Loading
sequenceDiagram
  participant Client
  participant Handler as SessionHandler
  participant SessionService
  participant SessionRepository
  participant LoginAudit
  Client->>Handler: POST /login/session (user_id, device_id)
  Handler->>SessionService: CreateLoginSession
  SessionService->>SessionRepository: CreateLoginSession
  SessionRepository-->>SessionService: Session
  SessionService->>LoginAudit: RecordLoginSession
  LoginAudit-->>SessionService: recorded
  SessionService-->>Handler: Session
  Handler-->>Client: JSON (session_id, audit_event)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • mohanagy/madar#334: Both PRs update the benchmark-suite documentation surface and configuration; this PR adds the Python and Go service fixtures and their prompts, while the related PR revises the suite's public framing and evidence statements in the same benchmark-suite metadata files.

Poem

🐰 Two fixtures bloom in benchmark light,
Python swift, Go's HTTP flight,
Sessions log with audit care,
Shared design beyond compare,
Suite expands its testing dare! 🚀

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title 'feat: add public benchmark language fixtures' accurately describes the main change: adding Python and Go service fixtures to the public benchmark suite.
Description check ✅ Passed The PR description includes a clear summary of changes, a detailed test plan covering unit and build tests, and references the related issue #469, meeting the template requirements.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch issue-469-benchmark-suite

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 golangci-lint (2.12.2)

level=error msg="[linters_context] typechecking error: pattern ./...: directory prefix . does not contain main module or its selected dependencies"


Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/benchmarks/suite/fixtures/go-service/internal/repository/session_repository.go`:
- Around line 15-21: The CreateLoginSession method currently builds session IDs
from userID and deviceID without validation; change its signature from
CreateLoginSession(userID string, deviceID string) Session to
CreateLoginSession(userID string, deviceID string) (Session, error), validate
that neither userID nor deviceID is empty (check in
SessionRepository.CreateLoginSession), return a non-nil error when either is
empty, and only construct Session{ID: userID + ":" + deviceID, UserID: userID,
DeviceID: deviceID} when both are present; update all callers to handle the
(Session, error) return so malformed IDs like ":" cannot be produced.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: c12cdd51-8e73-460a-867a-82a1a418bf5b

📥 Commits

Reviewing files that changed from the base of the PR and between 103b7eb and 7b11521.

📒 Files selected for processing (25)
  • docs/benchmarks/suite/README.md
  • docs/benchmarks/suite/fixtures/go-service/README.md
  • docs/benchmarks/suite/fixtures/go-service/cmd/api/main.go
  • docs/benchmarks/suite/fixtures/go-service/go.mod
  • docs/benchmarks/suite/fixtures/go-service/internal/audit/login_audit.go
  • docs/benchmarks/suite/fixtures/go-service/internal/httpapi/session_handler.go
  • docs/benchmarks/suite/fixtures/go-service/internal/repository/session_repository.go
  • docs/benchmarks/suite/fixtures/go-service/internal/service/session_service.go
  • docs/benchmarks/suite/fixtures/go-service/internal/service/session_service_test.go
  • docs/benchmarks/suite/fixtures/python-service/README.md
  • docs/benchmarks/suite/fixtures/python-service/app/__init__.py
  • docs/benchmarks/suite/fixtures/python-service/app/audit.py
  • docs/benchmarks/suite/fixtures/python-service/app/main.py
  • docs/benchmarks/suite/fixtures/python-service/app/repositories/__init__.py
  • docs/benchmarks/suite/fixtures/python-service/app/repositories/session_repository.py
  • docs/benchmarks/suite/fixtures/python-service/app/routes/__init__.py
  • docs/benchmarks/suite/fixtures/python-service/app/routes/session_routes.py
  • docs/benchmarks/suite/fixtures/python-service/app/services/__init__.py
  • docs/benchmarks/suite/fixtures/python-service/app/services/session_service.py
  • docs/benchmarks/suite/fixtures/python-service/requirements.txt
  • docs/benchmarks/suite/fixtures/python-service/tests/test_session_service.py
  • docs/benchmarks/suite/repos.json
  • docs/benchmarks/suite/tasks.json
  • tests/unit/benchmark-suite-docs.test.ts
  • tests/unit/benchmark-suite.test.ts

Comment on lines +15 to +21
func (r *SessionRepository) CreateLoginSession(userID string, deviceID string) Session {
return Session{
ID: userID + ":" + deviceID,
UserID: userID,
DeviceID: deviceID,
}
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add input validation for empty strings.

The method concatenates userID and deviceID without validation. Empty strings would produce malformed session IDs like ":" or ":device", which could confuse benchmark tests or results.

🛡️ Suggested validation
 func (r *SessionRepository) CreateLoginSession(userID string, deviceID string) Session {
+	if userID == "" || deviceID == "" {
+		panic("userID and deviceID must not be empty")
+	}
 	return Session{
 		ID:       userID + ":" + deviceID,
 		UserID:   userID,
 		DeviceID: deviceID,
 	}
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func (r *SessionRepository) CreateLoginSession(userID string, deviceID string) Session {
return Session{
ID: userID + ":" + deviceID,
UserID: userID,
DeviceID: deviceID,
}
}
func (r *SessionRepository) CreateLoginSession(userID string, deviceID string) Session {
if userID == "" || deviceID == "" {
panic("userID and deviceID must not be empty")
}
return Session{
ID: userID + ":" + deviceID,
UserID: userID,
DeviceID: deviceID,
}
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/benchmarks/suite/fixtures/go-service/internal/repository/session_repository.go`
around lines 15 - 21, The CreateLoginSession method currently builds session IDs
from userID and deviceID without validation; change its signature from
CreateLoginSession(userID string, deviceID string) Session to
CreateLoginSession(userID string, deviceID string) (Session, error), validate
that neither userID nor deviceID is empty (check in
SessionRepository.CreateLoginSession), return a non-nil error when either is
empty, and only construct Session{ID: userID + ":" + deviceID, UserID: userID,
DeviceID: deviceID} when both are present; update all callers to handle the
(Session, error) return so malformed IDs like ":" cannot be produced.

@mohanagy mohanagy merged commit 9990781 into next Jun 2, 2026
7 checks passed
@mohanagy mohanagy deleted the issue-469-benchmark-suite branch June 2, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant