Skip to content

feat(b2b): org analytics dashboard for the manager surface - #3679

Open
blarghmatey wants to merge 2 commits into
mainfrom
feat/b2b-org-analytics-dashboard
Open

feat(b2b): org analytics dashboard for the manager surface#3679
blarghmatey wants to merge 2 commits into
mainfrom
feat/b2b-org-analytics-dashboard

Conversation

@blarghmatey

Copy link
Copy Markdown
Member

What are the relevant tickets?

N/A — no GitHub issue. This is the frontend piece of the B2B self-serve analytics work.

Related PRs, and the required deploy order:

  1. mitodl/mitxonline#3789 — exposes sso_organization_id on OrganizationPageSerializer and adds the ?sso_organization_id= filter to the manager org endpoint. Must deploy first.
  2. mitodl/ol-analytics-api#13 — keys the org endpoints on the Keycloak org UUID. Deploying this before #3789 fails open: an older manager endpoint ignores the unknown filter param and returns all of the caller's managed orgs.
  3. This PR — safe to merge at any point; the dashboard is behind a feature flag and reports itself unavailable until the API exists.

Identifier analysis: mitodl/hq#11845.

Description (What does it do?)

Adds the reporting half of the B2B org-manager dashboard, alongside the seat-administration half that already exists in ContractAdminPage. Four sections at /dashboard/organization/[orgSlug]/analytics, each backed by one of ol-analytics-api's org-scoped materialized views:

Section Endpoint Backing view
Contract utilization (KPI cards) /contract-utilization mv_b2b_contract_utilization
Monthly engagement (line chart) /engagement-trend mv_b2b_monthly_engagement_trend
Course performance (table) /enrollment-funnel mv_b2b_enrollment_completion_funnel
Program funnel (bars + table) /program-funnel mv_b2b_program_funnel

A query factory for the fifth endpoint (/content-engagement) is included; its panel is deliberately out of scope for this PR.

Reachable from a "View analytics" link added to the contract admin page header, behind the same flag.

Charting: @mui/x-charts, not recharts

The original scope called for recharts. After evaluating both against this codebase I went with @mui/x-charts@^8:

  • Theme integration. The app is MUI v6 + emotion throughout, with design tokens in theme.custom.colors via ol-components. x-charts reads that theme directly, so axis, tooltip and legend styling comes from the same tokens as everything else. Recharts styles per-component through props, so every token would have to be threaded by hand.
  • Dependency weight. Recharts 3 ships @reduxjs/toolkit, react-redux, immer and es-toolkit as runtime dependencies — a second state-management stack inside a chart library, in an app that uses TanStack Query. Roughly 18MB installed vs ~6.6MB for x-charts.
  • Compatibility. @mui/x-charts@8 peer-accepts @mui/material ^6.0.0 (we're on 6.4.5) and React 19. The charts needed here (Line, Bar) are MIT/community; only the dedicated Funnel widget is Pro, and this uses ordinary bars.

Trade-off worth knowing: x-charts pulls @mui/utils@^7 as a direct dependency alongside MUI v6, so that small utils package is duplicated in the tree.

Organizations are keyed on the Keycloak UUID, not the slug

Per ol-analytics-api#13, every org endpoint filters on sso_organization_id — the only identifier stable across the JWT, MITx Online and StarRocks. The route keeps [orgSlug] (human-facing, consistent with its sibling routes) and AnalyticsContent resolves slug → manager org → UUID.

Because mitxonline#3789 has not been cut into a release of @mitodl/mitxonline-api-axios (we're pinned to 2026.7.7), the generated OrganizationPage type does not declare the field yet. Analytics/orgUuid.ts reads it off the wire and returns null when absent — so pointed at an older MITx Online, the page shows "Analytics is not available for this organization yet" rather than issuing a request with undefined in the path. That shim is marked for deletion once the client is regenerated.

Suppressed values are never rendered as zero

The API nulls out any learner count below its k-anonymity floor, plus every rate and average derived from it. Those nulls are carried through as nulls:

  • Table cells render a marked placeholder with the reason, exposed to assistive tech via aria-label, not hover alone.
  • A suppressed month is a gap in the trend line, not a dip to zero.
  • The program funnel is paired with a table of the same numbers — a bar chart can only omit a bar, which reads as zero.

Auth

Unchanged from the rest of the app: cookie-based, withCredentials + CSRF, no bearer token. The analytics API sits behind APISIX, which mints the JWT from the session cookie. Authorization is the API's job; this component only resolves which org is meant, via MITx Online's manager org list — the same source ContractAdminPage uses — so a non-manager never issues an analytics request.

Chart colors were validated, not chosen by eye

Hues come from the smoot-design token set, then checked against lightness band, chroma floor, CVD separation under simulated protanopia/deuteranopia, a normal-vision separation floor, and ≥3:1 contrast on the surface. Two slots are stepped versions of brand hues because the tokens themselves fall outside the readable band (orange #FAB005 is far too light to carry a 2px line; lightBlue is too pale to distinguish from the card). Green and red are deliberately excluded from the series palette so they stay reserved for status. Rationale is documented in Analytics/chartPalette.ts — changing a value there means re-running the check, not swapping a hex.

Funnel stages take a single-hue light→dark ramp rather than three unrelated hues, because stage order carries meaning. The trend chart deliberately omits the view's total_videos_watched / total_problems_attempted / total_chatbot_interactions columns: those are counts of events, orders of magnitude above the learner counts, and putting them on the same plot would need a second y-axis.

Incidental changes

  • components/B2BTable — the contract admin page's table primitives are extracted so both manager surfaces share them (including the sub-md reflow where rows become label/value stacks). ContractAdminPage's 122 tests still pass unchanged.
  • structuredClone polyfill in frontends/jest-shared-setup.ts — jsdom does not provide it and @mui/x-charts calls it, so without this any test rendering a chart dies with a ReferenceError. Implemented with the v8 serializer rather than a JSON round-trip so Dates/Maps/Sets survive.
  • NEXT_PUBLIC_ANALYTICS_API_BASE_URL is optional in the yup schema. Environments without an analytics deployment still boot; bootstrapApiClients() simply skips configuring that client and the page reports itself unavailable. This avoids breaking startup for every environment that doesn't have the service yet.

How can this be tested?

Automated — all green:

yarn workspace frontends run lint-check     # clean
yarn workspace api run global:typecheck     # clean
yarn workspace main run global:typecheck    # clean
cd frontends/main && NODE_ENV=test npx jest src/app-pages/DashboardPage src/app-pages/ContractAdminPage src/bootstrap
cd frontends/api  && NODE_ENV=test npx jest src/analytics

30 new tests covering: the five query factories (correct URL per endpoint, paging params, per-org key namespacing, org-id URL encoding); formatters; both charts rendering with the validated palette; and the page's access / availability / freshness / suppression behaviour.

Note on the full suite: AssignSeatsSection, CoursePage, NewsEditor and SearchPage fail intermittently under parallel load. All pass in isolation, and AssignSeatsSection and NewsEditor fail identically on an untouched main checkout — pre-existing flakes, not regressions from this branch.

Manual, once the dependencies are in place:

  1. Deploy feat(b2b): look up managed orgs by sso_organization_id; expose the org UUID mitxonline#3789, then fix(b2b): key org endpoints on the Keycloak org UUID (sso_organization_id) ol-analytics-api#13.
  2. Add an APISIX route proxying to the analytics API with the JWT plugin (mirroring the existing /ai/* learn-ai route), and set NEXT_PUBLIC_ANALYTICS_API_BASE_URL to that route — not to the API directly; the API expects the JWT APISIX mints from the session cookie.
  3. Create the b2b-analytics-dashboard PostHog flag and enable it for your user.
  4. As an org manager, visit /dashboard/organization/<org>/analytics.

Worth exercising specifically:

  • Not a manager of that org → "Access denied" (decided before any analytics request is issued).
  • An analytics 403 → propagates to the route error boundary. This is intentional and not handled in the component: makeBrowserQueryClient sets throwOnError for 400/401/403, so those never reach an isError branch. I wrote an in-component 403 branch first and removed it once a test proved it was unreachable.
  • Org without a Keycloak UUID (or NEXT_PUBLIC_ANALYTICS_API_BASE_URL unset) → "not available" notice, and no request is issued.
  • An org whose data is below the anonymity floor → placeholders reading "Withheld: too few learners…", never 0.
  • A view that has never refreshed (as_of: null) → "Data not yet refreshed" rather than an implied freshness.
  • Multiple contracts → one KPI card group per contract (see below).

Locally without the API, set nothing and confirm the page renders the unavailable notice rather than erroring.

Additional Context

KPI cards are per-contract, not rolled up org-wide — deliberately. mv_b2b_contract_utilization's grain is org × contract, and none of the three headline figures can be honestly aggregated here: active_learners counts distinct learners per contract, so summing double-counts anyone on two contracts; and seat_utilization_pct / completion_rate_pct are rates, where averaging across contracts of different sizes is the wrong number and recomputing from raw counts is impossible whenever a count has been suppressed. Most orgs have one contract, so this reads as a single KPI row.

A bug this caught: formatDate("2026-01-15") was rendering as Jan 14new Date() parses a date-only ISO string as UTC midnight, which toLocaleDateString then renders as the previous day for anyone west of Greenwich. A contract starting on the 15th must not display as starting on the 14th. Fixed for both contract dates and the YYYY-MM trend labels; regression tests cover both.

Follow-up work already identified (not in this PR):

  • Add OpenAPI export + TypeScript client generation to ol-analytics-api, replacing the hand-written types in frontends/api/src/analytics/types.ts. Worth flagging for whoever picks it up: the b2b_dashboard tenant is a mounted sub-app, so it owns its own /openapi.json and the root schema does not include it; and its endpoints are registered in a loop with a runtime-parametrized generic, so the generated schema needs checking that each row model is named distinctly rather than collapsed into one OrgAnalyticsResponse.
  • Remove the sso_organization_id shim once the regenerated mitxonline client lands.
  • Build the content-engagement panel (factory and types are already here).
  • Provision the APISIX route, the env var, and the PostHog flag.

Screenshots are omitted because no environment can render the dashboard until the analytics API is deployed. I'll attach desktop and mobile captures once it's up in CI/RC.

Adds the reporting half of the B2B org-manager dashboard, alongside the
existing seat-administration half (ContractAdminPage): contract KPI cards,
a monthly engagement trend, per-course-run performance, and a program funnel,
each backed by one of ol-analytics-api's org-scoped materialized views.

Keyed on the Keycloak org UUID (sso_organization_id), not the org slug or
MITx Online's numeric id — that is the only identifier stable across the JWT,
MITx Online and StarRocks, and the one the analytics API filters on (see
mitodl/ol-analytics-api#13, mitodl/mitxonline#3789). The published mitxonline
client does not declare the field yet, so it is read off the wire through a
shim that returns null when absent; the dashboard reports itself unavailable
rather than issuing a request with `undefined` in the path.

Charts use @mui/x-charts rather than recharts. x-charts reads the existing MUI
v6 theme directly, so axis, tooltip and legend styling comes from the same
tokens as the rest of the app instead of being threaded through props by hand;
it also avoids pulling Redux, immer and a second lodash-alike into the bundle,
which recharts 3 ships as runtime dependencies. Chart colors are MIT Learn
tokens checked against the data-viz palette rules (lightness band, chroma
floor, CVD separation, contrast) rather than picked by eye — two slots are
stepped versions of brand hues that fall outside the readable band.

Numbers the API withholds under its k-anonymity floor arrive as null. They are
rendered as an explained placeholder, never coerced to zero: a suppressed month
is a gap in the trend line, and the program funnel is paired with a table so a
withheld stage can say so instead of just missing a bar.

Auth stays cookie-based; the analytics API sits behind APISIX, which mints the
JWT from the session. NEXT_PUBLIC_ANALYTICS_API_BASE_URL is optional so
environments without the service still boot.

Also extracts the contract admin page's table primitives into
components/B2BTable so both surfaces share them, and polyfills structuredClone
in the jest setup — jsdom lacks it and x-charts requires it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SjK3vzqB6nvLHKUF7686Pj
Copilot AI review requested due to automatic review settings July 24, 2026 21:55
@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown

OpenAPI Changes

No changes detected

View full changelog

Unexpected changes? Ensure your branch is up-to-date with main (consider rebasing).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an org-scoped B2B analytics dashboard (reporting half of the org-manager surface) to the Next.js frontend, backed by new ol-analytics-api query factories and an optional analytics API client configuration so environments without the service can still boot.

Changes:

  • Introduces /dashboard/organization/[orgSlug]/analytics with four analytics sections, behind a new PostHog feature flag.
  • Adds a new analytics API client + React Query query factories/types/test-utils for ol-analytics-api, plus optional runtime configuration (NEXT_PUBLIC_ANALYTICS_API_BASE_URL).
  • Extracts shared B2B table primitives to components/B2BTable and links to the analytics dashboard from the contract admin page.

Reviewed changes

Copilot reviewed 36 out of 37 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
yarn.lock Adds @mui/x-charts and related transitive dependencies.
frontends/main/validateEnv.js Adds optional NEXT_PUBLIC_ANALYTICS_API_BASE_URL to env schema.
frontends/main/src/test-utils/setupJest.tsx Configures analytics API client + env var in main workspace Jest setup.
frontends/main/src/components/B2BTable/B2BTable.tsx New shared table primitives for B2B manager surfaces.
frontends/main/src/common/urls.ts Adds organizationAnalyticsView route helper.
frontends/main/src/common/feature_flags.ts Adds B2BAnalyticsDashboard feature flag.
frontends/main/src/bootstrap/api.ts Optionally configures analytics API client based on NEXT_PUBLIC_ANALYTICS_API_BASE_URL.
frontends/main/src/app/(site)/dashboard/organization/[orgSlug]/analytics/page.tsx New App Router page wiring for analytics route.
frontends/main/src/app-pages/DashboardPage/AnalyticsContent.tsx Main analytics dashboard content: org resolution, feature-flag gating, and section composition.
frontends/main/src/app-pages/DashboardPage/AnalyticsContent.test.tsx Page-level tests covering access, availability, freshness, and suppression behavior.
frontends/main/src/app-pages/DashboardPage/Analytics/SectionHeader.tsx New per-section header with as_of freshness handling.
frontends/main/src/app-pages/DashboardPage/Analytics/ProgramFunnelChart.tsx New program funnel chart + accompanying accessible table.
frontends/main/src/app-pages/DashboardPage/Analytics/orgUuid.ts Shim to read sso_organization_id off the wire until mitxonline client is regenerated.
frontends/main/src/app-pages/DashboardPage/Analytics/format.tsx New formatting utilities + suppression marker component (privacy floor).
frontends/main/src/app-pages/DashboardPage/Analytics/format.test.tsx Tests for date/month parsing and suppression rendering.
frontends/main/src/app-pages/DashboardPage/Analytics/EngagementTrendChart.tsx New engagement trend line chart using @mui/x-charts.
frontends/main/src/app-pages/DashboardPage/Analytics/CoursePerformanceTable.tsx New course performance table using shared B2B table primitives.
frontends/main/src/app-pages/DashboardPage/Analytics/ContractKpiCards.tsx New KPI card rendering for contract utilization.
frontends/main/src/app-pages/DashboardPage/Analytics/charts.test.tsx Smoke tests ensuring charts render and use the intended palette.
frontends/main/src/app-pages/DashboardPage/Analytics/chartPalette.ts Defines validated chart palette and non-data “ink” colors.
frontends/main/src/app-pages/ContractAdminPage/ContractAdminPage.tsx Reuses extracted table primitives and adds “View analytics” link behind the new flag.
frontends/main/package.json Adds @mui/x-charts dependency.
frontends/jest-shared-setup.ts Adds structuredClone polyfill needed by @mui/x-charts under jsdom.
frontends/api/src/test-utils/setupJest.ts Configures analytics API client in api workspace Jest setup.
frontends/api/src/runtime/index.ts Adds optional analytics client support and isAnalyticsConfigured().
frontends/api/src/analytics/types.ts New hand-written response types/envelopes for analytics endpoints.
frontends/api/src/analytics/test-utils/urls.ts New analytics URL builders for tests (absolute URLs).
frontends/api/src/analytics/test-utils/index.ts Exports analytics test factories/urls.
frontends/api/src/analytics/test-utils/factories.ts Factories for analytics response shapes used in tests.
frontends/api/src/analytics/hooks/organizations/queries.ts New React Query query factories/keys for org analytics endpoints.
frontends/api/src/analytics/hooks/organizations/queries.test.ts Tests for URL correctness, key namespacing, paging, and encoding.
frontends/api/src/analytics/hooks/organizations/index.ts Public exports for analytics org hooks and types.
frontends/api/src/analytics/clients.ts New axios client wrapper for org-scoped analytics endpoints.
frontends/api/src/analytics/axios.ts Creates configurable axios instance for analytics API.
frontends/api/package.json Exposes analytics hooks/types/test-utils via package exports.
env/frontend.local.example.env Documents NEXT_PUBLIC_ANALYTICS_API_BASE_URL for local dev.
env/frontend.env Adds NEXT_PUBLIC_ANALYTICS_API_BASE_URL wiring for local env template.

Comment thread env/frontend.local.example.env Outdated
…ates

Both env comments implied the variable controls whether the analytics
dashboard is visible. It does not: reachability is gated by the
b2b-analytics-dashboard PostHog flag, and with the flag on and this unset
the route still renders, reporting itself unavailable. The variable only
decides whether there is an API to load data from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SjK3vzqB6nvLHKUF7686Pj
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants