[Portal] Add Create Application baseline metrics by fungc-io · Pull Request #5806 · authgear/authgear-server

fungc-io · 2026-07-09T11:34:03Z

Summary

Baseline usage instrumentation for the Create Application flow, so we can measure funnel drop-off, activation, and time-to-integration. This is PR-1 of 2: it instruments the current (legacy) Create Application screen and stands up the north-star pipeline, so a clean "before" baseline accrues before the framework-first wizard ships (PR-2 will emit the same events tagged wizard_version: "framework_first").

No auth-server / token-path changes — the north-star is derived from existing audit data.

What it adds

Frontend funnel events on the legacy Create Application screen via the existing useCapture() GTM hook, all tagged wizard_version: "legacy":
- createApplication.viewed (once per mount), createApplication.selected-type (application_type), createApplication.created (client_id, application_type).
application.first_auth forwarder — derives first successful auth per client from existing _audit_log rows (user.authenticated / m2m.token.created, which already carry client_id) and forwards them to PostHog:
- AuditDBReadStore.GetFirstAuthTimeByClientID (tenant-scoped, parameterized).
- PosthogIntegration.ForwardFirstAuthEvents — idempotent via a deterministic UUIDv5 per (app_id, client_id); event timestamp = the real audit auth time (so batch lag never inflates time-to-integration).
- New batch subcommand portal analytic posthog first-auth, mirroring the existing posthog group/user jobs.
Report doc docs/analytics/create-application-wizard-metrics.md — the two PostHog insights (person drop-off funnel; client_id activation funnel + HogQL) and caveats.

Deployment contract

Schedule portal analytic posthog first-auth on a cron well under 35 days (daily/hourly). The forwarder scans a 35-day lookback (≤ the 90-day audit retention); the interval must stay under the lookback or first-auths occurring in a gap are missed.

Verification

go build ./..., go vet, and go test ./pkg/lib/analytic/... pass (incl. a determinism test for the event uuid).
Portal npm run typecheck / lint / fmt clean.
The three frontend events were browser-verified firing with correct payloads, and viewed fires exactly once per mount.
Tenant scoping (auto-injected WHERE app_id = ?) and SQL parameterization confirmed; the audit query is covered by the existing _audit_log(app_id, activity_type, created_at) index.

Known follow-ups (non-blocking)

Raw count() of application.first_auth can double-count clients active beyond the 35-day lookback (same uuid, later timestamp); documented — always aggregate with min() / funnel first-match (as the insights do).
The dev-only useCapture fallback instability also exists at GetStartedScreen; better fixed centrally in gtm_v2.tsx than per call site.

tung2744 · 2026-07-21T10:38:43Z

 	},
 }
+
+var cmdAnalyticPosthogFirstAuth = &cobra.Command{


@fungc-io
Instead of adding extra command which require a cronjob to run it, can we simply send an event at the moment the first auth occcurs?

Done — reworked in the latest commits. The batch subcommand + cronjob is gone; application.first_auth is now sent at the moment the first auth occurs:

Added FirstAuthSink (pkg/lib/analytic/first_auth_sink.go), a new event.Sink registered in event.NewService. It reacts to user.authenticated and m2m.token.created, taking app_id / client_id from the event context.

Once-per-client is enforced with SETNX on the analytic Redis (app:<app_id>:posthog-first-auth:<client_id>, 90-day TTL). The event uuid stays a deterministic UUIDv5 of (app_id, client_id), so PostHog dedupes even if the Redis key is ever lost.

Delivery is fire-and-forget (detached context + goroutine, same pattern as non-blocking webhook delivery), so the auth response never waits on PostHog. Missing credentials → the sink is a no-op.

Removed analytic posthog first-auth, ForwardFirstAuthEvents, and the audit-DB query GetFirstAuthTimeByClientID. No cronjob needed.

One deployment note: the main server now needs ANALYTIC_POSTHOG_ENDPOINT / ANALYTIC_POSTHOG_APIKEY set (previously only the portal had PostHog credentials), since the sink runs where auth happens.

ref DEV-2392

Expose PostHog credentials to server binaries via EnvironmentConfig.Analytic (ANALYTIC_POSTHOG_ENDPOINT / ANALYTIC_POSTHOG_APIKEY), extract a shared analytic.NewPosthogCredentials, and provide *config.AnalyticConfig in DI. ref DEV-2392

Add FirstAuthSink, an event.Sink that forwards a single application.first_auth event to PostHog the first time each OAuth client authenticates. Dedup is a SETNX on the analytic Redis; delivery is a detached fire-and-forget goroutine so the auth response is never blocked. Extract buildFirstAuthEvent so the deterministic-uuid event shape is shared. ref DEV-2392

Add the sink as a parameter of event.NewService and register analytic.FirstAuthSinkDependencySet in the common dependency set. Regenerate wire for the auth, admin, resolver, and redisqueue graphs. ref DEV-2392

The real-time FirstAuthSink replaces the batch job, so drop the 'analytic posthog first-auth' subcommand, PosthogIntegration.ForwardFirstAuthEvents, and the audit-DB query GetFirstAuthTimeByClientID. No cronjob is needed. ref DEV-2392

ref DEV-2392

These graphs also construct event.NewService and must pass the new FirstAuthSink argument. Companion to the event-service wiring commit. ref DEV-2392

tung2744 · 2026-07-22T04:37:05Z

+	logger := FirstAuthSinkLogger.GetLogger(ctx)
+
+	// Only successful-auth events count as a "first auth".
+	if e.Type != nonblocking.UserAuthenticated && e.Type != nonblocking.M2MTokenCreated {


Should this check user.created too?

tung2744 · 2026-07-22T04:42:29Z

+func (s *FirstAuthSink) markFirstAuth(ctx context.Context, appID string, clientID string, at time.Time) (bool, error) {
+	key := firstAuthDedupKey(appID, clientID)
+	var keyWasSet bool
+	err := s.AnalyticRedis.WithConnContext(ctx, func(ctx context.Context, conn redis.Redis_6_0_Cmdable) error {
+		var err error
+		keyWasSet, err = conn.SetNX(ctx, key, at.UTC().Format(time.RFC3339), firstAuthDedupTTL).Result()
+		return err
+	})
+	if err != nil {
+		return false, err
+	}
+	return keyWasSet, nil
+}
+
+func firstAuthDedupKey(appID string, clientID string) string {
+	return fmt.Sprintf("app:%s:posthog-first-auth:%s", appID, clientID)
+}


Is it possible to check audit db instead of storing a new key in redis?
The current implementation may send duplicated events if firstAuthDedupTTL passed

But actually if retention period of audit log passed there will also be duplicated events. Maybe just keep the current implementation then.

fungc-io added 7 commits July 9, 2026 11:31

[Portal] Instrument legacy Create Application funnel events

1a209d4

[Portal] Fire createApplication.viewed once per mount

c5c12c0

Add AuditDBReadStore.GetFirstAuthTimeByClientID

cfc348b

Add PostHog application.first_auth forwarder

58efceb

Add 'analytic posthog first-auth' subcommand

05eb785

doc: Add Create Application wizard PostHog report spec

0f23451

Clarify first_auth idempotency limits and assert geoip_disable

9cba4ad

fungc-io requested a review from tung2744 July 15, 2026 08:50

fungc-io assigned tung2744 Jul 15, 2026

tung2744 reviewed Jul 21, 2026

View reviewed changes

fungc-io added 8 commits July 21, 2026 15:52

doc: Design real-time application.first_auth event sink

1f60ad3

ref DEV-2392

doc: Add implementation plan for real-time first_auth sink

825789a

ref DEV-2392

[Portal] Wire FirstAuthSink into the event service

46bcba2

Add the sink as a parameter of event.NewService and register analytic.FirstAuthSinkDependencySet in the common dependency set. Regenerate wire for the auth, admin, resolver, and redisqueue graphs. ref DEV-2392

[Portal] Remove batch first-auth forwarder

f85394b

The real-time FirstAuthSink replaces the batch job, so drop the 'analytic posthog first-auth' subcommand, PosthogIntegration.ForwardFirstAuthEvents, and the audit-DB query GetFirstAuthTimeByClientID. No cronjob is needed. ref DEV-2392

doc: Update first_auth metrics doc for real-time event sink

f0d8095

ref DEV-2392

[Portal] Regenerate background and e2e wire for FirstAuthSink

e0b1341

These graphs also construct event.NewService and must pass the new FirstAuthSink argument. Companion to the event-service wiring commit. ref DEV-2392

fungc-io requested a review from tung2744 July 21, 2026 16:49

tung2744 reviewed Jul 22, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Portal] Add Create Application baseline metrics#5806

[Portal] Add Create Application baseline metrics#5806
fungc-io wants to merge 15 commits into
authgear:mainfrom
fungc-io:fungc-io/create-application-baseline-metrics

fungc-io commented Jul 9, 2026

Uh oh!

tung2744 Jul 21, 2026

Uh oh!

fungc-io Jul 21, 2026

Uh oh!

tung2744 Jul 22, 2026

Uh oh!

tung2744 Jul 22, 2026

Uh oh!

tung2744 Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

fungc-io commented Jul 9, 2026

Summary

What it adds

Deployment contract

Verification

Known follow-ups (non-blocking)

Uh oh!

tung2744 Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

fungc-io Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

tung2744 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

tung2744 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

tung2744 Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants