CXP-533 Cap event feed lookback to 90 days to prevent timeout#108
Conversation
Google page tokens expire after ~24h. A cursor left mid-pagination would keep requesting the full historical window on every retry, causing HTTP timeouts on large orgs. Enforce a 31-day cutoff in unmarshalPageToken: clamp defaultStart and reset stale or unparseable cursors, clearing the expired NextPageToken so the next call starts fresh from the cutoff. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
JavierCarnelli-ConductorOne
requested review from
a team,
FeliLucero1,
agustin-conductor,
johnallers,
luisina-santos,
mateoHernandez123,
mateovespConductor and
sergiocorral-conductorone
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| cutoff := time.Now().Add(-maxEventFeedLookback) | ||
| if defaultStart == nil || defaultStart.AsTime().Before(cutoff) { | ||
| // There's lag on these events, so we're going to start roughly when google says events should come in | ||
| // https://support.google.com/a/answer/7061566?fl=1&sjid=13551023455982018638-NC (Data Retention and Lag Times) | ||
| defaultStart = timestamppb.New(cutoff) | ||
| } |
There was a problem hiding this comment.
🟡 Suggestion: The original code defaulted to a 2-hour lookback for fresh starts (nil defaultStart, empty cursor) to match Google's documented event delivery lag. This change widens that default to 31 days, which on large orgs could cause the same timeout issue the PR aims to prevent. Consider preserving the 2-hour default for fresh starts while clamping only stale cursors and caller-provided start times to 31 days.
The Google lag-time comment (carried over from the original) was written for the 2-hour window and is misleading when applied to the 31-day cutoff.
There was a problem hiding this comment.
C1 always provides a value, so I would be surprised if we ever hit the 2-hour window. Two hours is too small for useful usage data, so that really wasn't a good default to begin with.
Connector PR Review: CXP-533 Cap event feed lookback to 90 days to prevent timeoutBlocking Issues: 0 | Suggestions: 0 | Threads Resolved: 0 Review SummaryThe new commit widens Security IssuesNone found. Correctness IssuesNone found. SuggestionsNone. |
FeliLucero1
left a comment
FeliLucero1
left a comment
There was a problem hiding this comment.
worth looking at the bot comment
JavierCarnelli-ConductorOne
changed the title
4 participants
Google page tokens expire after ~24h. A cursor left mid-pagination would keep requesting the full historical window on every retry, causing HTTP timeouts on large orgs. Enforce a 90-day cutoff in unmarshalPageToken: clamp defaultStart and reset stale or unparseable cursors, clearing the expired NextPageToken so the next call starts fresh from the cutoff.