Skip to content

Support profiles custom events filters #482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Blaumaus wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Support profiles custom events filters #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion backend/apps/cloud/src/analytics/analytics.controller.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2127,7 +2127,7 @@ export class AnalyticsController {
timeBucket = getLowestPossibleTimeBucket(period, from, to)
}

const [filtersQuery, filtersParams, appliedFilters] =
const [filtersQuery, filtersParams, appliedFilters, customEVFilterApplied] =
this.analyticsService.getFiltersQuery(filters, DataType.ANALYTICS)

const safeTimezone = this.analyticsService.getSafeTimezone(timezone)
Expand Down Expand Up @@ -2157,6 +2157,7 @@ export class AnalyticsController {
take,
skip,
profileType,
customEVFilterApplied,
)

return { profiles, appliedFilters, take, skip }
Expand Down
123 changes: 88 additions & 35 deletions backend/apps/cloud/src/analytics/analytics.service.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4940,6 +4940,7 @@ export class AnalyticsService {
take = 30,
skip = 0,
profileType: 'all' | 'anonymous' | 'identified' = 'all',
customEVFilterApplied = false,
): Promise<object[]> {
let profileTypeFilter = ''
if (profileType === 'anonymous') {
Expand All @@ -4948,42 +4949,94 @@ export class AnalyticsService {
profileTypeFilter = `AND profileId LIKE '${AnalyticsService.PROFILE_PREFIX_USER}%'`
}

let allProfileDataCTE: string

if (customEVFilterApplied) {
allProfileDataCTE = `
all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
AND profileId IN (
SELECT DISTINCT profileId
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)`
Comment on lines +4954 to +4998
Copy link

@coderabbitai coderabbitai bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Avoid mixing filtered event totals with unfiltered analytics totals.

In this branch, ${filtersQuery} is applied to the customEV arm but not to the analytics arm. Once a profile matches one filtered event, the row starts accumulating unfiltered pageviews/sessions while eventsCount stays filtered, so the profile totals no longer describe the same slice of data.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/apps/cloud/src/analytics/analytics.service.ts` around lines 4954 -
4998, The analytics arm inside allProfileDataCTE is missing ${filtersQuery},
causing pageviews/sessions to be unfiltered while events are filtered; update
the analytics SELECT (the first UNION ALL branch in allProfileDataCTE) to
include the same ${filtersQuery} (and keep ${profileTypeFilter} and pid checks)
so that analytics rows are restricted to the same event-level filters used for
customEV, ensuring eventsCount and pageview/session totals represent the same
filtered slice.

} else {
allProfileDataCTE = `
all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)`
Comment on lines +4954 to +5035
Copy link

@coderabbitai coderabbitai bot Mar 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Constrain all_profile_data to the requested time window.

groupFrom/groupTo are already passed into this method, but none of the new CTE branches use them. That makes /analytics/profiles aggregate all-time rows, so sessionsCount, pageviewsCount, eventsCount, firstSeen, and lastSeen ignore the selected period.

🐛 Proposed fix 
         all_profile_data AS (
           SELECT
             profileId,
@@
           FROM analytics
           WHERE pid = {pid:FixedString(12)}
             AND profileId IS NOT NULL
             AND profileId != ''
             ${profileTypeFilter}
+            AND created BETWEEN {groupFrom:String} AND {groupTo:String}
             AND profileId IN (
               SELECT DISTINCT profileId
               FROM customEV
               WHERE pid = {pid:FixedString(12)}
                 AND profileId IS NOT NULL
                 AND profileId != ''
                 ${profileTypeFilter}
+                AND created BETWEEN {groupFrom:String} AND {groupTo:String}
                 ${filtersQuery}
             )
@@
           FROM customEV
           WHERE pid = {pid:FixedString(12)}
             AND profileId IS NOT NULL
             AND profileId != ''
             ${profileTypeFilter}
+            AND created BETWEEN {groupFrom:String} AND {groupTo:String}
             ${filtersQuery}
         )`
@@
         all_profile_data AS (
           SELECT
             profileId,
@@
           FROM analytics
           WHERE pid = {pid:FixedString(12)}
             AND profileId IS NOT NULL
             AND profileId != ''
             ${profileTypeFilter}
+            AND created BETWEEN {groupFrom:String} AND {groupTo:String}
             ${filtersQuery}
@@
           FROM customEV
           WHERE pid = {pid:FixedString(12)}
             AND profileId IS NOT NULL
             AND profileId != ''
             ${profileTypeFilter}
+            AND created BETWEEN {groupFrom:String} AND {groupTo:String}
             ${filtersQuery}
         )`
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@backend/apps/cloud/src/analytics/analytics.service.ts` around lines 4954 -
5035, The all_profile_data CTE currently omits the requested time window, so
update the CTE construction (both branches where customEVFilterApplied is
true/false and inside the analytics SELECT, the customEV SELECT, and the IN
subquery) to constrain rows to the groupFrom/groupTo range by adding the created
time filter (use groupFrom/groupTo parameters into the ClickHouse query
placeholders, e.g., created >= {groupFrom} AND created <= {groupTo} or an
equivalent BETWEEN expression). Ensure you add this filter alongside existing
WHERE clauses in the code that builds allProfileDataCTE (references:
allProfileDataCTE variable, groupFrom, groupTo, customEVFilterApplied,
filtersQuery, profileTypeFilter) so
sessionsCount/pageviewsCount/eventsCount/firstSeen/lastSeen reflect the selected
period.

}

const query = `
WITH all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
),
WITH ${allProfileDataCTE},
profile_aggregated AS (
SELECT
profileId,
Expand Down
3 changes: 2 additions & 1 deletion backend/apps/community/src/analytics/analytics.controller.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2068,7 +2068,7 @@ export class AnalyticsController {
timeBucket = getLowestPossibleTimeBucket(period, from, to)
}

const [filtersQuery, filtersParams, appliedFilters] =
const [filtersQuery, filtersParams, appliedFilters, customEVFilterApplied] =
this.analyticsService.getFiltersQuery(filters, DataType.ANALYTICS)

const safeTimezone = this.analyticsService.getSafeTimezone(timezone)
Expand Down Expand Up @@ -2098,6 +2098,7 @@ export class AnalyticsController {
take,
skip,
profileType,
customEVFilterApplied,
)

return { profiles, appliedFilters, take, skip }
Expand Down
123 changes: 88 additions & 35 deletions backend/apps/community/src/analytics/analytics.service.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5182,6 +5182,7 @@ export class AnalyticsService {
take = 30,
skip = 0,
profileType: 'all' | 'anonymous' | 'identified' = 'all',
customEVFilterApplied = false,
): Promise<object[]> {
let profileTypeFilter = ''
if (profileType === 'anonymous') {
Expand All @@ -5190,42 +5191,94 @@ export class AnalyticsService {
profileTypeFilter = `AND profileId LIKE '${AnalyticsService.PROFILE_PREFIX_USER}%'`
}

let allProfileDataCTE: string

if (customEVFilterApplied) {
allProfileDataCTE = `
all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
AND profileId IN (
SELECT DISTINCT profileId
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)`
} else {
allProfileDataCTE = `
all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
)`
}

const query = `
WITH all_profile_data AS (
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
1 AS isPageview,
0 AS isEvent
FROM analytics
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
UNION ALL
SELECT
profileId,
psid,
cc,
os,
br,
dv,
created,
0 AS isPageview,
1 AS isEvent
FROM customEV
WHERE pid = {pid:FixedString(12)}
AND profileId IS NOT NULL
AND profileId != ''
${profileTypeFilter}
${filtersQuery}
),
WITH ${allProfileDataCTE},
profile_aggregated AS (
SELECT
profileId,
Expand Down