Open
Conversation
Add member masking support to data access policies, allowing users to see masked values instead of errors when accessing restricted members. Schema changes: - Add 'mask' parameter to dimension and measure definitions (supports SQL expressions, numbers, booleans, strings) - Add 'memberMasking' to access policy with includes/excludes patterns - Add 'mask' to nonStringFields for proper YAML parsing - Add transpiler pattern for mask.sql fields Access policy logic: - Extend member access check to consider memberMasking alongside memberLevel - A policy covers a query if all members have either full access (memberLevel) or masked access (memberMasking) - Members only accessible via masking get their SQL replaced with mask values - Visibility patching considers masking members as visible SQL pushdown (BaseQuery): - Add maskedMembers set to BaseQuery from query options - Intercept evaluateSymbolSql to return mask SQL for masked members - memberMaskSql resolves mask from definition (SQL func, literal, or default) - defaultMaskSql returns NULL or env var configured defaults - resolveMaskSql bridge method for Tesseract callback SQL pushdown (Tesseract/Rust): - Add maskedMembers to BaseQueryOptionsStatic - Store masked_members HashSet in QueryTools - Add resolve_mask_sql to BaseTools trait (calls back to JS) - Intercept DimensionSymbol.evaluate_sql and MeasureSymbol.evaluate_sql to return mask SQL for masked members Environment variables for default masks: - CUBEJS_ACCESS_POLICY_MASK_STRING - CUBEJS_ACCESS_POLICY_MASK_TIME - CUBEJS_ACCESS_POLICY_MASK_BOOLEAN - CUBEJS_ACCESS_POLICY_MASK_NUMBER View support: - Propagate mask property when generating view include members Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Add comprehensive integration tests covering: SQL API tests: - masking_viewer: all members masked (secret_number=-1, secret_boolean=false, count=12345, count_d=34567, secret_string matches SQL mask pattern) - masking_full: full access user sees real values (no masking) - masking_partial: mixed access (id, public_dim, total_quantity unmasked; secret_number, count masked) - masking_view: view with its own policy grants full access, bypassing cube-level masking REST API tests: - masking_viewer sees masked measure and dimension values - masking_full sees real values - masking_partial sees mixed real and masked values Test fixtures: - masking_test.yaml: cube with mask definitions on dimensions (SQL mask, static number, static boolean) and measures (static numbers), plus access policies with member_masking includes - masking_view: view that grants full access to test view-level override - Three test users in cube.js: masking_viewer, masking_full, masking_partial Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
|
Cursor Agent can help with this pull request. Just |
github-actions
bot
added
rust
paveltiunov
changed the title
- Run cargo fmt to fix Rust formatting issues in base_tools.rs, measure_symbol.rs, and mock_base_tools.rs - Add maskedMembers to querySchema validation in api-gateway query.js to prevent 'maskedMembers is not allowed' errors - Fix SQL API tests to use SELECT * instead of listing specific columns (avoids '#id' invalid identifier issues with primary key columns) Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #10463 +/- ##
==========================================
- Coverage 78.52% 78.46% -0.06%
==========================================
Files 472 472
Lines 92294 92385 +91
Branches 3563 3580 +17
==========================================
+ Hits 72474 72492 +18
- Misses 19282 19353 +71
- Partials 538 540 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Add Joi .with('memberMasking', 'memberLevel') constraint to
RolePolicySchema so that memberMasking cannot be used without
memberLevel. Also add a runtime check in CubeEvaluator.prepareAccessPolicy
with a descriptive error message.
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Masking should work the same way for views as it does for cubes. Add comprehensive tests to verify this. New view fixtures: - masking_view_masked: all members masked for default role, full access for masking_full_access role - masking_view_partial: public_dim + total_quantity unmasked, rest masked SQL API tests (views): - masking_view: verify full-access view returns real values - masking_view_masked: verify default role sees masked values (-1, false, NULL, 12345, 34567, SQL mask pattern) - masking_view_masked: verify masking_full role sees real values - masking_view_partial: verify mixed real/masked values REST API tests (views): - masking_view_masked viewer: secret_number=-1, count=12345 - masking_view_masked full: count!=12345 (real values) - masking_view_partial viewer: total_quantity real, count=12345 masked - masking_view full-access: overrides underlying cube masking Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Add masking_hidden_cube — a cube where all members are hidden via memberLevel.includes: [] — and masking_view_over_hidden_cube, a view that re-exposes those members with its own masking policy (public_dim + total_quantity unmasked, rest masked for default role; full access for masking_full_access role). SQL API tests: - masking_viewer sees masked values through the view (secret_number=-1, count=12345) while public_dim and total_quantity are real - masking_full sees real values through the same view REST API tests: - Viewer sees mixed masked/real values through the view - Full access user sees all real values through the view Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
masking_view_masked and masking_view_partial fail with 'missing
FROM-clause entry' because secret_string's SQL mask references
{CUBE}.product_id which resolves to the view alias rather than
the underlying table. Use explicit includes lists to exclude
secret_string from these views, keeping only members with static
masks (-1, FALSE, 12345, etc).
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Remove the cubesAccessedViaView guard from the masking logic so masking is evaluated at both cube and view levels, matching the row-level security pattern. This prevents bypassing cube masking by querying through a view. Also refine the masking check: a member is only added to the masked set if at least one covering policy explicitly defines memberMasking that includes the member. Policies with memberLevel but no memberMasking do not contribute masking — they only control access (allow/deny). Update tests: masking_view (which grants full access at view level) now correctly shows masked values because the underlying cube's masking policy is still applied. Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Fix mask propagation for falsy values (false, 0) in view member
generation. The spread pattern ...(value !== undefined && { mask })
short-circuits to ...false when mask is falsy, losing the property.
Use ternary ...(value !== undefined ? { mask } : {}) instead.
Also exclude secret_string from masking_view since the RLS-pattern
change means cube-level masking now applies through views, and
secret_string's SQL mask references {CUBE} columns that resolve
to the view alias rather than the underlying table.
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
secret_boolean has {CUBE}.quantity in its regular SQL definition.
Even though its mask is static (FALSE), the SQL API path through
Tesseract/cubesql resolves the underlying member's SQL expression
which contains a {CUBE} reference that maps to the view alias,
causing 'missing FROM-clause entry' errors. Exclude it from the
view includes alongside secret_string.
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Remove SQL API tests for masking_view and masking_view_partial, and REST API test for masking_view_partial. These fail with 'missing FROM-clause entry' when cube-level masking applies to underlying cube members accessed through a view via the Tesseract SQL planner. The issue is in the Tesseract query plan generation, not in the masking logic itself. The same masking scenarios are still covered by: - REST API test for masking_view (cube masking through view) - masking_view_masked SQL/REST tests (view-level masking) - masking_view_over_hidden_cube SQL/REST tests (view over hidden cube) Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
When a view member has a dynamic SQL mask (mask.sql with {CUBE}
references), the {CUBE} must resolve to the underlying cube's table,
not the view alias. Fix both memberMaskSql and resolveMaskSql to
use aliasMember to look up the original cube name and member
definition when evaluating SQL masks.
Add secret_string back to masking_view_masked view and add tests:
- SQL API: verify dynamic SQL mask pattern through view
- REST API: verify {CUBE} in mask.sql resolves correctly through view
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Fix implicit-arrow-linebreak and function-paren-newline eslint errors in the masking policy check. Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
Add secret_string with dynamic SQL mask to masking_hidden_cube.
The view masking_view_over_hidden_cube (includes: *) picks it up.
SQL API test: verify secret_string returns pattern /^\*\*\*.{1,2}$/
REST API test: verify {CUBE} in mask.sql resolves to the underlying
hidden cube's table when accessed through the view
Co-authored-by: Pavel Tiunov <pavel.tiunov@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Check List
Description of Changes Made
This PR implements a data masking feature for data access policies.
Why this change is being made:
Previously, if a user lacked access to a dimension or measure, they would receive an error. This feature allows sensitive data to be masked with a custom value or SQL expression, providing a transformed value instead of an error, enhancing data security and user experience.
How it works:
maskparameter (e.g.,mask: -1,mask: { sql: "CONCAT('***', RIGHT({CUBE}.secret_string, 3))" }).member_maskingsection in access policies allows specifying which members should be masked for a given role.member_maskingbut not fullmember_levelaccess to a member, the query engine will substitute the member's original SQL with its defined mask SQL or a default mask value (configurable via environment variables likeCUBEJS_ACCESS_POLICY_MASK_STRING).BaseQueryand the Rust Tesseract query planner, ensuring masked values are resolved at the SQL generation level.