Skip to content

refactor: Reuse FilteredValueToken and fix architectural layering#1

Closed
rosomri wants to merge 52 commits into
masterfrom
inner_expression_parentheses_proposal
Closed

refactor: Reuse FilteredValueToken and fix architectural layering#1
rosomri wants to merge 52 commits into
masterfrom
inner_expression_parentheses_proposal

Conversation

@rosomri

@rosomri rosomri commented Apr 13, 2026

Copy link
Copy Markdown
Owner

Summary

This PR addresses the architectural concerns raised in harttle/liquidjs PR #863 review comments by:

  1. Reusing the existing FilteredValueToken instead of creating a duplicate GroupedExpressionToken
  2. Fixing the layering violation where tokens depended on templates (Filter instances)
  3. Moving filter resolution from parse time to render time

Architectural Changes

1. Reuse FilteredValueToken instead of GroupedExpressionToken

Before:

readGroupOrRange(): { type: 'range', range: RangeToken } | { type: 'groupedExpression', groupedExpression: GroupedExpressionToken }

After:

readGroupOrRange(): FilteredValueToken | RangeToken

This eliminates code duplication and leverages the existing FilteredValueToken class that was already designed for this purpose.

2. Fix Layering Violation

Problem: FilteredValueToken had resolvedFilters?: Filter[], causing tokens to depend on the template layer (violation of tokens → render → templates hierarchy).

Solution:

  • Removed resolvedFilters from FilteredValueToken
  • Added liquid reference to Context for runtime filter resolution
  • Build Filter instances at render time in evalFilteredValueToken()

Before:

export class FilteredValueToken extends Token {
  public resolvedFilters?: Filter[]  // ❌ Token depends on template layer
}

After:

export class FilteredValueToken extends Token {
  // Only has FilterToken[] - proper layering maintained ✅
}

3. Lazy Filter Resolution (Render Time)

Before: Filters were resolved at parse time via resolveGroupedExpressionFilters():

// In tag constructors
resolveGroupedExpressionFilters(this.collection, liquid)

After: Filters are resolved lazily at render time:

function * evalFilteredValueToken (token: FilteredValueToken, ctx: Context, lenient: boolean) {
  assert(ctx.liquid, 'FilteredValueToken evaluation requires liquid instance in context')
  let val = yield token.initial.evaluate(ctx, lenient)
  
  for (const filterToken of token.filters) {
    const filterImpl = ctx.liquid.filters[filterToken.name]
    const filter = new Filter(filterToken, filterImpl, ctx.liquid)
    val = yield filter.render(val, ctx)
  }
  
  return val
}

Changes by File

Core Architecture

  • src/context/context.ts: Added liquid?: any reference to Context for runtime filter resolution
  • src/liquid.ts: Pass liquid: this when creating Context instances
  • src/tokens/filtered-value-token.ts: Removed resolvedFilters (layering fix)
  • src/render/expression.ts: Build Filter instances at render time in evalFilteredValueToken()

Simplification

  • src/parser/tokenizer.ts: Simplified readGroupOrRange() to return FilteredValueToken | RangeToken
  • src/template/value.ts: Removed resolveGroupedExpressionFilters() function entirely
  • src/tags/case.ts, for.ts, tablerow.ts: Removed explicit resolveGroupedExpressionFilters() calls

Cleanup

  • src/tokens/grouped-expression-token.ts: Deleted (replaced by FilteredValueToken)
  • src/parser/token-kind.ts: Marked GroupedExpression as deprecated
  • src/util/type-guards.ts: Updated isGroupedExpressionToken() as deprecated alias

Test Results

✅ All 1537 tests pass

Test Suites: 86 passed, 86 total
Tests:       1537 passed, 1537 total

Benefits

  1. Proper Layering: Tokens → Render → Templates (no backward dependencies)
  2. Code Reuse: Eliminated duplicate GroupedExpressionToken class
  3. Lazy Evaluation: Filters resolved at render time, not parse time
  4. Simpler Code: Removed recursive resolveGroupedExpressionFilters() function
  5. Backward Compatible: All existing tests pass

Question for Review

This implementation follows @jg-rp's suggestion to add liquid to Context. The alternative mentioned in the PR comments was to pass Liquid to the Tokenizer constructor, but that would require changes to filter runtime tokenization support.

Does the approach of adding liquid to Context align with your architectural vision? Or would you prefer the Tokenizer constructor approach?

Related to harttle#863

rosomri and others added 30 commits October 27, 2025 22:40
…y compatibility (harttle#828)

* feat(filters): add base64 encode and decode

* fix: use Object.defineProperty for cross-platform btoa/atob mocking

* docs(filters): update docs

* docs(filters): update version
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
# [10.24.0](harttle/liquidjs@v10.23.0...v10.24.0) (2025-10-27)

### Features

* **filters:** Add base64_encode and base64_decode filters for Shopify compatibility ([harttle#828](harttle#828)) ([86fc135](harttle@86fc135))
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* Fix Path Traversal fallback

* Update loader.ts

Fixed nested

* Update loader.ts

padding fix

* refactor: reuse root enforcing

* docs: update test case and docs

---------

Co-authored-by: MorielHarush <93482738+MorielHarush@users.noreply.github.com>
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
# [10.25.0](harttle/liquidjs@v10.24.0...v10.25.0) (2026-03-07)

### Bug Fixes

* path traversal vulnerability, [harttle#851](harttle#851) ([harttle#855](harttle#855)) ([3cd024d](harttle@3cd024d))

### Features

* export error types, resolving [harttle#837](harttle#837) ([harttle#840](harttle#840)) ([71aa1b1](harttle@71aa1b1))
## [10.25.1](harttle/liquidjs@v10.25.0...v10.25.1) (2026-03-22)

### Bug Fixes

* mem limiter for invalid ranges ([95ddefc](harttle@95ddefc))
* treat args for replace_first as literal ([35d5230](harttle@35d5230))
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
## [10.25.2](harttle/liquidjs@v10.25.1...v10.25.2) (2026-03-25)

### Bug Fixes

* handle undefined replacement argument in replace filter ([harttle#864](harttle#864)) ([0ad2b11](harttle@0ad2b11))
…l-and-Loop-Tags' of https://github.com/skynetigor/liquidjs into 833_Support-Value-Expressions-as-Operands-in-Conditional-and-Loop-Tags

Made-with: Cursor

# Conflicts:
#	src/parser/tokenizer.ts
skynetigor and others added 22 commits April 3, 2026 16:48
…l-and-Loop-Tags' of https://github.com/skynetigor/liquidjs into 833_Support-Value-Expressions-as-Operands-in-Conditional-and-Loop-Tags
…cenarios for enabled and disabled grouped expressions in case, for, if, unless tags, ensuring proper handling of expressions and error throwing for invalid syntax.
* fix: use realpath for fs.contains

* chore: reset file mode changes

Made-with: Cursor

* fix: Windows compat for contains/containsSync and toLiquidAsync arg order

Made-with: Cursor
## [10.25.3](harttle/liquidjs@v10.25.2...v10.25.3) (2026-04-06)

### Bug Fixes

* precise memoryLimit for string replace ([abc058b](harttle@abc058b))
* use realpath for fs.contains ([harttle#867](harttle#867)) ([529dd67](harttle@529dd67))
Use _getFromScope for property access in sort/sort_natural filters to respect the ownPropertyOnly security option, preventing prototype chain traversal that could leak sensitive inherited properties.

Also extract shared sortBy helper, add orderedCompare with nil handling consistent with caseInsensitiveCompare and Ruby Liquid.

Made-with: Cursor
## [10.25.4](harttle/liquidjs@v10.25.3...v10.25.4) (2026-04-07)

### Bug Fixes

* sort and sort_natural filters bypass ownPropertyOnly ([harttle#869](harttle#869)) ([e743da0](harttle@e743da0))
* docs: update README.md [skip ci]

* docs: update .all-contributorsrc [skip ci]

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
…arttle#875)

Convert bin/ shell scripts to Node.js and npm scripts using shx and npm-run-all2. Remove unused build-icons.sh. Inlined simple scripts (build-docs-liquid, build-apidoc) as npm scripts.

Made-with: Cursor
## [10.25.5](harttle/liquidjs@v10.25.4...v10.25.5) (2026-04-07)

### Bug Fixes

* enforce root containment for renderFile/parseFile lookups ([harttle#870](harttle#870)) ([f41c1fc](harttle@f41c1fc))
* null date should return empty ([harttle#868](harttle#868)) ([harttle#872](harttle#872)) ([4f9a499](harttle@4f9a499))
* rounding negative away from zero when half ([harttle#873](harttle#873)) ([1cdf10b](harttle@1cdf10b))
The readGroupedExpression() test suite was duplicated twice in the spec file. Removed the duplicate block to avoid redundant test execution.
Extract inline grouped expression variable extraction logic into a dedicated
function for consistency with other extractors (extractFilteredValueVariables,
extractPropertyAccessVariable).

This addresses PR harttle#863 comment 7 - improves code organization and
maintainability.
collection: ValueToken | GroupedExpressionToken

Addresses PR harttle#863 comment 5.
…lters

Addresses PR review comments 4, 6, 8, 9 - moves grouped expression evaluation
from parse-time resolution to render-time lazy evaluation following the
generator-based async/sync duality pattern used throughout liquidjs.

Key changes:
- Replace resolvedValue (Value instance) with resolvedFilters (Filter[])
- Rename resolveGroupedExpressions() to resolveGroupedExpressionFilters()
- Move evaluation logic to evalGroupedExpressionToken() at render time
- Build Filter instances at parse time (carry liquid reference for render)
- Evaluate expression and apply filters lazily via generators
- Add support for tablerow tag with grouped expressions
- Remove duplicate getFilter() method in Value class

Maintains proper layering (tokens → render → templates) and consistency
with Value.value() pattern. Filter resolution still happens at parse time
since it requires liquid.filters access, but actual evaluation is deferred
to render time.

Tags that store raw ValueToken (for, case when-values, tablerow) still need
explicit resolveGroupedExpressionFilters() calls. Tags that wrap with
new Value() get automatic recursive resolution via Value constructor.
Replace GroupedExpressionToken with existing FilteredValueToken to avoid
code duplication and fix layering violation where tokens depended on
templates (Filter instances).

Key changes:
- Reuse FilteredValueToken instead of GroupedExpressionToken
- Simplify readGroupOrRange() to return FilteredValueToken | RangeToken
- Add liquid reference to Context for runtime filter resolution
- Build Filter instances at render time in evalFilteredValueToken()
- Remove resolveGroupedExpressionFilters() and parse-time resolution
- Remove explicit resolution calls from tag constructors

This maintains proper architectural layering (tokens → render → templates)
with no backward dependencies, as requested in PR review feedback.

All 1537 tests pass.
@rosomri rosomri closed this Apr 13, 2026
@rosomri
rosomri deleted the inner_expression_parentheses_proposal branch April 13, 2026 15:36
@rosomri
rosomri restored the inner_expression_parentheses_proposal branch April 13, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants