change: use RFC 9535 syntax for `jwt-role-claim-key` config by taimoorzaeem · Pull Request #4984 · PostgREST/postgrest

taimoorzaeem · 2026-06-03T09:59:34Z

BREAKING CHANGE

Breaks the string comparison operators implemented in #3813. Those can
be replaced with regex searches using JSON Path search() function.

Implementation
Tests
Docs
Changelog entry

Closes #4914.

taimoorzaeem · 2026-06-03T10:15:29Z

Now that we are following a standard, I am not sure if we should keep #4603 because it complicates the neat design. Besides, its use case seems very limited to the issue in #4599.

@steve-chavez @wolfgangwalther @merl1n0 WDYT?

steve-chavez · 2026-06-03T18:54:24Z

@taimoorzaeem One sec, so #4603 was never released so this is not really a breaking change right?

This gives us a chance to just have an amend commit, not change.

On top of it, slice operator can be used by adding the pipe symbol at the end like $.roles[0] | [1:].

Does it cover the same use case as #4599? If so, I agree with removing #4603.

Would like to see some docs to better understand the feature.

taimoorzaeem · 2026-06-04T10:16:58Z

@taimoorzaeem One sec, so #4603 was never released so this is not really a breaking change right?

Yeah not because of that, it is breaking because we lose #3813, which was released in v13. It is replaced by the standard's $.roles[?@search(<key>, <regex>)].

Does it cover the same use case as #4599? If so, I agree with removing #4603.

It covers it but is not part of the standard and I implemented the slice operator downstream. I was talking about removing the string slice because to me it seems like a bad practice to further slice a selected role value in the JWT claims. Just wanted to know any opinion on it. I will write a detailed comment on this, if that is still not clear.

Would like to see some docs to better understand the feature.

Yeah, will do, that would make it clear.

taimoorzaeem · 2026-06-05T19:59:27Z

The new syntax for jwt-role-claim-key:

$.postgrest.roles[0] | [1:0]
# ^^^^^^^^^^^^^^^^^^
# The first part here is the standard JSON Path implemented in aeson-jsonpath library

$.postgrest.roles[0] | [1:0]
#                    ^^^^^^^
# The second part is the optional slice part which is not part of the standard. Technically,
# JSON Path is a selector so it can't modify a selected value. We implemented the string slicing
# part in #4603. So to keep it, I have designed it to optionally specify that after
# a pipe '|' symbol in this PR.

Now, I was talking a bit against keeping the slice operator in a previous comment because I think our goal is to follow the standard so we don't frequently change it and break things. That way, it is also easier to document and maintain it.

I am not sure about how useful string slicing feature is. The use case in #4599 seems very niche to me. 😕

wolfgangwalther · 2026-06-05T20:41:01Z

Now, I was talking a bit against keeping the slice operator in a previous comment because I think our goal is to follow the standard so we don't frequently change it and break things. That way, it is also easier to document and maintain it.

I am not sure about how useful string slicing feature is. The use case in #4599 seems very niche to me. 😕

I agree on removing the slicing again.

steve-chavez · 2026-06-05T21:54:13Z

$.postgrest.roles[0] | [1:0]

That looks good to me.

Now, I was talking a bit against keeping the slice operator in a previous comment because I think our goal is to follow the standard so we don't frequently change it and break things. That way, it is also easier to document and maintain it.

But the standard doesn't cover the use case in #4599 right? The workaround on #4594 (comment) pollutes the db 😿

I don't think there is a problem if later on we break this config, it's after all server-side so the change is easy and in one place; unlike new functionality at the API level on which a breaking change then requires updating thousands of clients.

I am not sure about how useful string slicing feature is. The use case in #4599 seems very niche to me. 😕

IIRC it was based on another standard (ref) so maybe not that niche. If anything we should make more clear that WLCG is supported in the docs.

taimoorzaeem · 2026-06-08T06:20:23Z

I don't think there is a problem if later on we break this config, it's after all server-side so the change is easy and in one place; unlike new functionality at the API level on which a breaking change then requires updating thousands of clients.

👍

If anything we should make more clear that WLCG is supported in the docs.

It is supported already if we go by the workaround on #4594 (comment).

The workaround [...] pollutes the db

IMO, the standard (ref) should be responsible for providing clean JWT profiles.

wolfgangwalther · 2026-06-08T07:33:45Z

Just a thought: Let's not pretend to support that WLCG thing if all we're supporting is extracting that first top-level group. The example in #4594 (comment) has multiple groups, including subgroups - and we're not anywhere close to properly supporting that.

steve-chavez · 2026-06-08T19:35:42Z

But the standard doesn't cover the use case in #4599 right? The workaround on #4594 (comment) pollutes the db

The above is not true as mentioned on #4594 (reply in thread), that spec defines the name to have a /.

I agree on removing the slicing again.

Agree now, no objection from my side.

This reverts commit fe0386e. As discussed in #4984 (comment).

taimoorzaeem · 2026-06-11T19:01:34Z

+    jwt-role-claim-key = "$$.postgrest.roles[?(@ == 'author')]"
+    jwt-role-claim-key = "$$.postgrest.roles[?@.search(@, '^au')]"


When specifying this in the config file, it requires double $ chars to get parsed correctly. Works fine with single $ char in both db and env configs. Maybe something related to configurator-ng? Hm, will look into it.

configurator-ng allows to reference environment variables, right? I think that happens via $, so this must be escaped, yes.

We even have a hint at this behavior in the docs for jwt-secret: https://docs.postgrest.org/en/v14/references/configuration.html#jwt-secret

configurator-ng allows to reference environment variables, right?

Yeah, ref: https://github.com/PostgREST/configurator-pg#file-format.

We even have a hint at this behavior in the docs for jwt-secret: https://docs.postgrest.org/en/v14/references/configuration.html#jwt-secret

Cool, will document the same for this config as well.

wolfgangwalther · 2026-06-13T16:39:11Z

- ``^==`` selects the first array element that starts with the right operand
- ``==^`` selects the first array element that ends with the right operand
- ``*==`` selects the first array element that contains the right operand
+You can quickly try out JSON Path by visiting https://serdejsonpath.live/. If result has multiple values, first one gets selected.


Suggested change

You can quickly try out JSON Path by visiting https://serdejsonpath.live/. If result has multiple values, first one gets selected.

You can quickly try out JSON Path by visiting https://serdejsonpath.live/. If the result has multiple values, the first one gets selected.

I'm confused by that second sentence - is it really related to the serdejsonpath.live website?

If not, it should probably be a separate paragraph. Maybe one of them as some kind of hint or so?

Why should we promote that website? It's one more link that can break and postgres already provides jsonpath, if anything we should promote the latter.

is it really related to the serdejsonpath.live website? If not, it should probably be a separate paragraph.

Yeah it is not related, will move it.

Why should we promote that website? It's one more link that can break and postgres already provides jsonpath, if anything we should promote the latter.

No specific reason to promote it, any website that can help users to quickly try it out could work here. postgres jsonpath isn't fully RFC 9535 compliant, for example: the following doesn't work when it should:

SELECT jsonb_path_query('{"names": ["alice", "bob"]}'::jsonb, '$.names[?search(@, "alice|bob")]');

wolfgangwalther · 2026-06-13T16:52:04Z

+  escapeDoubleQuotes jsPathDump
+    where
+      -- When dumping, an extra $ is prefixed to escape the other $ char
+      jsPathDump = "$" <> JSP.dumpQuery query


I don't think this is enough. $ can occur multiple times in such a query. I'm not sure whether the json path syntax allows multiple references to the root document... but I'm sure there is a way to put a $ into a string / regular identifier somehow or have it in a regex for search() or elsewhere?

Great catch! Yes, JSON Path syntax allows referencing the root path again. We should just escape the $ char all over.

steve-chavez · 2026-06-16T00:19:31Z

-dumpJSPath (JSPIdx i) = "[" <> show i <> "]"
-dumpJSPath (JSPFilter cond) = "[?(@" <> expr <> ")]"
-  where
-    expr =
-      case cond of
-        EqualsCond text     -> " == " <> show text
-        NotEqualsCond text  -> " != " <> show text
-        StartsWithCond text -> " ^== " <> show text
-        EndsWithCond text   -> " ==^ " <> show text
-        ContainsCond text   -> " *== " <> show text
+-- | Dump JSPath
+--   e.g. "$.property[0].attr.detail[?(@ == "role1")]"
+dumpJSPath :: JSPath -> Text
+dumpJSPath (JSPath query) = (escapeDollarChar . escapeDoubleQuotes) jsPathDump
+    where
+      jsPathDump = JSP.dumpQuery query
+      escapeDoubleQuotes = T.replace "\"" "\\\""
+      -- When dumping, $ must be escaped
+      escapeDollarChar = T.replace "$" "$$"

 -- | Evaluate JSPath on a JSON
-walkJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
-walkJSPath x                      []                = x
-walkJSPath (Just (JSON.Object o)) (JSPKey key:rest) = walkJSPath (KM.lookup (K.fromText key) o) rest
-walkJSPath (Just (JSON.Array ar)) (JSPIdx idx:rest) = walkJSPath (ar V.!? idx) rest
-walkJSPath (Just (JSON.Array ar)) [JSPFilter jspFilter] = case jspFilter of
-    EqualsCond txt     -> findFirstMatch (==) txt ar
-    NotEqualsCond txt  -> findFirstMatch (/=) txt ar
-    StartsWithCond txt -> findFirstMatch T.isPrefixOf txt ar
-    EndsWithCond txt   -> findFirstMatch T.isSuffixOf txt ar
-    ContainsCond txt   -> findFirstMatch T.isInfixOf txt ar
-  where
-    findFirstMatch matchWith pattern = find (\case
-      JSON.String txt -> pattern `matchWith` txt
-      _               -> False)
-walkJSPath _                      _                 = Nothing
+evaluateJSPath :: Maybe JSON.Value -> JSPath -> Maybe JSON.Value
+evaluateJSPath Nothing _ = Nothing
+evaluateJSPath (Just json) (JSPath query) = role


Q: Are we covering in loadtests any performance improvements or regressions when using the JSPath? If not, could we add a new loadtest?

Looks like for loadtest we only cover the default scenario of $.role:

postgrest/nix/tools/generate_targets.py

Lines 28 to 38 in fae6253

def generate_jwt(

now: int,

exp_inc: Optional[int],

rsa_private_key: Optional[jwt.algorithms.RSAAlgorithm],

) -> str:

"""Generate an HS256 or RS256 JWT"""

payload = {

"sub": f"user_{random.getrandbits(32)}",

"iat": now,

"role": "postgrest_test_author",

}

But yeah that's not good enough. We should add a loadtest I think. Will look into it.

Let's make sure we're not adding yet another entirely separate loadtest for this.

It should be enough to just change some/all of the JWT tests to use at least some, possibly all, tokens with a more complex jwt-role-claim-key. It's entirely OK to test multiple things at once.

wolfgangwalther · 2026-06-16T19:42:59Z

 .. note::

-  The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
+  - If the result of JSON Path query contains multiple values, the first one gets selected.


Suggested change

- If the result of JSON Path query contains multiple values, the first one gets selected.

- If the result of the JSON Path query contains multiple values, the first one gets selected.

Maybe even better: "If the JSON Path query returns multiple values, ..."

wolfgangwalther · 2026-06-16T19:44:19Z

-  The string comparison operators are implemented as a custom extension to the JSPath and does not strictly follow the `RFC 9535 <https://www.rfc-editor.org/rfc/rfc9535.html>`_.
+  - If the result of JSON Path query contains multiple values, the first one gets selected.
+  - Only when using the :ref:`file_config`, all ``$`` characters in the value must be escaped with an additional ``$`` char. For :ref:`env_variables_config` and :ref:`in_db_config`, only use a single ``$`` char.
+  - In our implementation, only the `search()` function from JSON Path is available for filtering.


Suggested change

- In our implementation, only the `search()` function from JSON Path is available for filtering.

- Currently, `search()` is the only JSON Path filter implemented.

Can we maybe link "JSON Path filter" to an authoritative source on what other filters there should be?

Sure, will link https://www.rfc-editor.org/rfc/rfc9535.html#name-function-extensions here.

wolfgangwalther · 2026-06-16T19:46:48Z

+- Config `jwt-role-claim-key` now uses RFC 9535 syntax for JSON Path by @taimoorzaeem in #4984
+  + Removes the string comparison operators implemented in #3813 in favor of using JSON Path `search()` function


I think we need a vastly expanded changelog entry. "migration guide" style, with examples what people where using previously and should be using now etc.

WDYT?

Yes, agreed. It would be much easier for users.

taimoorzaeem · 2026-06-20T07:02:35Z

I have run loadtest with:

$ postgrest-loadtest -k jwt-hs

Array Index

Performance loss is little.

# payload = {
#   ... ,
#   "postgrest" : { "roles" : ["other", "postgrest_test_author"] }
# }

# .postgrest.roles[1] (main)

Requests      [total, rate, throughput]         200001, 6272.43, 6272.40
Duration      [total, attack, wait]             31.886s, 31.886s, 420ns
Latencies     [min, mean, 50, 90, 95, 99, max]  420ns, 158.564µs, 143.674µs, 245.273µs, 281.492µs, 344.152µs, 3.764ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

# $.postgrest.roles[1] (HEAD)

Requests      [total, rate, throughput]         200001, 5864.22, 5864.19
Duration      [total, attack, wait]             34.105s, 34.105s, 551ns
Latencies     [min, mean, 50, 90, 95, 99, max]  551ns, 169.711µs, 157.525µs, 273.823µs, 294.209µs, 360.819µs, 20.214ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

Regex search

Performance loss is significant because regex search is expensive as compared to a prefix check.

# payload = {
#   ... ,
#   "postgrest" : { "roles" : ["other", "postgrest_test_author"] }
# }

# .postgrest.roles[?(@ ^== "postgrest_")] (main)

Requests      [total, rate, throughput]         200001, 6096.12, 6096.09
Duration      [total, attack, wait]             32.808s, 32.808s, 401ns
Latencies     [min, mean, 50, 90, 95, 99, max]  401ns, 163.184µs, 155.913µs, 250.21µs, 285.162µs, 349.578µs, 3.733ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

# $.postgrest.roles[?search(@, '^postgrest_')] (HEAD)

Requests      [total, rate, throughput]         200001, 3356.67, 3356.65
Duration      [total, attack, wait]             59.583s, 59.583s, 912ns
Latencies     [min, mean, 50, 90, 95, 99, max]  912ns, 296.792µs, 273.828µs, 432.092µs, 467.252µs, 556.505µs, 3.84ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

wolfgangwalther · 2026-06-21T11:02:54Z

Array Index

Performance loss is little.

I think we should investigate this, because it affects everyone.

I didn't look at the code again, but are we making sure that we're only parsing or string config value for the jspath once? Where else could we lose that performance?

Regex search

Performance loss is significant because regex search is expensive as compared to a prefix check.

I wouldn't care about this. We're just comparing apples and oranges here, so not really a meaningful number. Also very much a special case anyway.

taimoorzaeem · 2026-06-22T12:37:38Z

I didn't look at the code again, but are we making sure that we're only parsing or string config value for the jspath once? Where else could we lose that performance?

Yes, only once. I found the culprit, I was doing an extra case check which was impacting performance.

Updated loadtest result:

Array Index

Slight improvement now:

# payload = {
#   ... ,
#   "postgrest" : { "roles" : ["other", "postgrest_test_author"] }
# }

# .postgrest.roles[1]

Requests      [total, rate, throughput]         200001, 6126.28, 6126.25
Duration      [total, attack, wait]             32.646s, 32.646s, 501ns
Latencies     [min, mean, 50, 90, 95, 99, max]  501ns, 162.392µs, 154.355µs, 268.83µs, 289.308µs, 354.268µs, 18.913ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

# $.postgrest.roles[1]

Requests      [total, rate, throughput]         200001, 6281.94, 6281.91
Duration      [total, attack, wait]             31.837s, 31.837s, 501ns
Latencies     [min, mean, 50, 90, 95, 99, max]  501ns, 158.278µs, 146.285µs, 256.881µs, 288.409µs, 354.087µs, 3.472ms
Bytes In      [total, mean]                     0, 0.00
Bytes Out     [total, mean]                     0, 0.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      0:1  200:200000

BREAKING CHANGE Breaks the string comparison operators implemented in PostgREST#3813. Those can be replaced with regex searches using JSON Path `search()` function. Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>

taimoorzaeem marked this pull request as draft June 3, 2026 09:59

taimoorzaeem added the breaking change A bug fix or enhancement that would cause a breaking change label Jun 3, 2026

taimoorzaeem force-pushed the break/aeson-jsonpath branch from 0af8877 to 6746de2 Compare June 3, 2026 10:08

taimoorzaeem changed the title ~~break: use RFC 9535 syntax for jwt-role-claim-key config~~ change: use RFC 9535 syntax for jwt-role-claim-key config Jun 3, 2026

taimoorzaeem force-pushed the break/aeson-jsonpath branch from 6746de2 to 6079929 Compare June 3, 2026 10:16

taimoorzaeem force-pushed the break/aeson-jsonpath branch from 6079929 to a871fcf Compare June 5, 2026 19:42

taimoorzaeem mentioned this pull request Jun 11, 2026

Revert "add: string slicing operator for jwt-role-claim-key" #5000

Merged

taimoorzaeem added a commit that referenced this pull request Jun 11, 2026

Revert "add: string slicing operator for jwt-role-claim-key"

e9a0166

This reverts commit fe0386e. As discussed in #4984 (comment).

taimoorzaeem force-pushed the break/aeson-jsonpath branch from a871fcf to e0af2c9 Compare June 11, 2026 18:54

taimoorzaeem commented Jun 11, 2026

View reviewed changes

taimoorzaeem force-pushed the break/aeson-jsonpath branch 2 times, most recently from efc2b04 to 6cfdc13 Compare June 12, 2026 20:15

taimoorzaeem marked this pull request as ready for review June 12, 2026 20:15

wolfgangwalther reviewed Jun 13, 2026

View reviewed changes

taimoorzaeem marked this pull request as draft June 15, 2026 08:35

taimoorzaeem force-pushed the break/aeson-jsonpath branch from 6cfdc13 to e5b61ca Compare June 15, 2026 08:36

taimoorzaeem marked this pull request as ready for review June 15, 2026 14:30

taimoorzaeem requested review from steve-chavez and wolfgangwalther June 15, 2026 14:30

steve-chavez reviewed Jun 16, 2026

View reviewed changes

taimoorzaeem marked this pull request as draft June 16, 2026 07:38

wolfgangwalther reviewed Jun 16, 2026

View reviewed changes

taimoorzaeem mentioned this pull request Jun 17, 2026

test(load): jwt loadtests with complex jwt-role-claim-key #5013

Closed

taimoorzaeem force-pushed the break/aeson-jsonpath branch from e5b61ca to d9e8170 Compare June 22, 2026 12:35

taimoorzaeem force-pushed the break/aeson-jsonpath branch 2 times, most recently from b57d5f4 to f3ea248 Compare June 22, 2026 17:27

change: use RFC 9535 syntax for jwt-role-claim-key config

90c3bee

BREAKING CHANGE Breaks the string comparison operators implemented in PostgREST#3813. Those can be replaced with regex searches using JSON Path `search()` function. Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>

taimoorzaeem force-pushed the break/aeson-jsonpath branch from f3ea248 to 90c3bee Compare June 22, 2026 17:27

taimoorzaeem marked this pull request as ready for review June 22, 2026 17:29

		jwt-role-claim-key = "$$.postgrest.roles[?(@ == 'author')]"
		jwt-role-claim-key = "$$.postgrest.roles[?@.search(@, '^au')]"

	You can quickly try out JSON Path by visiting https://serdejsonpath.live/. If result has multiple values, first one gets selected.
	You can quickly try out JSON Path by visiting https://serdejsonpath.live/. If the result has multiple values, the first one gets selected.

	def generate_jwt(
	now: int,
	exp_inc: Optional[int],
	rsa_private_key: Optional[jwt.algorithms.RSAAlgorithm],
	) -> str:
	"""Generate an HS256 or RS256 JWT"""
	payload = {
	"sub": f"user_{random.getrandbits(32)}",
	"iat": now,
	"role": "postgrest_test_author",
	}

	- If the result of JSON Path query contains multiple values, the first one gets selected.
	- If the result of the JSON Path query contains multiple values, the first one gets selected.

	- In our implementation, only the `search()` function from JSON Path is available for filtering.
	- Currently, `search()` is the only JSON Path filter implemented.

		- Config `jwt-role-claim-key` now uses RFC 9535 syntax for JSON Path by @taimoorzaeem in #4984
		+ Removes the string comparison operators implemented in #3813 in favor of using JSON Path `search()` function

Uh oh!

Uh oh!

Conversation

taimoorzaeem commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

taimoorzaeem commented Jun 3, 2026

Uh oh!

steve-chavez commented Jun 3, 2026

Uh oh!

taimoorzaeem commented Jun 4, 2026

Uh oh!

taimoorzaeem commented Jun 5, 2026

Uh oh!

wolfgangwalther commented Jun 5, 2026

Uh oh!

steve-chavez commented Jun 5, 2026

Uh oh!

taimoorzaeem commented Jun 8, 2026

Uh oh!

wolfgangwalther commented Jun 8, 2026

Uh oh!

steve-chavez commented Jun 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

wolfgangwalther Jun 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taimoorzaeem Jun 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taimoorzaeem Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

taimoorzaeem commented Jun 20, 2026

Array Index

Regex search

Uh oh!

wolfgangwalther commented Jun 21, 2026

Array Index

Regex search

Uh oh!

taimoorzaeem commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Array Index

Uh oh!

Reviewers

taimoorzaeem commented Jun 3, 2026 •

edited

Loading

wolfgangwalther Jun 12, 2026 •

edited

Loading

taimoorzaeem Jun 15, 2026 •

edited

Loading

taimoorzaeem Jun 17, 2026 •

edited

Loading

taimoorzaeem commented Jun 22, 2026 •

edited

Loading