Skip to content

Implement uri.parse and uri.is_valid builtins#156

Open
shanksmp wants to merge 3 commits into
open-policy-agent:mainfrom
shanksmp:issue-135-uri-builtins
Open

Implement uri.parse and uri.is_valid builtins#156
shanksmp wants to merge 3 commits into
open-policy-agent:mainfrom
shanksmp:issue-135-uri-builtins

Conversation

@shanksmp

Copy link
Copy Markdown
Contributor

Closes #135

Summary

  • implement uri.parse
  • implement uri.is_valid
  • register both URI builtins with the evaluator
  • preserve decoded and raw URI path components
  • support IPv6 hostnames and optional URI components

Testing

  • URI compliance tests pass
  • evaluator compilation passes

The complete evaluator build also encountered unrelated Windows-specific failures in existing time-format and filesystem-path tests.

@shanksmp
shanksmp requested a review from a team as a code owner July 14, 2026 03:53
shanksmp added 2 commits July 13, 2026 22:57
Signed-off-by: Shashank-MP <shashankmechana2001@gmail.com>
Signed-off-by: Shashank-MP <shashankmechana2001@gmail.com>
@shanksmp
shanksmp force-pushed the issue-135-uri-builtins branch from 0fa89f1 to 686becf Compare July 14, 2026 03:58

@sspaink sspaink left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Output shape (port-as-string, raw_path = cmp.Or(rawPath, path), omitted empty components, empty-string→false and non-string handling) all correctly match OPA's reference v1/topdown/uri.go, and the happy-path compliance cases pass. However, the parser choice — java.net.URI (RFC 2396, strict, server-based authority) — diverges from Go's net/url.Parse (lenient) on common inputs the fixture doesn't exercise, so the builtins silently produce results that differ from OPA. Details inline. All share one root cause, so the fix is best made at the parser level (e.g. fall back to getAuthority() parsing when getHost() is null, lowercase the scheme, and use a parser that mirrors net/url acceptance). Worth extending the compliance fixture with an underscore-host case and a special-char case to catch these.

result.setProperty("scheme", new RegoString(uri.getScheme()));
}

String hostname = uri.getHost();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hostname and port are silently dropped for registry-based authorities. java.net.URI.getHost() returns null whenever the authority isn't a strict RFC-2396 server-based host (e.g. a hostname containing an underscore), and getPort() then returns -1. For uri.parse("http://my_host:8080/path") this returns {scheme:"http", path:"/path", raw_path:"/path"} — hostname and port missing — whereas OPA returns {scheme:"http", hostname:"my_host", port:"8080", path:"/path", raw_path:"/path"}. Underscores are common in internal/Docker/service hostnames, and http://my_host.example.com/ triggers it too. Consider falling back to parsing uri.getAuthority() when getHost() is null.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. URI authority parsing no longer depends on java.net.URI.getHost(). The authority is parsed directly, so registry-based hostnames such as my_host and their ports are preserved. I also added a regression test for http://my_host:8080/path, verifying that hostname is my_host and port is "8080".

URI uri = new URI(input);
RegoObject result = new RegoObject();

if (uri.getScheme() != null && !uri.getScheme().isEmpty()) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scheme is not lowercased, unlike OPA. Go's url.Parse lowercases the scheme, so uri.parse("HTTP://EXAMPLE.COM") returns scheme:"http" in OPA but scheme:"HTTP" here (URI.getScheme() preserves case). Lower impact, but a real output difference for any mixed-case scheme.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. The parsed scheme is now normalized to lowercase using Locale.ROOT, matching OPA’s behavior. I also added a regression test for uri.parse("HTTP://EXAMPLE.COM") that verifies the returned scheme is "http".

String input = getArg(args, 0, RegoString.class).getValue();

try {
URI uri = new URI(input);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uri.parse throws on inputs OPA parses successfully. new URI(...) rejects chars that are legal to net/url.Parse, e.g. uri.parse("https://example.com/path with spaces") or .../p|pe throw URISyntaxExceptionBuiltinError, whereas OPA returns a normal object with the raw path (path:"/path with spaces"). This turns a successful evaluation into an error.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. uri.parse now uses the shared OPA-compatible parsing logic instead of passing the input directly to new URI(...). It accepts literal spaces and pipes in paths and returns the expected path and raw_path. Regression tests were added for both cases.

}

try {
new URI(input.getValue());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uri.is_valid returns false for inputs OPA considers valid. Same java.net.URI strictness as in parse: uri.is_valid("https://example.com/path with spaces") and uri.is_valid("http://example.com/p|pe") return false here, but Go's url.Parse accepts both so OPA returns true.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in the latest commit. uri.is_valid now uses the same shared OPA-compatible parsing logic as uri.parse, so inputs containing literal spaces or pipes in their paths are accepted consistently. Regression tests were added for both cases.

Signed-off-by: Shashank-MP <shashankmechana2001@gmail.com>
@shanksmp

Copy link
Copy Markdown
Contributor Author

Post this general comment after replying to the inline comments:
I’ve addressed the reported compatibility differences in the latest commit.
Instead of adding only a getAuthority() fallback, I introduced shared parsing logic used by both uri.parse and uri.is_valid, because the underlying java.net.URI strictness affected multiple behaviors. The update covers:

  • lowercasing mixed-case schemes
  • preserving registry-based hostnames and ports
  • accepting literal spaces and pipes in paths
  • keeping uri.parse and uri.is_valid consistent
  • rejecting control characters and malformed percent escapes

I added focused UriBuiltinsTest regression tests rather than modifying the generated compliance fixture directly. The new regression tests and the existing URI compliance tests pass.
This change targets the reported differences with OPA’s URI behavior; I have not assumed that it reproduces every edge case of Go’s net/url parser. Please let me know if you would prefer these cases to also be added to the compliance fixture.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement URI builtins (uri.*)

2 participants