[Sandbox SQL snapshot] Pin org.opensearch.query:* (unified-query-*) snapshots to mavenLocal#21578
Conversation
The remote OpenSearch Snapshots maven repo (ci.opensearch.org/ci/dbc/snapshots) only republishes from sql/main, not from sql/feature/mustang-ppl-integration, so its 3.7.0.0-SNAPSHOT jars trail the feature branch by however many merges (currently missing PPL_REX_MAX_MATCH_LIMIT, CALCITE_ENGINE_ENABLED, …). The sandbox-check workflow's pre-step opensearch-project#21569 publishes feature-branch unified-query jars to mavenLocal, but Gradle's default SNAPSHOT resolution weighs the remote's explicit <buildNumber>/<timestamp> metadata higher than mavenLocal's <localCopy>true>, so the stale remote wins even when mavenLocal has a newer <lastUpdated>. Confirmed via dependencyInsight: every consumer was binding unified-query-api:3.7.0.0-SNAPSHOT:20260507.224009-12 (60kB, 42 classes, no PPL_REX_MAX_MATCH_LIMIT field reference) instead of the locally-published 3.7.0.0-SNAPSHOT (29kB, 21 classes, has the field). The runtime cluster inherited that stale class via the test-ppl-frontend plugin bundle, which is why every IT touching `rex` failed plan-time with `NullPointerException: Cannot invoke "java.lang.Integer.intValue()" because the return value of "Settings.getSettingValue(PPL_REX_MAX_MATCH_LIMIT)" is null` once the unified path tried to read the setting. Fix: tell the OpenSearch Snapshots remote to refuse `org.opensearch.query` artifacts via mavenContent { excludeGroup }. Three sites declare the remote: * sandbox/build.gradle subprojects { repositories } — applies to every sandbox subproject including qa. * sandbox/plugins/analytics-backend-datafusion/build.gradle — own declaration; left in place for module isolation, filtered identically. * sandbox/plugins/test-ppl-frontend/build.gradle — also pin mavenLocal as the only source for org.opensearch.query so the bundlePlugin task bundles the freshly-published feature-branch jar rather than the stale timestamped one Gradle would otherwise pick. Verified locally: bundled unified-query-api drops 60kB → 29kB, the UnifiedQueryContext$Builder constant pool now references PPL_REX_MAX_MATCH_LIMIT, and RexCommandIT goes 0/16 → 16/16 against the same locally-published jars the CI workflow already produces. Drop this filter once the SQL feature branch merges to sql/main and the remote OpenSearch Snapshots repo catches up — at that point every 3.7.0.0-SNAPSHOT publish will carry the rex max-match default and the mavenLocal preference becomes redundant. Signed-off-by: Jialiang Liang <jiallian@amazon.com>
CI fallout from the prior commit's `excludeGroup 'org.opensearch.query'`
filter on the OpenSearch Snapshots remote: the parent subprojects block
no longer carried mavenLocal, so analytics-engine's testImplementation /
internalClusterTest configurations had no repository at all serving
org.opensearch.query, failing with `Could not find
org.opensearch.query:unified-query-api:3.6.0.0-SNAPSHOT` (and -core / -ppl).
Two pieces:
1. sandbox/build.gradle subprojects { repositories } — also declare
mavenLocal scoped to the org.opensearch.query group via mavenContent
{ includeGroup }. mavenLocal becomes the authoritative source for
unified-query SNAPSHOTs (populated by the sandbox-check workflow's
publishUnifiedQueryPublicationToMavenLocal pre-step) without leaking
into resolution for any other group.
2. sandbox/plugins/analytics-engine/build.gradle — bump
sqlUnifiedQueryVersion from 3.6.0.0-SNAPSHOT → 3.7.0.0-SNAPSHOT.
The 3.6 jars don't exist in mavenLocal (only the 3.7 feature-branch
build does), so the older pin was the proximate cause of the CI
resolution failure. Aligning with test-ppl-frontend's already-3.7
declaration also keeps the unified-query consumers consistent.
Signed-off-by: Jialiang Liang <jiallian@amazon.com>
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit 84bfec6.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
|
Confirmed the above change fixed the issue - here is a green run example on rex PR: cae2cb0 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #21578 +/- ##
============================================
+ Coverage 73.48% 73.52% +0.03%
+ Complexity 74646 74620 -26
============================================
Files 5980 5980
Lines 338777 338777
Branches 48848 48848
============================================
+ Hits 248964 249097 +133
+ Misses 70026 69788 -238
- Partials 19787 19892 +105 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
These dependencies (unified-query-api/core/ppl) were added in opensearch-project#21578 and then reverted in opensearch-project#21580. They were incorrectly re-introduced during conflict resolution. Signed-off-by: bowenlan-amzn <bowenlan23@gmail.com> Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Description
Follow-up to #21569. The sandbox-check workflow's pre-step
publishUnifiedQueryPublicationToMavenLocalpublishes the SQL feature branch'sunified-query-*jars to mavenLocal so analytics-engine sandbox plugins build against the latest API surface. The publish itself works, but Gradle's default SNAPSHOT resolution still prefers the stale remote — every test that exercises the unified-query path picks up jars built fromsql/main, not the feature branch.Confirmed via
dependencyInsightWhy mavenLocal loses
Comparing
maven-metadata.xmlfrom each source forunified-query-api:3.7.0.0-SNAPSHOT:<lastUpdated><buildNumber>20260508200435(newer)<localCopy>true</localCopy>3.7.0.0-SNAPSHOT20260507224009(older)123.7.0.0-20260507.224009-12Gradle's resolver weighs the remote's explicit
<buildNumber>+ timestamp metadata higher than mavenLocal's bare<localCopy>true>, so the older remote wins even when mavenLocal is newer. End-to-end consequence:test-ppl-frontendbundles the stale 60kBunified-query-api-3.7.0.0-SNAPSHOT.jar(42 classes) instead of the freshly-published 29kB feature-branch jar (21 classes), and the runtime cluster ends up running the olderUnifiedQueryContext$Builderthat's missingPPL_REX_MAX_MATCH_LIMIT,CALCITE_ENGINE_ENABLED, and any other field the feature branch has added butsql/mainhasn't received yet.This causes
RexCommandITand any other IT exercising rex max-match (or anything else introduced post-sql/main) to fail withCannot invoke 'java.lang.Integer.intValue()' because Settings.getSettingValue(...) is nullat plan time, even though the workflow publish step succeeds.Fix
Tell Gradle's repository content filters that
org.opensearch.queryis owned by mavenLocal:sandbox/build.gradlesubprojects { repositories }— addmavenLocal { mavenContent { includeGroup 'org.opensearch.query' } }and addmavenContent { excludeGroup 'org.opensearch.query' }to the existing OpenSearch Snapshots remote. Every sandbox subproject now resolves the unified-query group exclusively from mavenLocal; everything else still goes to the remote unchanged.sandbox/plugins/analytics-backend-datafusion/build.gradle— sameexcludeGroupfilter on its own remote declaration (the file-local block isn't covered by the parentsubprojectsblock, which only adds repos rather than replacing them).sandbox/plugins/test-ppl-frontend/build.gradle— sameincludeGroup/excludeGrouppair locally sobundlePluginactually picks the mavenLocal jar at bundle time.sandbox/plugins/analytics-engine/build.gradle— bumpsqlUnifiedQueryVersionfrom3.6.0.0-SNAPSHOT→3.7.0.0-SNAPSHOTto matchtest-ppl-frontend. The 3.6 line was stale; with the new filter the remote no longer serves it, and the older pin causedCould not find org.opensearch.query:unified-query-api:3.6.0.0-SNAPSHOTat internal-cluster-test resolution.Verification
./gradlew :sandbox:plugins:test-ppl-frontend:bundlePlugin --refresh-dependencies— bundledunified-query-api-3.7.0.0-SNAPSHOT.jardrops 60kB → 29kB and the constant pool now referencesPPL_REX_MAX_MATCH_LIMIT../gradlew :sandbox:qa:analytics-engine-rest:integTest --tests RexCommandIT— 0/16 → 16/16../gradlew :sandbox:plugins:analytics-engine:compileJava :sandbox:plugins:analytics-engine:compileTestJava— green (the 3.7 bump resolves cleanly through mavenLocal now).Drop conditions
This is a temporary workaround mirroring the temporary nature of #21569. Drop both filters once the SQL feature branch merges to
sql/mainand the OpenSearch Snapshots remote starts republishing the unified-query group. At that point Gradle resolution will be correct without any content-filter intervention.