fix(workflow-operator): re-validate redirect hops in HF remote-URL fetches to close an SSRF bypass

[PG1204]

What changes were proposed in this PR?

Closes an SSRF bypass in the HuggingFace inference operator's _fetch_remote_url (generated Python in PythonCodegenBase.scala).

The helper hardens remote fetches (https-only, rejects private/loopback/link-local/ reserved addresses incl. the 169.254.169.254 metadata endpoint, size cap) but validated only the original URL, then called requests.get(...), which follows redirects by default. Redirects were never re-checked, so a 302 to http://169.254.169.254/... or an internal host was fetched without re-running the scheme/address checks. Every remote fetch in the operator routes through this helper, user-provided image/audio URLs and provider-returned media URLs, so a malicious input or hostile provider response could
reach internal services and exfiltrate the content via the result column.

The fix follows redirects manually so every hop gets the same scrutiny as the original:

• Extracted the scheme + address checks into a _validate_remote_url helper so it can run per hop.
• _fetch_remote_url sets allow_redirects=False and walks the chain in a bounded loop (MAX_REDIRECT_HOPS = 5), validating before each request; relative Location values are resolved via urljoin and re-validated.
• Fails closed on a missing Location or an over-long chain; intermediate responses are closed. Size cap and raise_for_status unchanged.
• Hardened the address check to an allowlist stance: it now also requires a globally-routable address (not ip.is_global) alongside the existing predicates. This additionally blocks the CGNAT/shared range (100.64.0.0/10) the predicate list missed and stays correct across Python versions, while keeping the explicit predicates (e.g. multicast, which CPython reports as global).

Net effect: a redirect can no longer downgrade the scheme or point the worker at a non-public address; legitimate https→https (and relative) redirects still work.

Any related issues, documentation, discussions?

Closes #6967

How was this PR tested?

Added a generated-code test to HuggingFaceInferenceOpDescSpec pinning the fix:
allow_redirects=False, validation running before each request, interception of all redirect statuses (301/302/303/307/308), relative-Location resolution, the hop cap, the fail-closed errors, and the globally-routable-only address check.

sbt "WorkflowOperator/testOnly org.apache.texera.amber.operator.huggingFace.*"

Full HF package passes (123 tests), including PythonCodeRawInvalidTextSpec, which py_compiles the generated Python of all 117 operators. scalafmt clean.

Beyond the source-level assertions, the redirect logic was manually verified by executing the generated operator Python against simulated redirect scenarios (no network): http-downgrade, private/RFC1918, 169.254.169.254 metadata, CGNAT, multicast, IPv6 loopback/ULA, IPv4-mapped, userinfo-trick, and mixed public+private hosts are each blocked and never fetched, while legitimate https→https and relative-Location redirects still succeed and the hop cap is enforced.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Fable 5 in compliance with ASF.

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @anishshiva7
  You can notify them by mentioning @anishshiva7 in a comment.

[github-actions]

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 7 worse · ⚪ 6 noise (<±5%) · 0 without baseline

Compared against main 469e8f0 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	471	0.287	20,786/25,471/25,471 us	🟢 -10.7% / 🔴 +69.6%
🔴	bs=100 sw=10 sl=64	1,152	0.703	84,565/119,313/119,313 us	🔴 +13.7% / 🟢 -16.0%
⚪	bs=1000 sw=10 sl=64	1,380	0.842	727,042/759,350/759,350 us	⚪ within ±5% / 🟢 +33.9%

Baseline details

Latest main 469e8f0 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	471 tuples/sec	524 tuples/sec	787.55 tuples/sec	-10.1%	-40.2%
bs=10 sw=10 sl=64	MB/s	0.287 MB/s	0.32 MB/s	0.481 MB/s	-10.3%	-40.3%
bs=10 sw=10 sl=64	p50	20,786 us	19,051 us	12,255 us	+9.1%	+69.6%
bs=10 sw=10 sl=64	p95	25,471 us	28,521 us	15,802 us	-10.7%	+61.2%
bs=10 sw=10 sl=64	p99	25,471 us	28,521 us	19,008 us	-10.7%	+34.0%
bs=100 sw=10 sl=64	throughput	1,152 tuples/sec	1,217 tuples/sec	997.81 tuples/sec	-5.3%	+15.5%
bs=100 sw=10 sl=64	MB/s	0.703 MB/s	0.743 MB/s	0.609 MB/s	-5.4%	+15.4%
bs=100 sw=10 sl=64	p50	84,565 us	81,614 us	100,690 us	+3.6%	-16.0%
bs=100 sw=10 sl=64	p95	119,313 us	104,917 us	107,316 us	+13.7%	+11.2%
bs=100 sw=10 sl=64	p99	119,313 us	104,917 us	113,823 us	+13.7%	+4.8%
bs=1000 sw=10 sl=64	throughput	1,380 tuples/sec	1,383 tuples/sec	1,030 tuples/sec	-0.2%	+33.9%
bs=1000 sw=10 sl=64	MB/s	0.842 MB/s	0.844 MB/s	0.629 MB/s	-0.2%	+33.9%
bs=1000 sw=10 sl=64	p50	727,042 us	717,917 us	981,213 us	+1.3%	-25.9%
bs=1000 sw=10 sl=64	p95	759,350 us	761,949 us	1,027,605 us	-0.3%	-26.1%
bs=1000 sw=10 sl=64	p99	759,350 us	761,949 us	1,055,466 us	-0.3%	-28.1%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,424.84,200,128000,471,0.287,20786.06,25470.71,25470.71
1,100,10,64,20,1736.41,2000,1280000,1152,0.703,84565.07,119312.82,119312.82
2,1000,10,64,20,14495.96,20000,12800000,1380,0.842,727042.49,759350.20,759350.20

[PG1204]

/request-review @xuang7