Skip to content

Unified Query Translator for PANGAEA + NOAA#36

Merged
khider merged 6 commits intomainfrom
query-translater
Apr 6, 2026
Merged

Unified Query Translator for PANGAEA + NOAA#36
khider merged 6 commits intomainfrom
query-translater

Conversation

@doswal
Copy link
Copy Markdown
Collaborator

@doswal doswal commented Apr 2, 2026

Summary

This PR introduces a unified query translation layer for PANGAEA, aligning it with NOAA’s structured interface in pyleotups. It standardizes parameter naming, enables multi-value logical operations, and improves documentation consistency across datasets.

Key Features & Improvements

1. Unified Query Interface

Replaces backend-specific parameters with a consistent API across providers

Introduces:

  • variable_name mapped to: NOAA → cvWhats, PANGAEA → parameter,
  • investigators mappped to: PANGAEA → author

Removes reliance on raw q input in favor of structured parameters

2. PANGAEA Query Builder

Adds build_pangaea_query() to translate unified kwargs into PANGAEA-compatible query strings

Supports:

  • search_text
  • investigators → author:
  • variable_name → parameter:
  • geographic bounds → bbox

Ensures safe query construction and prevents malformed queries

3. Logical Operators for Multi-Value Parameters

Adds *_and_or support (aligned with NOAA design):

variable_name_and_or = "and" / "or"
investigators_and_or = "and" / "or"
topic_and_or = "and" / "or"

Implements logic in query builder:
AND → implicit (space-separated)
OR → explicit ((A OR B))

4. Topic Filtering (PANGAEA-specific)

Introduces new parameter:

topic to reduce search space based on requirements

Supports:

  • Multiple topics with AND/OR logic
  • Adds validation against predefined topic set
  • Invalid topics are skipped with warnings

6. Improved Docstrings (PANGAEA)

Upgrades PangaeaDataset.search_studies() documentation to match NOAA standards

  • Detailed parameter descriptions
  • Notes on query translation
  • Multiple usage examples (jupyter-execute)
  • Improves clarity for both users and developers

7. Test Updates

  • Updates PANGAEA tests to align with new unified interface
  • Replaces deprecated q usage with search_text
  • Ensures compatibility with new query builder logic

Design Philosophy

This PR moves pyleotups toward a backend-agnostic query interface, where:

  • Users interact with a consistent API
  • Backend-specific logic is handled internally via query builders
  • Differences between NOAA (structured API) and PANGAEA (text-based search) are abstracted away

⚠️ Known Limitations

  • PANGAEA search remains text-based and less structured than NOAA
  • Temporal (age-based) filtering is not supported natively in PANGAEA
  • Topic filtering depends on metadata completeness and may not always be exhaustive

doswal added 6 commits April 1, 2026 10:46
…d adding necessary TUPS to Pangaea translator/query builder
…ame; map to cvWhats (NOAA) and parameter (PANGAEA); update query builders and validation logic; maintain backward compatibility for cv_whats
…terms, notes and examples

Correcting typos and imporving docstrings clarity
@khider khider merged commit 6ade0b9 into main Apr 6, 2026
1 check passed
@khider khider deleted the query-translater branch April 6, 2026 18:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants