Skip to content

feat: Add Grid Snapper utility for coordinate alignment#116

Open
dhruv-surti wants to merge 1 commit into
mainfrom
feature/grid-snapper
Open

feat: Add Grid Snapper utility for coordinate alignment#116
dhruv-surti wants to merge 1 commit into
mainfrom
feature/grid-snapper

Conversation

@dhruv-surti
Copy link
Copy Markdown
Collaborator

@dhruv-surti dhruv-surti commented May 5, 2026

Grid Snapper Utility

Description

This PR adds the Grid Snapper utility to the data-engineering module. Grid Snapper is a core tool that converts raw GPS coordinates from different data sources (NASA FIRMS, Open-Meteo, ELVIS) into unified location_ids, enabling perfect data alignment across all observation tables.

What Problem Does This Solve

Different data sources use different coordinate systems:

  • NASA FIRMS: -37.8147, 145.0892
  • Open-Meteo: -37.8149, 145.0891
  • ELVIS: -37.81472, 145.08923

Without alignment, these coordinates cannot be joined in the database. Grid Snapper solves this by snapping all coordinates to a 0.1° grid and assigning a single location_id to each grid cell.

Changes Made

  • Added grid_snapper/grid_snapper.py - Core utility with 4-step alignment process
  • Added grid_snapper/README.md - Quick start guide
  • Added grid_snapper/__init__.py - Python package initialization
  • Added grid_snapper/docs/GRID_SNAPPER_TUTORIAL.md - Complete tutorial with examples
  • Added grid_snapper/screenshots/ - Step-by-step demo screenshots

How It Works

  1. Validate - Check if coordinate is inside Victoria bounds
  2. Snap - Round to nearest 0.1° grid (~11 km resolution)
  3. Lookup - Check if snapped coordinate exists in Location_Registry
  4. Get or Create - Return existing location_id or create new one

Usage

All extraction pipelines (fire, weather, vegetation, topography) should use Grid Snapper:

from grid_snapper import GridSnapper

snapper = GridSnapper()
location_id = snapper.get_location_id(-37.8147, 145.0892)

Testing

All tests passing:

  • Valid Victorian coordinate → location_id created
  • Same grid cell → same location_id returned
  • Outside Victoria → None (rejected)
  • Null coordinate → None (handled gracefully)

Impact

  • Enables fire + weather data to be joined on location_id + time_id
  • Provides foundation for AI model training
  • Standardizes coordinate alignment across all pipelines
  • Reduces data lineage issues

Checklist

  • Code follows architecture alignment rules
  • Data validation implemented
  • Security: No hardcoded API keys
  • No raw CSV/JSON/ZIP files committed
  • Documentation included
  • All tests passing
  • .gitignore protects sensitive data

Related

Part of FireFusion bushfire forecasting system - Deakin University Capstone

@wilsnd
Copy link
Copy Markdown
Collaborator

wilsnd commented May 10, 2026

Good for initial grid snapping, however improvements can be made in the future by reading the nearest value lat/long instead of just rounding it up. Good work

Copy link
Copy Markdown
Collaborator

@wilsnd wilsnd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work

Copy link
Copy Markdown
Collaborator

@HaNGUYEN-96 HaNGUYEN-96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work! This Gid Snapper tool is very helpful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants