Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions .github/workflows/pyliny.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
name: Code Quality

on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main, develop ]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Python 3.10
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pre-commit

- name: Cache pre-commit environment
uses: actions/cache@v3
with:
path: ~/.cache/pre-commit
key: pre-commit-${{ runner.os }}-${{ hashFiles('.pre-commit-config.yaml') }}
restore-keys: |
pre-commit-${{ runner.os }}-

- name: Run pre-commit hooks
run: |
pre-commit run --all-files --show-diff-on-failure
97 changes: 97 additions & 0 deletions .github/workflows/pypi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: Build and publish python package

on:
push:
tags:
- "v*"
workflow_dispatch:
inputs:
version:
description: 'Version to publish (e.g., 0.1.0)'
required: false
type: string

jobs:
publish:
runs-on: ubuntu-latest
permissions:
contents: write # Required for creating GitHub releases
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # Fetch all history for tags

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.10"

- name: Extract version from tag or input
id: get_version
run: |
if [ "${{ github.event_name }}" == "workflow_dispatch" ] && [ -n "${{ github.event.inputs.version }}" ]; then
echo "version=${{ github.event.inputs.version }}" >> $GITHUB_OUTPUT
elif [ -n "${{ github.ref_name }}" ]; then
# Extract version from tag (e.g., v0.1.0 -> 0.1.0)
VERSION=$(echo "${{ github.ref_name }}" | sed 's/^v//')
echo "version=$VERSION" >> $GITHUB_OUTPUT
else
# Fallback: extract from pyproject.toml
VERSION=$(grep -E '^version = ' pyproject.toml | sed 's/version = "\(.*\)"/\1/')
echo "version=$VERSION" >> $GITHUB_OUTPUT
fi
echo "Publishing version: $(cat $GITHUB_OUTPUT | grep version | cut -d'=' -f2)"

- name: Verify version matches pyproject.toml
run: |
PUBLISH_VERSION="${{ steps.get_version.outputs.version }}"
FILE_VERSION=$(grep -E '^version = ' pyproject.toml | sed 's/version = "\(.*\)"/\1/')
if [ "$PUBLISH_VERSION" != "$FILE_VERSION" ]; then
echo "Error: Version mismatch!"
echo "Tag/Input version: $PUBLISH_VERSION"
echo "pyproject.toml version: $FILE_VERSION"
exit 1
fi
echo "✓ Version verified: $PUBLISH_VERSION"

- name: Install build tools
run: |
python -m pip install --upgrade pip
pip install build twine

- name: Build package
run: python -m build

- name: Check package contents
run: |
echo "Built files:"
ls -lh dist/
echo ""
echo "Package info:"
python -m twine check dist/*

- name: Publish to PyPI
env:
TWINE_USERNAME: __token__
TWINE_PASSWORD: ${{ secrets.PYPI_TOKEN }}
run: twine upload dist/*

- name: Create GitHub Release
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v1
with:
tag_name: ${{ github.ref_name }}
name: Release ${{ steps.get_version.outputs.version }}
body: |
## Release ${{ steps.get_version.outputs.version }}

Published to PyPI: https://pypi.org/project/spherical-deepkriging/${{ steps.get_version.outputs.version }}/

### Installation
```bash
pip install spherical-deepkriging==${{ steps.get_version.outputs.version }}
```
draft: false
prerelease: false
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
41 changes: 41 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Test

on:
push:
branches: ["main", "develop"]
pull_request:
branches: ["main", "develop"]

jobs:
test:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -e ".[dev,mrts]"
- name: Test with pytest
run: |
pytest tests/ \
-m "not slow and not integration" \
--cov=spherical_deepkriging \
--cov-config=.coveragerc \
--cov-fail-under=90 \
--cov-report=xml \
--cov-report=term \
-v --tb=short
env:
PYTHONPATH: ${{ github.workspace }}
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -152,3 +152,6 @@ dmypy.json

# Cython debug symbols
cython_debug/

# Coverage configuration
.coveragerc
Empty file added =0.4.0,
Empty file.
1 change: 0 additions & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,3 @@ global-exclude *.so *.pyd *.dylib *.dll *.a *.lib

# Include C++ sources for the spherical basis extension in sdist builds.
recursive-include spherical_deepkriging/basis_functions/mrts_sphere/cpp_extensions *.cpp *.h *.hpp *.txt *.py CMakeLists.txt

1 change: 0 additions & 1 deletion MIGRATION_NOTE.txt

This file was deleted.

23 changes: 16 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
SHELL := /bin/bash
EXECUTABLE := poetry run
CONDA_BIN ?= /home/egpivo/miniconda3/bin/conda
TEST_ENV ?= spherical-deepkriging

.PHONY: clean install-dev setup-jupyter-kernel build-spherical-cpp test run-local-jupyter help
.PHONY: clean install-dev install-test-deps setup-jupyter-kernel build-spherical-cpp test run-local-jupyter help

## Clean up temporary files
clean:
Expand All @@ -17,13 +18,19 @@ clean:
## Install development dependencies
install-dev:
@echo "Installing development dependencies..."
@$(SHELL) envs/conda/build_conda_env.sh -c spherical-deepkriging && \
@$(SHELL) envs/conda/build_conda_env.sh -c $(TEST_ENV) && \
$(MAKE) install-test-deps && \
$(MAKE) setup-jupyter-kernel

## Install test dependencies in the Conda environment
install-test-deps:
@echo "Installing test dependencies into $(TEST_ENV)..."
@$(CONDA_BIN) run -n $(TEST_ENV) python -m pip install -e ".[dev]"

## Install Jupyter dependencies and register kernel
setup-jupyter-kernel:
@echo "Setting up Jupyter kernel..."
@$(SHELL) envs/jupyter/setup_jupyter_kernel.sh -k spherical-deepkriging
@$(SHELL) envs/jupyter/setup_jupyter_kernel.sh -k $(TEST_ENV)

## Build spherical basis C++ extension (mrts_sphere)
build-spherical-cpp:
Expand All @@ -37,7 +44,8 @@ build-spherical-cpp:
## Run tests
test:
@echo "Running tests..."
@$(EXECUTABLE) pytest --cov=spherical_deepkriging
@$(CONDA_BIN) run -n $(TEST_ENV) python -c "import pytest, pytest_cov" >/dev/null 2>&1 || (echo "Missing test dependencies in Conda env $(TEST_ENV). Run: make install-test-deps" && exit 1)
@PYTHONPATH=. $(CONDA_BIN) run -n $(TEST_ENV) python -m pytest --cov=spherical_deepkriging --cov-config=.coveragerc

## Start Jupyter server locally
run-local-jupyter:
Expand All @@ -49,8 +57,9 @@ help:
@echo "Available targets:"
@echo " clean : Clean up temporary files"
@echo " install-dev : Install dependencies, build package/C++ extension, and setup Jupyter kernel"
@echo " install-test-deps : Install pytest/coverage tooling into the Conda env"
@echo " setup-jupyter-kernel : Install Jupyter deps and register kernel for spherical-deepkriging"
@echo " build-spherical-cpp : Build spherical basis C++ extension"
@echo " build-spherical-cpp : Build spherical basis C++ extension"
@echo " test : Run tests"
@echo " run-local-jupyter : Start Jupyter server locally"
@echo " help : Display this help message"
@echo " help : Display this help message"
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Spherical DeepKriging
[![Tests](https://github.com/STLABTW/spherical-deepkriging/workflows/Test/badge.svg)](https://github.com/STLABTW/spherical-deepkriging/actions)
[![codecov](https://codecov.io/github/STLABTW/spherical-deepkriging/graph/badge.svg?token=OF0LKVDII6)](https://codecov.io/github/STLABTW/spherical-deepkriging)

A deep learning framework for spatial prediction on the sphere, combining DeepKriging with spherical harmonic basis functions (MRTS-sphere), Wendland basis, and Universal Kriging.

Expand All @@ -20,7 +22,7 @@ The package currently provides the following basis families:

The default feedforward hidden-block design is illustrated below:

![DeepKriging hidden block](artifacts/hidden_block.png)
<img src="artifacts/hidden_block.png" alt="DeepKriging hidden block" width="60%" />

---

Expand Down
5 changes: 3 additions & 2 deletions envs/jupyter/utils.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@ update_conda_env_path() {

check_kernel_availability() {
local KERNEL_NAME=$1
local KERNEL_PATH=$(jupyter kernelspec list | grep -o "^${KERNEL_NAME} .*" | cut -d' ' -f2)
local KERNEL_PATH
KERNEL_PATH=$(jupyter kernelspec list 2>/dev/null | awk -v k="${KERNEL_NAME}" '$1==k {print $2; exit}')

if [ -z "${KERNEL_PATH}" ]; then
echo -e "${FG_RED}Kernel '${KERNEL_NAME}' is not available${FG_RESET}"
Expand All @@ -48,7 +49,7 @@ set_jupyter_kernel_path() {
ipython kernel install --name "${KERNEL_NAME}" --user
fi

KERNEL_PATH=$(jupyter kernelspec list | grep -o "^${KERNEL_NAME} .*" | cut -d' ' -f2)
KERNEL_PATH=$(jupyter kernelspec list 2>/dev/null | awk -v k="${KERNEL_NAME}" '$1==k {print $2; exit}')

if [ -n "${KERNEL_PATH}" ]; then
KERNEL_DIR="${KERNEL_PATH}"
Expand Down
65 changes: 60 additions & 5 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,67 @@ conda activate spherical-deepkriging
cd examples/simulation
```

Main scripts:

- `run_stdscaler_nonoise_50reps.py`
- `run_stdscaler_outliers_50reps.py`
### Reproducing Simulation Experiment

These scripts checkpoint progress and can be resumed safely.
The `examples/simulation/` folder contains self-contained Python scripts and output notebooks for reproducing the 50-repeat simulation study used in the paper. Each script runs a full experiment, checkpoints after every repeat, and can be safely interrupted and resumed.

Pre-executed output notebooks are also included in `examples/simulation/` for result inspection.
#### Experiment scripts

| Script | Scenario | Output checkpoint |
|--------|----------|-------------------|
| `run_stdscaler_nonoise_50reps.py` | Local plus nonstationary signal, no noise | `results_sphere_stdscaler_nonoise_50reps_checkpoint.csv` |
| `run_stdscaler_outliers_50reps.py` | Local plus nonstationary signal with outliers | `results_sphere_stdscaler_outliers_50reps_checkpoint.csv` |

Each script benchmarks the following models over 50 random seeds:

- `OLS_wendland` — Ordinary Least Squares with Wendland basis
- `OLS_sphere` — Ordinary Least Squares with spherical harmonic (MRTS-sphere) basis
- `DeepKriging_wendland` — DeepKriging with Wendland basis
- `DeepKriging_mrts` — DeepKriging with MRTS basis
- `DeepKriging_sphere` — DeepKriging with MRTS-sphere basis (MSE loss)
- `DeepKriging_sphere_Huber` — DeepKriging with MRTS-sphere basis (Huber loss, robust to outliers)
- `UniversalKriging` — Universal Kriging with MRTS-sphere basis

#### How to run

Make sure the Conda environment is activated, then navigate to the Rerun folder and run:

```bash
conda activate deepkriging
cd notebook/simulation/Rerun

# No-noise scenario
python run_stdscaler_nonoise_50reps.py

# Outliers scenario
python run_stdscaler_outliers_50reps.py
```

Each script:
- Runs 50 independent repeats with different random seeds.
- Saves a checkpoint CSV after every completed repeat — safe to kill (`Ctrl+C`) and re-run; already-completed repeats are skipped automatically.
- Prints a per-repeat results table (MSPE, RMSE, MAE, R², Time) to stdout.
- Prints a final mean±std summary table across all 50 repeats at the end.

#### Key hyperparameters (configured inside each script)

| Parameter | Value |
|-----------|-------|
| Training samples | 2500 |
| Repeats | 50 |
| Huber delta | 1.345 |
| Knot / order budget | 1400 |
| Batch size | see script |

#### Output notebooks

Pre-executed output notebooks are included in `examples/simulation/Rerun/` for reference:

| Notebook | Description |
|----------|-------------|
| `simulation_spherical_nn_sim_RSHC_c15_GP_pure_50reps_out.ipynb` | GP signal, no noise |
| `simulation_spherical_nn_sim_RSHC_c15_GP_noise_50reps_out.ipynb` | GP signal with noise |
| `simulation_spherical_nn_sim_RSHC_c15_GP_outliers_50reps_out.ipynb` | GP signal with outliers |
| `simulation_spherical_nn_sim_RSHC_c15_noGP_pure_50reps_out.ipynb` | Local extreme signal, no noise |
| `simulation_spherical_nn_sim_RSHC_c15_noise_noGP_50reps_out.ipynb` | Local extreme signal with noise |s
Loading
Loading