Open
Conversation
- React frontend with MUI components (Header, Sidebar, Footer, RightToolbar) - 18 QuickLook pages, Pulsar, Modeling, Simulator, Utilities sections - FastAPI backend with Stingray service layer - Integrated log console, system resources, notifications - Python backend lifecycle management via Electron IPC - Pixi environment and dev scripts
- Add complete EventList load/save/delete/info API endpoints - Extend DataService with URL loading, file size checks, memory cleanup - Add NotificationToast component for global alerts - Update DataIngestion page with advanced options and event list management - Add sample data documentation (DATA_README.txt) - Update gitignore for large NICER dataset
- Add GTI validation checking for empty, invalid, and out-of-bounds events - Fix temp file cleanup in URL loading with try/finally - Fix FITS writing to use Astropy Table for OGIP compliance - Fix directory creation edge case for paths without directories - Display GTI warnings in UI with tooltip
- Move resource monitor, backend status, about dialog to RightToolbar - Merge help/contact/support into single Help menu - Add backend status dot indicator on processing icon - Remove Settings button (already in Header) - Delete Footer.tsx - no longer needed - Fix RightToolbar height to account for LogPanel
- Fix CPU percent always showing ~0 by adding interval=0.1 to psutil - Filter /api/status from uvicorn access logs to prevent log flooding - Increase polling frequency from 5s to 2s for faster updates - Remove non-functional CPU/RAM limit sliders from resources popover - Add Electron process resource monitoring via IPC - Enhance data service with better GTI validation
- Add /load-batch-stream SSE endpoint for streaming batch load results - Use asyncio.wrap_future + asyncio.wait for non-blocking async iteration - Files now appear in UI immediately as each one completes loading - Add console logging for SSE events, GTI warnings, and Stingray warnings - Capture Stingray library warnings during file loading - Update progress bar to show determinate progress (X of Y files) - Normalize CPU percentages by core count in resource monitoring
- Add LogStreamManager to capture Python logging and warnings - Add /api/logs/stream SSE endpoint for real-time log delivery - Add LogStreamClient in frontend with auto-reconnect logic - Connect LogPanel to SSE stream when backend is ready - Include /api/logs/test endpoint for development testing
- Add localStorage persistence for last loaded files with restore button - Browse now appends files instead of replacing selection - Add "Clear All" button for file selection - Selection clears after load attempt (success or failure) - Add GTI validation and inverted GTI fix in backend - Add comprehensive data quality validation checks - Add save format dialog and event notes support
- Add archive service for querying HEASARC data archive - Add archive routes and API client for archive operations - Add FileBrowserDialog component for local file browsing - Add HeasarcBrowserPanel for browsing remote archive data - Update data service with additional validation and features - Add pyvo dependency for Virtual Observatory access - Update pixi.lock with new dependencies
Wrap synchronous service calls with asyncio.to_thread() in data routes to prevent blocking the FastAPI event loop during file operations. This allows resource monitoring and other async tasks to continue while files are being loaded, saved, or processed. Affected endpoints: - POST /api/data/load - POST /api/data/load-url - POST /api/data/save - POST /api/data/load-by-time-range - POST /api/data/load-by-event-count - POST /api/data/metadata
There was a problem hiding this comment.
Pull request overview
Migrates the project to an Electron + React (electron-vite) desktop app architecture backed by a local FastAPI Python service, adding a new UI shell, API clients, and backend endpoints.
Changes:
- Added Electron main/preload processes with IPC plumbing and a Python backend lifecycle manager.
- Added a new React app structure (router, layout, placeholder pages), Zustand stores, and frontend API modules.
- Added a FastAPI backend with state/services, log streaming (SSE), archive/download helpers, and dev/build scripts/config.
Reviewed changes
Copilot reviewed 94 out of 103 changed files in this pull request and generated 31 comments.
Show a summary per file
| File | Description |
|---|---|
| tsconfig.node.json | Adds TS config for node/build tooling references. |
| tsconfig.json | Adds strict TS config, path aliases, and project references for renderer/electron code. |
| test_astropy_roundtrip.py | Removes an ad-hoc Astropy roundtrip test script. |
| src/types/electron.d.ts | Adds renderer typings for the preload-exposed Electron API. |
| src/store/uiStore.ts | Introduces global UI state store (panels, processing state, notifications, resources). |
| src/store/logStore.ts | Adds centralized log store + filtering and external log injection helper. |
| src/pages/Utilities/StatisticalFunctions/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Utilities/MissionIO/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Utilities/Misc/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Utilities/IO/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Utilities/GTI/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Simulator/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/VariableEnergySpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/RmsEnergySpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/PowerSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/LightCurve/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/LagEnergySpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/ExcessVarianceSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/EventList/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/DynamicalPowerSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/DeadTimeCorrections/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/CrossSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/CrossCorrelation/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/CovarianceSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/Coherence/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/Bispectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/AvgPowerSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/AvgCrossSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/AvgCovarianceSpectrum/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/QuickLook/AutoCorrelation/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Pulsar/Phaseogram/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Pulsar/PhaseFolding/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Pulsar/PeriodSearch/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/NotFound/index.tsx | Adds a 404 page for unknown routes. |
| src/pages/Modeling/ModelBuilder/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Modeling/MLEFitting/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Modeling/MCMCFitting/index.tsx | Adds placeholder “coming soon” page. |
| src/pages/Home/index.tsx | Adds a new home page with quick-access navigation cards. |
| src/main.tsx | Adds React entrypoint mounting <App />. |
| src/index.css | Adds global styles (scrollbars, animations, plot container sizing, titlebar regions). |
| src/components/layout/MainLayout.tsx | Adds app shell layout (header/sidebar/right toolbar/log panel/notifications). |
| src/components/layout/Header.tsx | Adds header with backend status, search UI, and controls. |
| src/components/common/PageTemplate.tsx | Adds reusable placeholder/“coming soon” template. |
| src/components/common/NotificationToast.tsx | Adds toast notifications driven by UI store. |
| src/api/timingApi.ts | Adds frontend client functions for timing endpoints. |
| src/api/spectrumApi.ts | Adds frontend client functions for spectrum endpoints. |
| src/api/logApi.ts | Adds SSE log stream client that pushes backend logs into log store. |
| src/api/lightcurveApi.ts | Adds frontend client functions for lightcurve endpoints. |
| src/api/index.ts | Re-exports API modules/types from a central barrel. |
| src/api/exportApi.ts | Adds frontend client functions for export endpoints. |
| src/api/client.ts | Adds fetch-based API client with Electron-backed dynamic port discovery. |
| src/api/archiveApi.ts | Adds frontend client for HEASARC search + SSE download streaming. |
| src/App.tsx | Adds router + theme + backend readiness context and wiring. |
| scripts/start-backend.sh | Adds helper script to start backend via Pixi. |
| scripts/setup-python.sh | Adds helper script to create venv and install backend requirements. |
| scripts/package.sh | Adds packaging helper for bundling backend resources into dist. |
| scripts/dev.sh | Adds unified dev launcher (backend + electron-vite dev). |
| scripts/build.sh | Adds build helper for Electron app build. |
| python-backend/utils/performance_monitor.py | Adds backend performance monitoring utilities. |
| python-backend/utils/log_stream.py | Adds Python logging/warnings capture and SSE streaming support. |
| python-backend/utils/error_handler.py | Adds standardized error handling/formatting utilities. |
| python-backend/utils/init.py | Exposes backend utility classes. |
| python-backend/services/timing_service.py | Adds timing operations (bispectrum, power colors, lags, coherence). |
| python-backend/services/state_manager.py | Adds backend in-memory state manager with memory cleanup helpers. |
| python-backend/services/spectrum_service.py | Adds spectrum operations (power/cross/dynamical spectra, rebin, list/delete). |
| python-backend/services/lightcurve_service.py | Adds lightcurve operations (create/rebin/list/delete). |
| python-backend/services/export_service.py | Adds export operations (CSV/HDF5/FITS + bispectrum CSV). |
| python-backend/services/base_service.py | Adds shared service base class (result shaping + error handling). |
| python-backend/services/init.py | Exports service classes for backend use. |
| python-backend/routes/timing_routes.py | Adds FastAPI routes for timing endpoints. |
| python-backend/routes/spectrum_routes.py | Adds FastAPI routes for spectrum endpoints. |
| python-backend/routes/log_routes.py | Adds FastAPI SSE routes for backend log streaming. |
| python-backend/routes/lightcurve_routes.py | Adds FastAPI routes for lightcurve endpoints. |
| python-backend/routes/export_routes.py | Adds FastAPI routes for export endpoints. |
| python-backend/routes/archive_routes.py | Adds FastAPI routes for HEASARC search + download-to-disk streaming. |
| python-backend/routes/init.py | Exposes route modules. |
| python-backend/requirements.txt | Adds pip requirements for the new backend. |
| python-backend/main.py | Adds FastAPI app startup, CORS config, status/shutdown endpoints, and port selection. |
| pixi.toml | Adds Pixi project/tasks/dependencies for backend + dev workflow. |
| package.json | Updates Electron build/package scripts and frontend tooling dependencies. |
| index.html | Adds renderer HTML entry with CSP and app metadata. |
| files/data/DATA_README.txt | Adds documentation for sample data included in the repo. |
| electron/pythonManager.ts | Adds backend process management and health/port handling for Electron. |
| electron/preload.ts | Exposes a typed, constrained IPC API to the renderer via contextBridge. |
| electron/menu.ts | Adds native app menu with navigation hooks. |
| electron/main.ts | Adds Electron main process window creation + backend startup + log buffering. |
| electron/ipcHandlers.ts | Adds IPC handlers for dialogs, filesystem, python control, app info, resources, etc. |
| electron.vite.config.ts | Adds electron-vite build configuration for main/preload/renderer. |
| .gitignore | Updates ignores for Node/Electron/Python/Pixi build artifacts and large data files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…ndling - Add comprehensive job management infrastructure: - Job models with status, progress, and result tracking - JobManager service for background task execution - Job routes with SSE streaming for real-time progress - Frontend job store (Zustand) and useJobStream hook - JobStatusPanel component for UI display - jobApi for frontend-backend communication - Improve log streaming: - Clear log history on new Electron session to prevent stale log replay - Remove unnecessary SSE delay for faster log delivery - Simplify UI components: - Streamline DataIngestion page - Clean up Sidebar, MainLayout, and RightToolbar - Simplify dev.sh startup script
…and locate_data fixes - Add obs_data passthrough (prnb, ra/dec) for mission-specific directory lookup - Construct RXTE archive URLs using proposal number and AO cycle estimation - Fix locate_data to use coordinate-based queries and proper API workflow - Add date/time column sorting and MJD-to-human-readable conversion in browser - Improve exposure column detection for NuSTAR (exposure_a) and XMM (duration) - Add debug test scripts for HEASARC column, locate, and URL testing
Pass port as positional arg instead of --port flag to match main.py's sys.argv[1] parsing. Scope pkill pattern to project-specific path to avoid killing unrelated python processes.
…rowser Add 5 new X-ray missions with catalog entries, FTP directory URL patterns, browse URLs, and mission-specific exposure handling (IXPE per-detector-unit, ASCA sequence_number obsid column).
- Fix broken _locate_observation_directory fallback that called query_mission_list with invalid params (always crashed with TypeError), replaced with ADQL query via query_tap - Remove dead query_mission_list call in get_observation_download_urls - Robustify file classification by stripping .gz/.bz2 compression extensions before pattern matching - Add search_by_obsid using ADQL TAP query (no coordinates needed) - Add post-query min_exposure and time_range filters to name/coordinate searches - Extract NICER processing_status and num_fpm from catalog results - Add ObsID search mode toggle, filter panel (min exposure, date range), and configurable max results to frontend - Show NICER processing_status as colored chip with tooltip explanation - Move resolved coordinates message to notification center
…nd Mode tooltips Only show FPMA instrument label on exposure chip when FPMB data is available (ADQL/ObsID search). For name/coordinate searches where query_region() doesn't return exposure_b, show plain exposure to avoid misleading users. Add explanatory tooltips to Mode column chips (SCIENCE vs SLEW). Include NuSTAR-specific UI enhancements: issue flag warnings, SLEW row dimming, and observation mode column.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.