Skip to content

pre-alpha#15

Merged
jairomelo merged 33 commits into
mainfrom
dev
May 26, 2026
Merged

pre-alpha#15
jairomelo merged 33 commits into
mainfrom
dev

Conversation

@jairomelo

Copy link
Copy Markdown
Member

This pull request introduces several significant enhancements and new features across the authentication, camera control, and database schema layers of the application. The most important changes include the addition of system logging, new endpoints and controls for camera devices, and several database migrations to support new features like project membership, extended project metadata, and improved record tracking.

Authentication and System Logging:

  • Added system event logging for key authentication actions such as user registration and login attempts, including both successful and failed logins. This provides an audit trail for access events and improves security monitoring. [1] [2]
  • Introduced a /setup/status endpoint to detect whether the system requires initial setup (i.e., no users exist yet), and updated the registration logic to restrict new user creation to admins after the first user is registered.

Camera API Enhancements:

  • Expanded the camera API with endpoints for previewing camera frames, flushing preview temp files, manual and live focus control, and arbitrary camera settings (exposure, white balance, zoom, etc.). These endpoints provide greater flexibility and control for camera operations from the frontend. [1] [2] [3] [4] [5]
  • Improved device information returned by the camera API to include calibration data, lens position, white balance gains, and camera capabilities such as aperture control and zoom support. [1] [2]
  • Enhanced the capture endpoint to allow linking records to a specific collection by collection_id, ensuring images are stored in the correct project/collection directory.

Database Schema Migrations:

System and Access Logging:

  • Added a new system_logs table to store logs of system events, supporting structured auditing of actions such as user creation and authentication events. [1] [2]

Project and Membership Management:

  • Introduced a project_members table to manage user membership and roles within projects, supporting collaborative workflows.

Project Metadata:

  • Added new columns (fondo, serie, signatura) to the projects table to support richer archival metadata.

Record Management:

  • Extended the records table with status, sequence, and rejection_note columns to better track record state and ordering.

Summary of Most Important Changes:

Authentication & System Logging

  • Added system event logging for user registration and login (success/failure), and restricted user registration to admins after initial setup. [1] [2] [3]
  • Added /setup/status endpoint to detect if system needs initial user setup.

Camera API Enhancements

  • Added endpoints for camera preview, flushing preview temp files, manual/lens focus controls, and arbitrary camera settings.
  • Improved device info to include calibration, lens position, AWB gains, and camera capabilities. [1] [2] [3]
  • Enhanced capture endpoint to support linking records to collections via collection_id. [1] [2] [3]

Database Migrations

  • Added system_logs table for structured event logging. [1] [2]
  • Added project_members table for project membership and roles.
  • Extended projects with fondo, serie, and signatura metadata fields.
  • Added status, sequence, and rejection_note fields to records table.

jairomelo added 30 commits May 19, 2026 19:57
…l device data

- picamera2_backend: add reset_camera() to evict broken cache entries,
  add apply_controls() for live parameter updates without full capture
- service: capture_preview_frame() retries once after reset_camera() on
  any backend error (fixes intermittent preview disconnect bug)
- cameras API: extend DeviceInfo with lens_position, awb_gains,
  has_aperture_control; add collection_id to CaptureRequest and
  DualCaptureRequest; populate calibration data from registry in
  /devices; pass collection_id when creating new Record rows
- Add GET/POST /cameras/focus/{camera_index} endpoints (FocusRequest/FocusResponse)
- Add get_focus() and set_focus() to capture/service.py
  - set_focus() clamps to 0–10 dioptres, switches to manual AF mode (AfMode=0)
  - get_focus() reads LensPosition from Picamera2 metadata
- Depends on apply_controls() from Phase 1 backend work
Records have a check constraint that prevents both project_id and
collection_id being set simultaneously. The capture endpoints were
looking up the project by name and setting project_id even when
collection_id was also provided, causing every capture from the
live-preview page to fail silently.

Fix: use project_id only when no collection_id is present; when a
collection is provided, its project association is implicit.
Images captured within a collection are now saved to:
  PROJECTS_ROOT/<project_name>/<collection_name>/images/main/

Previously all images for a project shared the same
PROJECTS_ROOT/<project_name>/images/main/ directory, making it
impossible to distinguish which collection each image belonged to
at the filesystem level.

- capture_image(), single_capture_image(), dual_capture_image() in
  service.py accept a new optional collection_name parameter; when
  set the path includes the sanitized collection name as a subdir
- Both capture API endpoints now look up the collection name from the
  DB and pass it down when collection_id is provided
- project_init() no longer pre-creates images/main|temp|trash dirs;
  capture_image() already creates these on demand per collection
Previously only the DB row was removed, leaving the image tree on disk.

- delete_project(): after commit, removes PROJECTS_ROOT/<name>/ with
  shutil.rmtree; tries both the raw project name and the sanitized
  (secure_project_filename) variant to handle historic inconsistencies
- delete_collection(): same approach for the collection subdirectory
  under its parent project (project/collection/)
- Errors during rmtree are logged as warnings and do not fail the
  HTTP response (DB deletion already succeeded)
Previously capture_preview_frame() used tempfile.mkstemp() to create a
new uniquely-named file per poll cycle. When the process was killed
(e.g. systemd restart) the finally-block cleanup was skipped, leaving
files behind. With an operator polling every few seconds this produced
hundreds of stale preview_c*.jpg files in /tmp.

Changes:
- Use a fixed per-camera path /tmp/dtk_preview_c{n}.jpg instead of
  mkstemp; overwrites the same file on every poll cycle so at most one
  file per camera ever exists
- Add a threading.Lock per camera so concurrent polling requests from
  multiple browser tabs serialise writes to the shared path
- Simplify the retry loop (no need to re-create the temp file)
- Add flush_preview_tmp() to delete any dtk_preview_c*.jpg files that
  survived a crash
- Add DELETE /cameras/preview/tmp endpoint so an admin/operator can
  trigger the flush from the settings page
…via stdlib

The module-level Picamera2.set_logging(Picamera2.BEBUG, ...) calls
caused two separate problems:
1. 'BEBUG' is a typo for 'DEBUG' — AttributeError on import, making
   all camera detection fail
2. set_logging() is deprecated in this picamera2 version; the 'output'
   kwarg expects a stream, not a file path, causing a non-fatal
   AttributeError on every import even after fixing the typo

Fix: remove set_logging() from calibration.py and camera_registry.py
entirely. Instead, attach the existing RotatingFileHandler from
subprocess_logger to the 'picamera2' stdlib logger in service.py.
This routes picamera2's internal debug output into the same
capture_service.log without using the deprecated API.
… hang

When the frontend live-preview polls capture_image() at ~80fps while a
full capture is also calling capture_image() on the same Picamera2
instance, picamera2's internal job queue enters an infinite spin loop
('Execute job' repeating endlessly) because two callers are racing on
capture_request() without any mutual exclusion.

Fix: add a threading.Lock per camera index in Picamera2Backend.
capture_image() now acquires this lock before touching the Picamera2
instance. Preview polling and full captures are serialised per camera:
- an in-flight full capture (AF + denoise warmup + save) blocks
  subsequent preview frames until it finishes
- a running preview frame blocks a capture trigger until that frame
  completes (~50ms), then the capture proceeds cleanly

Implementation:
- Add self._camera_locks dict + self._locks_mutex in __init__
- Add _get_camera_lock(camera_index) helper
- Split capture_image() into a public thin wrapper (acquires lock) and
  _capture_image_locked() (the former implementation)
setup_rotating_logger() checks whether a RotatingFileHandler for the
same file path is already attached before adding a new one. Without
this guard, uvicorn's auto-reloader reimports service.py while the
logger singleton (global, cached by name) still has its existing
handler, resulting in every log line being written twice.
Copilot AI review requested due to automatic review settings May 26, 2026 17:04
@jairomelo jairomelo changed the title Dev pre-alpha May 26, 2026
@jairomelo jairomelo merged commit e12d4b2 into main May 26, 2026
1 check failed
@jairomelo jairomelo removed the request for review from Copilot May 26, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant