Conversation
* refactor: improve migration system to go from v0 to latest with removed columns (and possibly tables) - All glory to God our Father and the Lord Jesus Christ and the Holy Spirit. - Enhanced migration system logic and user flow - Improved error handling and feedback during migrations - Simplified migration state/reload handling after migration completes - Updated and cleaned migration files and related utilities * chore: update .gitignore and enhance metro.config.js for production builds - Added 'migration-databases/' to .gitignore to prevent tracking of migration database files. - Removed deprecated production logic for useDrizzleStudio in metro.config.js to streamline module resolution. - Updated package.json and package-lock.json to include new dependencies for improved functionality. - Refactored hooks to conditionally use development or production versions of useDrizzleStudio and useExpoDb, enhancing performance and maintainability. * feat: enhance migration database management and add utility scripts - Updated .gitignore to include specific database files and scripts for better tracking. - Added new database file (1.0.db) for migration purposes. - Introduced scripts for consolidating databases, replacing databases in iOS Simulator, and restarting apps in the simulator, improving development workflow and database management. * refactor: reorganize testing client migrations and update related scripts - Updated .gitignore to include new testing client migration files and scripts for better management. - Refactored migration analysis documentation to reflect changes in database paths and added cleanup steps. - Introduced new scripts for consolidating databases, replacing databases in the iOS Simulator, and restarting apps, enhancing the testing workflow. - Added README for testing client migrations to provide clear instructions and file descriptions. * refactor: enhance migration scripts for iOS and Android support - Updated migration analysis documentation to reflect changes in script usage for both iOS and Android platforms. - Introduced unified scripts for replacing databases and restarting apps, improving the testing workflow. - Enhanced README for testing client migrations with clear instructions and platform detection details. - Added new scripts `replace-device-db.sh` and `restart-device-app.sh` to streamline database management across platforms. * chore: update .gitignore to include additional migration scripts - Added `replace-device-db.sh` and `restart-device-app.sh` to .gitignore for better management of testing client migrations. - Retained legacy iOS-only scripts for backwards compatibility. * chore: remove legacy iOS migration scripts from repository - Deleted `replace-ios-sim-db.sh` and `restart-ios-sim-app.sh` scripts from the repository as they are no longer needed for migration processes. - Updated .gitignore to reflect the removal of these legacy scripts, improving clarity in the testing client migrations. * Fix: Remove trailing semicolon from SQL migration Co-authored-by: realdinozoid <realdinozoid@gmail.com> * Enhance migration scripts with platform detection and initialization steps - Updated `analyze-migration-steps.md` to include critical wait time for app initialization after restart. - Modified `replace-device-db.sh` and `restart-device-app.sh` to support optional platform overrides for better flexibility in testing. - Added validation for platform overrides and improved messaging for users when both iOS and Android devices are detected. - Ensured existing WAL/SHM files are removed before copying new databases to maintain a clean state. * Refactor migration scripts to utilize JSON-first approach for data updates - Updated migration scripts from versions 2.0 to 2.1 and 2.1 to 2.2 to directly manipulate JSON data in the 'data' column of raw PowerSync tables. - Added preflight checks to ensure the existence of relevant raw tables before executing migrations. - Enhanced logging for better visibility during migration processes. * Enhance migration analysis documentation for comprehensive testing - Updated `analyze-migration-steps.md` to include detailed instructions for creating test data that covers all migration scenarios. - Improved clarity on migration requirements, including legacy fields, new fields, and edge cases. - Added verification steps to ensure successful migration execution and data integrity. - Emphasized the importance of checking for data inconsistencies and provided SQL query examples for validation. * Implement robust WebSocket connection handling in MCP server - Added retry logic for connecting to Expo devtools with exponential backoff and jitter to handle connection failures gracefully. - Introduced cleanup functions for managing connection states and handling process termination signals. - Enhanced error logging for unhandled promise rejections and uncaught exceptions to improve debugging. - Implemented a check for Expo dev server accessibility before attempting WebSocket connections to prevent unnecessary retries. * Enhance migration scripts and documentation for improved data handling - Updated `analyze-migration-steps.md` to include critical steps for consolidating the test database after data insertion, ensuring a clean state before copying to devices. - Refactored migration scripts to utilize the `ensureTableExists` function, ensuring necessary PowerSync tables are created if they do not exist. - Improved JSON-first approach in migration scripts, emphasizing the use of `json_extract` and `json_set` for data manipulation. - Enhanced logging and preflight checks to improve migration reliability and visibility during execution. * Refactor: Improve error handling and logging in migration Co-authored-by: realdinozoid <realdinozoid@gmail.com> * Add TruffleHog workflow for secrets scanning - Introduced a new GitHub Actions workflow for TruffleHog to scan for secrets in the codebase. - The workflow is triggered on pushes to the 'dev', 'main', and 'ci' branches. * refactor: remove unnecessary preflight checks from migration system * refactor: remove redundant 'local' argument from getRawTableName call * refactor: remove commented-out migration checks and testing code from System class * feat: introduce PreAuthMigrationCheck component for pre-auth migration handling - Added PreAuthMigrationCheck to manage migrations before user authentication, improving user experience. - Updated MigrationScreen to support both pre-auth and post-auth migration flows. - Enhanced System class with a new method to check for migrations needed before authentication. - Refactored layout to integrate PreAuthMigrationCheck, ensuring migrations are handled seamlessly before login. * feat: add initial database migration file - Introduced a new database migration file (0.0.db) to support upcoming migration processes. - This file serves as a foundational step for managing data migrations in the application. * fix: remove powersync system tables from test databases -- Drop all PowerSync system tables DROP TABLE IF EXISTS ps_oplog; DROP TABLE IF EXISTS ps_buckets; DROP TABLE IF EXISTS ps_crud; DROP TABLE IF EXISTS ps_kv; DROP TABLE IF EXISTS ps_migration; DROP TABLE IF EXISTS ps_stream_subscriptions; DROP TABLE IF EXISTS ps_sync_state; DROP TABLE IF EXISTS ps_tx; DROP TABLE IF EXISTS ps_untyped; DROP TABLE IF EXISTS ps_updated_rows; * fix: remove broken quest_closure view from 0.0.db test database The quest_closure view referenced a non-existent quest_closure_synced table, causing PowerSync initialization errors. Dropped the broken view and consolidated WAL files for both test databases. * fix: remove broken views from 0.0.db and consolidate test databases Remove broken quest_closure and quest_aggregates views from 0.0.db that referenced non-existent tables, causing PowerSync initialization errors. Consolidate WAL files for both 0.0.db and 1.0.db test databases. * fix: update 0.0.db to resolve binary differences Updated the 0.0.db file to address binary differences, ensuring consistency in the test database. This change follows the recent removal of broken views and consolidation of test databases. * feat: enhance migration process with cleanup utility for duplicate languoids - Updated migration script to include a new utility for cleaning up duplicate languoid records created during the migration from version 1.0 to 2.0. - Added a flag in the OpMetadata interface to mark languoids that require cleanup. - Implemented a callback to trigger the cleanup process after successful migrations, ensuring that any duplicates are relinked and removed appropriately. * test: Add Gresi language to 0.0.db for migration testing Add Gresi language record to synced language table so migrations can resolve language names correctly instead of using "Unknown Language". Data verified against Supabase production. * fix(test): add missing synced languages to 0.0.db Add Afrikaans, English, and Yanomami language records from Supabase to prevent "Unknown Language" languoids during migration 1.0→2.0. The migration creates languoids from language references, but defaults to "Unknown Language" when language records are missing from the synced language table. These additions ensure proper language names are used during migration testing. * fixed language cloud vs local uuid mismatch for 0.0 db * refactor: enhance languoid cleanup process and integrate immediate execution post-PowerSync initialization - Updated the cleanup utility to directly query Supabase for remote languoids, improving accuracy in identifying duplicates. - Implemented a transaction-based approach for relinking references and deleting duplicates to ensure atomicity. - Modified the migration cleanup process to run immediately after PowerSync initialization if internet connectivity is available, streamlining the cleanup workflow. - Removed redundant checks and improved logging for better traceability during the cleanup process. * fix: update 0.0.db to resolve binary differences and ensure consistency in test database * feat: implement removal of awaiting_cleanup flag for non-duplicate languoids - Added a new function to remove the awaiting_cleanup flag from languoid metadata. - Enhanced the cleanupDuplicateLanguoids function to handle languoids that do not exist in Supabase, ensuring proper metadata management. - Improved logging for better traceability during the cleanup process. * fix: resolve typecheck error in cleanup.ts and format code * refactor: remove redundant 'local' arguments from getRawTableName calls * chore: update 0.0.db to reflect recent changes and ensure consistency in client migrations * test * Add js-logger dependency and remove cleanup migration script * Refactor interface definitions and improve code formatting across multiple components - Updated interface definitions in DrawerLegendList, Input, Textarea, and LanguoidLinkSuggestion for better readability. - Enhanced formatting in Alert, DrawerContent, and useHybridSupabaseQuery for consistency. - Cleaned up code in useMicrophoneEnergy and NextGenNewTranslationModal for improved clarity and maintainability. * Implement degraded mode functionality and migration improvements - Introduced a degraded mode system to handle migration failures, allowing the app to continue functioning under certain conditions. - Added new service for managing degraded mode state, including retry logic and version checks. - Updated migration scripts to support degraded mode, ensuring the app can recover from repeated migration failures. - Enhanced the MigrationScreen and AppDrawer components to reflect degraded mode status and provide user feedback. - Updated database schema version to 3.0 and added a test migration that intentionally fails to validate degraded mode behavior. - Cleaned up AsyncStorage keys related to degraded mode during database replacement. * Update schema version to 2.2 and remove deprecated migration - Changed the application schema version from 3.0 to 2.2. - Deleted the test migration script for migrating from version 2.2 to 3.0, which was intended to simulate a failure for degraded mode testing. - Updated migration index to reflect the removal of the deprecated migration. * Enhance MigrationScreen and Degraded Mode Service functionality - Added logic to handle auto-start behavior based on degraded mode status and OTA updates in MigrationScreen. - Introduced new state variables to manage migration flow and prevent auto-start when in degraded mode without an OTA update. - Updated logging messages for clarity regarding migration conditions. - Refactored degraded mode checks to include both OTA updates and app version changes, improving migration reliability. - Adjusted the use of local storage for managing degraded mode state and retry counts. * Enhance migration and degraded mode handling - Updated MigrationScreen and System class to improve logging and clarity regarding degraded mode and OTA updates. - Refactored logic to ensure the app can continue functioning under specific conditions when in degraded mode. - Introduced new state management for tracking app version and update ID in local storage. - Enhanced local store functionality to support migration state and preferences, including offline undownload warning. - Improved overall code structure and readability across affected components. * Update component styling and error handling - Added a new guideline to avoid using `SafeAreaView` around `ScrollView` or `FlatList`, recommending the use of `contentInsetAdjustmentBehavior="automatic"` instead. - Refactored `AssetsDeletionDrawer` to remove unnecessary padding in the `View` component. - Enhanced `AssetSettingsModal` to conditionally render settings based on error state, improving user experience during error handling. - Updated `useAssetStatuses` to log detailed error information for asset and quest-asset link status fetching, enhancing debugging capabilities. - Simplified conditional checks in `BibleAssetsView` for better readability and performance. * trigger checks * Refactor degraded mode state management and database interaction - Simplified the retrieval of database results in the System class by consolidating the logic for handling different result formats. - Migrated the last failed version tracking from AsyncStorage to the Zustand store in the Degraded Mode Service, enhancing state management. - Updated local store to include last failed version state and corresponding setter function. - Removed redundant AsyncStorage operations related to failed version tracking, streamlining the degraded mode functionality. * chore: empty commit * Add new database file for client migrations (version 2.2) * Update package.json with a new script for package-lock maintenance and remove unused async storage dev tools from layout * Add cleanup logic to remove orphaned union views from database - Added orphaned view detection in createUnionViews() method - Queries for all existing union views that have corresponding _synced tables - Identifies views not in current drizzleSchema and drops them - Handles schema refactoring cases (e.g., language -> languoid migration) - Logs orphaned views found and cleanup operations performed Co-authored-by: Keean <keeandev@users.noreply.github.com> * cleanup only union views * Add migration backup/restore system with cross-platform support Before running client-side migrations, we now create backups of both the SQLite database and local store (AsyncStorage). If migrations fail, we restore from backup before entering degraded mode. Uses @utils/fileUtils for cross-platform file operations: - Native: expo-file-system - Web: OPFS (Origin Private File System) * Refactor import order and improve logging in migrationBackupService - Reordered imports for better readability. - Enhanced console log formatting in ensureBackupDir function for clarity. * fix clear degraded mode on app upgrade * Remove obsolete client migration database file (2.2.db) * Refactor database handling in migration scripts - Removed obsolete instructions for consolidating test database files. - Updated the replace-device-db.sh script to ensure WAL/SHM files are removed after database copying. - Added cleanup logic for test-cases database files and their associated WAL/SHM files to maintain a clean state. * merge dev into migration system branch (#731) * refactor: improve migration system to handle full migration chain (from v0 -> latest) - All glory to God (#641) * refactor: improve migration system to go from v0 to latest with removed columns (and possibly tables) - All glory to God our Father and the Lord Jesus Christ and the Holy Spirit. - Enhanced migration system logic and user flow - Improved error handling and feedback during migrations - Simplified migration state/reload handling after migration completes - Updated and cleaned migration files and related utilities * chore: update .gitignore and enhance metro.config.js for production builds - Added 'migration-databases/' to .gitignore to prevent tracking of migration database files. - Removed deprecated production logic for useDrizzleStudio in metro.config.js to streamline module resolution. - Updated package.json and package-lock.json to include new dependencies for improved functionality. - Refactored hooks to conditionally use development or production versions of useDrizzleStudio and useExpoDb, enhancing performance and maintainability. * feat: enhance migration database management and add utility scripts - Updated .gitignore to include specific database files and scripts for better tracking. - Added new database file (1.0.db) for migration purposes. - Introduced scripts for consolidating databases, replacing databases in iOS Simulator, and restarting apps in the simulator, improving development workflow and database management. * refactor: reorganize testing client migrations and update related scripts - Updated .gitignore to include new testing client migration files and scripts for better management. - Refactored migration analysis documentation to reflect changes in database paths and added cleanup steps. - Introduced new scripts for consolidating databases, replacing databases in the iOS Simulator, and restarting apps, enhancing the testing workflow. - Added README for testing client migrations to provide clear instructions and file descriptions. * refactor: enhance migration scripts for iOS and Android support - Updated migration analysis documentation to reflect changes in script usage for both iOS and Android platforms. - Introduced unified scripts for replacing databases and restarting apps, improving the testing workflow. - Enhanced README for testing client migrations with clear instructions and platform detection details. - Added new scripts `replace-device-db.sh` and `restart-device-app.sh` to streamline database management across platforms. * chore: update .gitignore to include additional migration scripts - Added `replace-device-db.sh` and `restart-device-app.sh` to .gitignore for better management of testing client migrations. - Retained legacy iOS-only scripts for backwards compatibility. * chore: remove legacy iOS migration scripts from repository - Deleted `replace-ios-sim-db.sh` and `restart-ios-sim-app.sh` scripts from the repository as they are no longer needed for migration processes. - Updated .gitignore to reflect the removal of these legacy scripts, improving clarity in the testing client migrations. * Fix: Remove trailing semicolon from SQL migration Co-authored-by: realdinozoid <realdinozoid@gmail.com> * Enhance migration scripts with platform detection and initialization steps - Updated `analyze-migration-steps.md` to include critical wait time for app initialization after restart. - Modified `replace-device-db.sh` and `restart-device-app.sh` to support optional platform overrides for better flexibility in testing. - Added validation for platform overrides and improved messaging for users when both iOS and Android devices are detected. - Ensured existing WAL/SHM files are removed before copying new databases to maintain a clean state. * Refactor migration scripts to utilize JSON-first approach for data updates - Updated migration scripts from versions 2.0 to 2.1 and 2.1 to 2.2 to directly manipulate JSON data in the 'data' column of raw PowerSync tables. - Added preflight checks to ensure the existence of relevant raw tables before executing migrations. - Enhanced logging for better visibility during migration processes. * Enhance migration analysis documentation for comprehensive testing - Updated `analyze-migration-steps.md` to include detailed instructions for creating test data that covers all migration scenarios. - Improved clarity on migration requirements, including legacy fields, new fields, and edge cases. - Added verification steps to ensure successful migration execution and data integrity. - Emphasized the importance of checking for data inconsistencies and provided SQL query examples for validation. * Implement robust WebSocket connection handling in MCP server - Added retry logic for connecting to Expo devtools with exponential backoff and jitter to handle connection failures gracefully. - Introduced cleanup functions for managing connection states and handling process termination signals. - Enhanced error logging for unhandled promise rejections and uncaught exceptions to improve debugging. - Implemented a check for Expo dev server accessibility before attempting WebSocket connections to prevent unnecessary retries. * Enhance migration scripts and documentation for improved data handling - Updated `analyze-migration-steps.md` to include critical steps for consolidating the test database after data insertion, ensuring a clean state before copying to devices. - Refactored migration scripts to utilize the `ensureTableExists` function, ensuring necessary PowerSync tables are created if they do not exist. - Improved JSON-first approach in migration scripts, emphasizing the use of `json_extract` and `json_set` for data manipulation. - Enhanced logging and preflight checks to improve migration reliability and visibility during execution. * Refactor: Improve error handling and logging in migration Co-authored-by: realdinozoid <realdinozoid@gmail.com> * Add TruffleHog workflow for secrets scanning - Introduced a new GitHub Actions workflow for TruffleHog to scan for secrets in the codebase. - The workflow is triggered on pushes to the 'dev', 'main', and 'ci' branches. * refactor: remove unnecessary preflight checks from migration system * refactor: remove redundant 'local' argument from getRawTableName call * refactor: remove commented-out migration checks and testing code from System class * feat: introduce PreAuthMigrationCheck component for pre-auth migration handling - Added PreAuthMigrationCheck to manage migrations before user authentication, improving user experience. - Updated MigrationScreen to support both pre-auth and post-auth migration flows. - Enhanced System class with a new method to check for migrations needed before authentication. - Refactored layout to integrate PreAuthMigrationCheck, ensuring migrations are handled seamlessly before login. * feat: add initial database migration file - Introduced a new database migration file (0.0.db) to support upcoming migration processes. - This file serves as a foundational step for managing data migrations in the application. * fix: remove powersync system tables from test databases -- Drop all PowerSync system tables DROP TABLE IF EXISTS ps_oplog; DROP TABLE IF EXISTS ps_buckets; DROP TABLE IF EXISTS ps_crud; DROP TABLE IF EXISTS ps_kv; DROP TABLE IF EXISTS ps_migration; DROP TABLE IF EXISTS ps_stream_subscriptions; DROP TABLE IF EXISTS ps_sync_state; DROP TABLE IF EXISTS ps_tx; DROP TABLE IF EXISTS ps_untyped; DROP TABLE IF EXISTS ps_updated_rows; * fix: remove broken quest_closure view from 0.0.db test database The quest_closure view referenced a non-existent quest_closure_synced table, causing PowerSync initialization errors. Dropped the broken view and consolidated WAL files for both test databases. * fix: remove broken views from 0.0.db and consolidate test databases Remove broken quest_closure and quest_aggregates views from 0.0.db that referenced non-existent tables, causing PowerSync initialization errors. Consolidate WAL files for both 0.0.db and 1.0.db test databases. * fix: update 0.0.db to resolve binary differences Updated the 0.0.db file to address binary differences, ensuring consistency in the test database. This change follows the recent removal of broken views and consolidation of test databases. * feat: enhance migration process with cleanup utility for duplicate languoids - Updated migration script to include a new utility for cleaning up duplicate languoid records created during the migration from version 1.0 to 2.0. - Added a flag in the OpMetadata interface to mark languoids that require cleanup. - Implemented a callback to trigger the cleanup process after successful migrations, ensuring that any duplicates are relinked and removed appropriately. * test: Add Gresi language to 0.0.db for migration testing Add Gresi language record to synced language table so migrations can resolve language names correctly instead of using "Unknown Language". Data verified against Supabase production. * fix(test): add missing synced languages to 0.0.db Add Afrikaans, English, and Yanomami language records from Supabase to prevent "Unknown Language" languoids during migration 1.0→2.0. The migration creates languoids from language references, but defaults to "Unknown Language" when language records are missing from the synced language table. These additions ensure proper language names are used during migration testing. * fixed language cloud vs local uuid mismatch for 0.0 db * refactor: enhance languoid cleanup process and integrate immediate execution post-PowerSync initialization - Updated the cleanup utility to directly query Supabase for remote languoids, improving accuracy in identifying duplicates. - Implemented a transaction-based approach for relinking references and deleting duplicates to ensure atomicity. - Modified the migration cleanup process to run immediately after PowerSync initialization if internet connectivity is available, streamlining the cleanup workflow. - Removed redundant checks and improved logging for better traceability during the cleanup process. * fix: update 0.0.db to resolve binary differences and ensure consistency in test database * feat: implement removal of awaiting_cleanup flag for non-duplicate languoids - Added a new function to remove the awaiting_cleanup flag from languoid metadata. - Enhanced the cleanupDuplicateLanguoids function to handle languoids that do not exist in Supabase, ensuring proper metadata management. - Improved logging for better traceability during the cleanup process. * fix: resolve typecheck error in cleanup.ts and format code * refactor: remove redundant 'local' arguments from getRawTableName calls * chore: update 0.0.db to reflect recent changes and ensure consistency in client migrations * test * Add js-logger dependency and remove cleanup migration script * Refactor interface definitions and improve code formatting across multiple components - Updated interface definitions in DrawerLegendList, Input, Textarea, and LanguoidLinkSuggestion for better readability. - Enhanced formatting in Alert, DrawerContent, and useHybridSupabaseQuery for consistency. - Cleaned up code in useMicrophoneEnergy and NextGenNewTranslationModal for improved clarity and maintainability. * Implement degraded mode functionality and migration improvements - Introduced a degraded mode system to handle migration failures, allowing the app to continue functioning under certain conditions. - Added new service for managing degraded mode state, including retry logic and version checks. - Updated migration scripts to support degraded mode, ensuring the app can recover from repeated migration failures. - Enhanced the MigrationScreen and AppDrawer components to reflect degraded mode status and provide user feedback. - Updated database schema version to 3.0 and added a test migration that intentionally fails to validate degraded mode behavior. - Cleaned up AsyncStorage keys related to degraded mode during database replacement. * Update schema version to 2.2 and remove deprecated migration - Changed the application schema version from 3.0 to 2.2. - Deleted the test migration script for migrating from version 2.2 to 3.0, which was intended to simulate a failure for degraded mode testing. - Updated migration index to reflect the removal of the deprecated migration. * Enhance MigrationScreen and Degraded Mode Service functionality - Added logic to handle auto-start behavior based on degraded mode status and OTA updates in MigrationScreen. - Introduced new state variables to manage migration flow and prevent auto-start when in degraded mode without an OTA update. - Updated logging messages for clarity regarding migration conditions. - Refactored degraded mode checks to include both OTA updates and app version changes, improving migration reliability. - Adjusted the use of local storage for managing degraded mode state and retry counts. * Enhance migration and degraded mode handling - Updated MigrationScreen and System class to improve logging and clarity regarding degraded mode and OTA updates. - Refactored logic to ensure the app can continue functioning under specific conditions when in degraded mode. - Introduced new state management for tracking app version and update ID in local storage. - Enhanced local store functionality to support migration state and preferences, including offline undownload warning. - Improved overall code structure and readability across affected components. * Update component styling and error handling - Added a new guideline to avoid using `SafeAreaView` around `ScrollView` or `FlatList`, recommending the use of `contentInsetAdjustmentBehavior="automatic"` instead. - Refactored `AssetsDeletionDrawer` to remove unnecessary padding in the `View` component. - Enhanced `AssetSettingsModal` to conditionally render settings based on error state, improving user experience during error handling. - Updated `useAssetStatuses` to log detailed error information for asset and quest-asset link status fetching, enhancing debugging capabilities. - Simplified conditional checks in `BibleAssetsView` for better readability and performance. * trigger checks * Refactor degraded mode state management and database interaction - Simplified the retrieval of database results in the System class by consolidating the logic for handling different result formats. - Migrated the last failed version tracking from AsyncStorage to the Zustand store in the Degraded Mode Service, enhancing state management. - Updated local store to include last failed version state and corresponding setter function. - Removed redundant AsyncStorage operations related to failed version tracking, streamlining the degraded mode functionality. * chore: empty commit * Add new database file for client migrations (version 2.2) * Update package.json with a new script for package-lock maintenance and remove unused async storage dev tools from layout * Add cleanup logic to remove orphaned union views from database - Added orphaned view detection in createUnionViews() method - Queries for all existing union views that have corresponding _synced tables - Identifies views not in current drizzleSchema and drops them - Handles schema refactoring cases (e.g., language -> languoid migration) - Logs orphaned views found and cleanup operations performed Co-authored-by: Keean <keeandev@users.noreply.github.com> * cleanup only union views * Add migration backup/restore system with cross-platform support Before running client-side migrations, we now create backups of both the SQLite database and local store (AsyncStorage). If migrations fail, we restore from backup before entering degraded mode. Uses @utils/fileUtils for cross-platform file operations: - Native: expo-file-system - Web: OPFS (Origin Private File System) * Refactor import order and improve logging in migrationBackupService - Reordered imports for better readability. - Enhanced console log formatting in ensureBackupDir function for clarity. * fix clear degraded mode on app upgrade * Remove obsolete client migration database file (2.2.db) * Refactor database handling in migration scripts - Removed obsolete instructions for consolidating test database files. - Updated the replace-device-db.sh script to ensure WAL/SHM files are removed after database copying. - Added cleanup logic for test-cases database files and their associated WAL/SHM files to maintain a clean state. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Keean <keeandev@users.noreply.github.com> * Revert "refactor: improve migration system to handle full migration chain (fr…" (#730) This reverts commit 2409a23. * languoid sync backwards compatible (#729) Last version introduced rpc call to fill languoid and languoid-related records' download_profiles field of a new project with creator's id. But this skips people who continue to make projects without updating the app. This update swaps out rpc call and function with trigger and function so that project_language_link new record triggers update in all languoid-related tables (download_profiles field) --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Keean <keeandev@users.noreply.github.com> Co-authored-by: CalJosKos <120157396+CalJosKos@users.noreply.github.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> Co-authored-by: Keean <keeandev@users.noreply.github.com> Co-authored-by: CalJosKos <120157396+CalJosKos@users.noreply.github.com>
* refactor: improve migration system to go from v0 to latest with removed columns (and possibly tables) - All glory to God our Father and the Lord Jesus Christ and the Holy Spirit.
- Enhanced migration system logic and user flow
- Improved error handling and feedback during migrations
- Simplified migration state/reload handling after migration completes
- Updated and cleaned migration files and related utilities
* chore: update .gitignore and enhance metro.config.js for production builds
- Added 'migration-databases/' to .gitignore to prevent tracking of migration database files.
- Removed deprecated production logic for useDrizzleStudio in metro.config.js to streamline module resolution.
- Updated package.json and package-lock.json to include new dependencies for improved functionality.
- Refactored hooks to conditionally use development or production versions of useDrizzleStudio and useExpoDb, enhancing performance and maintainability.
* feat: enhance migration database management and add utility scripts
- Updated .gitignore to include specific database files and scripts for better tracking.
- Added new database file (1.0.db) for migration purposes.
- Introduced scripts for consolidating databases, replacing databases in iOS Simulator, and restarting apps in the simulator, improving development workflow and database management.
* refactor: reorganize testing client migrations and update related scripts
- Updated .gitignore to include new testing client migration files and scripts for better management.
- Refactored migration analysis documentation to reflect changes in database paths and added cleanup steps.
- Introduced new scripts for consolidating databases, replacing databases in the iOS Simulator, and restarting apps, enhancing the testing workflow.
- Added README for testing client migrations to provide clear instructions and file descriptions.
* refactor: enhance migration scripts for iOS and Android support
- Updated migration analysis documentation to reflect changes in script usage for both iOS and Android platforms.
- Introduced unified scripts for replacing databases and restarting apps, improving the testing workflow.
- Enhanced README for testing client migrations with clear instructions and platform detection details.
- Added new scripts `replace-device-db.sh` and `restart-device-app.sh` to streamline database management across platforms.
* chore: update .gitignore to include additional migration scripts
- Added `replace-device-db.sh` and `restart-device-app.sh` to .gitignore for better management of testing client migrations.
- Retained legacy iOS-only scripts for backwards compatibility.
* chore: remove legacy iOS migration scripts from repository
- Deleted `replace-ios-sim-db.sh` and `restart-ios-sim-app.sh` scripts from the repository as they are no longer needed for migration processes.
- Updated .gitignore to reflect the removal of these legacy scripts, improving clarity in the testing client migrations.
* Fix: Remove trailing semicolon from SQL migration
Co-authored-by: realdinozoid <realdinozoid@gmail.com>
* Enhance migration scripts with platform detection and initialization steps
- Updated `analyze-migration-steps.md` to include critical wait time for app initialization after restart.
- Modified `replace-device-db.sh` and `restart-device-app.sh` to support optional platform overrides for better flexibility in testing.
- Added validation for platform overrides and improved messaging for users when both iOS and Android devices are detected.
- Ensured existing WAL/SHM files are removed before copying new databases to maintain a clean state.
* Refactor migration scripts to utilize JSON-first approach for data updates
- Updated migration scripts from versions 2.0 to 2.1 and 2.1 to 2.2 to directly manipulate JSON data in the 'data' column of raw PowerSync tables.
- Added preflight checks to ensure the existence of relevant raw tables before executing migrations.
- Enhanced logging for better visibility during migration processes.
* Enhance migration analysis documentation for comprehensive testing
- Updated `analyze-migration-steps.md` to include detailed instructions for creating test data that covers all migration scenarios.
- Improved clarity on migration requirements, including legacy fields, new fields, and edge cases.
- Added verification steps to ensure successful migration execution and data integrity.
- Emphasized the importance of checking for data inconsistencies and provided SQL query examples for validation.
* Implement robust WebSocket connection handling in MCP server
- Added retry logic for connecting to Expo devtools with exponential backoff and jitter to handle connection failures gracefully.
- Introduced cleanup functions for managing connection states and handling process termination signals.
- Enhanced error logging for unhandled promise rejections and uncaught exceptions to improve debugging.
- Implemented a check for Expo dev server accessibility before attempting WebSocket connections to prevent unnecessary retries.
* Enhance migration scripts and documentation for improved data handling
- Updated `analyze-migration-steps.md` to include critical steps for consolidating the test database after data insertion, ensuring a clean state before copying to devices.
- Refactored migration scripts to utilize the `ensureTableExists` function, ensuring necessary PowerSync tables are created if they do not exist.
- Improved JSON-first approach in migration scripts, emphasizing the use of `json_extract` and `json_set` for data manipulation.
- Enhanced logging and preflight checks to improve migration reliability and visibility during execution.
* Refactor: Improve error handling and logging in migration
Co-authored-by: realdinozoid <realdinozoid@gmail.com>
* Add TruffleHog workflow for secrets scanning
- Introduced a new GitHub Actions workflow for TruffleHog to scan for secrets in the codebase.
- The workflow is triggered on pushes to the 'dev', 'main', and 'ci' branches.
* chore: Add TruffleHog secret scanning workflow
- Introduced a new GitHub Actions workflow for TruffleHog to scan for secrets in the codebase during push and pull request events, enhancing security by identifying potential sensitive information leaks.
* chore: Update production test workflow configuration
- Changed the trigger for the production tests workflow from push to pull_request for better control over test execution.
- Updated environment variable references to use `env` instead of `secrets`, improving clarity and consistency.
* chore: Enhance TruffleHog workflow configuration
- Added a name to the TruffleHog secrets scanning workflow for better identification in the GitHub Actions interface.
* chore: Update TruffleHog workflow branch triggers
- Removed the 'main' branch from the push triggers for the TruffleHog secrets scanning workflow, streamlining the branches monitored for changes.
* refactor: remove unnecessary preflight checks from migration system
* chore: Update environment variable references in workflows and YAML files
- Removed outdated environment variable references from the preview and production test workflows.
- Updated appId references in the maestro YAML files to use MAESTRO_APP_ID instead of APP_ID.
- Adjusted usage instructions to reflect the new MAESTRO_* variable names for consistency.
* refactor: remove redundant 'local' argument from getRawTableName call
* refactor: remove commented-out migration checks and testing code from System class
* chore: Update eas.json to include withoutCredentials option for simulators
- Added "withoutCredentials": true to both preview-simulator and production-simulator configurations for improved credential management.
* chore: Enable screen recording in preview test workflow
- Added a parameter to enable screen recording during the execution of preview tests, enhancing debugging capabilities.
* feat: introduce PreAuthMigrationCheck component for pre-auth migration handling
- Added PreAuthMigrationCheck to manage migrations before user authentication, improving user experience.
- Updated MigrationScreen to support both pre-auth and post-auth migration flows.
- Enhanced System class with a new method to check for migrations needed before authentication.
- Refactored layout to integrate PreAuthMigrationCheck, ensuring migrations are handled seamlessly before login.
* feat: add initial database migration file
- Introduced a new database migration file (0.0.db) to support upcoming migration processes.
- This file serves as a foundational step for managing data migrations in the application.
* fix: remove powersync system tables from test databases
-- Drop all PowerSync system tables
DROP TABLE IF EXISTS ps_oplog;
DROP TABLE IF EXISTS ps_buckets;
DROP TABLE IF EXISTS ps_crud;
DROP TABLE IF EXISTS ps_kv;
DROP TABLE IF EXISTS ps_migration;
DROP TABLE IF EXISTS ps_stream_subscriptions;
DROP TABLE IF EXISTS ps_sync_state;
DROP TABLE IF EXISTS ps_tx;
DROP TABLE IF EXISTS ps_untyped;
DROP TABLE IF EXISTS ps_updated_rows;
* fix: remove broken quest_closure view from 0.0.db test database
The quest_closure view referenced a non-existent quest_closure_synced table,
causing PowerSync initialization errors. Dropped the broken view and consolidated
WAL files for both test databases.
* fix: remove broken views from 0.0.db and consolidate test databases
Remove broken quest_closure and quest_aggregates views from 0.0.db that
referenced non-existent tables, causing PowerSync initialization errors.
Consolidate WAL files for both 0.0.db and 1.0.db test databases.
* fix: update 0.0.db to resolve binary differences
Updated the 0.0.db file to address binary differences, ensuring consistency in the test database. This change follows the recent removal of broken views and consolidation of test databases.
* Fix EAS Maestro tests: Use Android 31 to prevent edge-to-edge emulator crashes
Edge-to-edge features require Android 12 (API 31) minimum. Configure Maestro tests to use Android 31 with default system image and nested virtualization.
* feat: enhance migration process with cleanup utility for duplicate languoids
- Updated migration script to include a new utility for cleaning up duplicate languoid records created during the migration from version 1.0 to 2.0.
- Added a flag in the OpMetadata interface to mark languoids that require cleanup.
- Implemented a callback to trigger the cleanup process after successful migrations, ensuring that any duplicates are relinked and removed appropriately.
* test: Add Gresi language to 0.0.db for migration testing
Add Gresi language record to synced language table so migrations
can resolve language names correctly instead of using "Unknown Language".
Data verified against Supabase production.
* fix(test): add missing synced languages to 0.0.db
Add Afrikaans, English, and Yanomami language records from Supabase
to prevent "Unknown Language" languoids during migration 1.0→2.0.
The migration creates languoids from language references, but defaults
to "Unknown Language" when language records are missing from the
synced language table. These additions ensure proper language names
are used during migration testing.
* fixed language cloud vs local uuid mismatch for 0.0 db
* fix: replace percent-based coordinates with test IDs for Maestro tests
- Add testID to onboarding close button (onboarding-close-button)
- Add testIDs to step indicators (onboarding-step-{step})
- Update register.yaml, sign-in.yaml, and reset-password.yaml to use test IDs instead of percent coordinates
May the grace of our Lord Jesus Christ, the love of God, and the fellowship of the Holy Spirit be with us all. Amen.
* refactor: enhance languoid cleanup process and integrate immediate execution post-PowerSync initialization
- Updated the cleanup utility to directly query Supabase for remote languoids, improving accuracy in identifying duplicates.
- Implemented a transaction-based approach for relinking references and deleting duplicates to ensure atomicity.
- Modified the migration cleanup process to run immediately after PowerSync initialization if internet connectivity is available, streamlining the cleanup workflow.
- Removed redundant checks and improved logging for better traceability during the cleanup process.
* fix: update 0.0.db to resolve binary differences and ensure consistency in test database
* chore: update EAS test workflows to use 'preview' profile (#668)
* Ci (#670)
* chore: update EAS test workflows to use Pixel 6 device
* chore: update EAS test workflows to use Pixel 3a and run on linux-medium
- Changed device identifier from Pixel 6 to Pixel 3a for both preview and production test workflows.
- Updated the execution environment to linux-medium for improved resource allocation.
* chore: update EAS test workflows to use 'preview' profile
- Changed the profile parameter from 'preview-simulator' to 'preview' in the Android test workflows for consistency and clarity.
* chore: update EAS test workflows to use Pixel 6 device
- Changed device identifier from Pixel 3a to Pixel 6 for both preview and production test workflows, ensuring consistency in testing environments.
* feat: implement removal of awaiting_cleanup flag for non-duplicate languoids
- Added a new function to remove the awaiting_cleanup flag from languoid metadata.
- Enhanced the cleanupDuplicateLanguoids function to handle languoids that do not exist in Supabase, ensuring proper metadata management.
- Improved logging for better traceability during the cleanup process.
* chore: update EAS test workflows to include Android system image package
- Added 'android_system_image_package' parameter for both preview and production test workflows to specify the system image for Android 31.
- Retained 'record_screen' parameter for consistency in test execution.
* refactor: update Supabase URL constants in deleteUser and generatePasswordResetLink functions
- Changed constants from SUPABASE_URL and SUPABASE_SERVICE_ROLE_KEY to MAESTRO_SUPABASE_URL and MAESTRO_SUPABASE_SERVICE_ROLE_KEY for better clarity and consistency.
- Improved error message in generatePasswordResetLink function to specify 'Supabase URL' instead of 'SUPABASE_URL'.
* Fix typecheck error in cleanup.ts and format files
- Fix type error where empty object {} was not assignable to OpMetadata type
- Ensure schema_version is preserved when removing awaiting_cleanup flag
- Add explicit OpMetadata type annotation and proper type checking
- Format cleanup.ts and index.ts files
Co-authored-by: realdinozoid <realdinozoid@gmail.com>
* remove old changes that came back
* remove redundant code
* remove duplicate comment
* fix jsx ending tag from `Pressable` to `View`
* chore: downgrade package version in package-lock.json to 2.0.12 and update resolved URLs and integrity hashes for several dependencies
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Create Recording View as a page * Update UI components * UI Changes * Replace Inserting Wheel with LegendList * Update Record Button * Adding function to selected between add or replace audio * Fix play all assets when no card is selected * Hide buttons when recording * Fix First Mount Error * Fix Boundary Component Size * Fix Boundary Component Size * Add Smooth Transaction * Indexing items in the list * Added button to link to Asset Detail * Change behavior to add verse labels * Improve performance removing render dependencies * Enhance scrolling position when recording * Change Selection Menu
* fixed upload message * fixed linter problems * potentially fixed an issue * Update SupabaseConnector.ts removed useless failed ops stuff * re-trigger CI: Maestro test infra error Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Ryder Wishart <ryderwishart@gmail.com> Co-authored-by: Keean <38674879+keeandev@users.noreply.github.com> Co-authored-by: CalJosKos <120157396+CalJosKos@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: calebfrontier <caleb@frontierrnd.com>
* Remove unnecessary __DEV__ definition in metro.config.js * Empty commit Co-authored-by: realdinozoid <realdinozoid@gmail.com> --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>
* Version * Highlight assets recorded last * Fix metadata when editing labels * Fix verses limit when recording * Removed auto-select when adding verse pills * Enhancing performance * Reducing dependencies
* add order_index to acl, remove merging, verse label default - Bump to 2.0.14 - Add order_index to asset_content_link to preserve segment order after merging - Backfill existing records with 1-based sequence from created_at (Supabase + local migrations) - Use 1-based indexing to avoid PowerSync silent no-op when writing default value (0) - All segment queries sort by order_index with created_at as tiebreaker - Add BEFORE INSERT trigger to auto-assign order_index based on created_at position for old client uploads Add reorder UI in asset detail carousel — tap "N of M" button to move a segment - Update merge logic to assign sequential order_index to new content links - Hide merge button behind enableMerge experimental feature flag (default off) - Make verse labeling on by default; remove prompt dialog - Bump schema version to 2.3 (min_required stays at 2.1) * fix typecheck, formatting
app handles all-zero acl records by defaulting to created at anyways
audio files were already uploaded successfully during the recording step. The publish flow is redundantly trying to queue them again, and fails because the local files were already moved. The error is a false alarm -- the data made it to the server.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Updates to Preview Branch (dev) ↗︎
Tasks are run on every commit but only new migration files are pushed.
View logs for this Workflow Run ↗︎. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces two new documentation commands for database migration workflows, updates command definitions, and adds a best practice for React Native scrollable components. The most significant changes are the addition of comprehensive guides for analyzing migration steps and for removing backwards compatibility code, which will help standardize and clarify these processes for the team.
Migration Documentation and Workflow Improvements:
.cursor/commands/analyze-migration-steps.md. This covers preparing test databases, analyzing migration paths, preparing test data, running migrations, verifying results, and cleanup..cursor/commands/remove-backwards-compatibility.md, including how to align schema definitions and application code with migration history, and critical warnings for both client and server code.Command and Tooling Updates:
shadcncommand tolocal-dband updated its implementation to run the local database server script in.cursor/mcp.json.shadcncommand definition to.cursor/mcp.jsonto preserve its previous functionality alongside the newlocal-dbcommand.React Native Component Best Practices:
.cursor/rules/creating-components.mdcrecommending the use of thecontentInsetAdjustmentBehavior="automatic"prop onScrollVieworFlatListinstead of wrapping them withSafeAreaView.