Various new fucntionality#3
Open
luisabaez wants to merge 13 commits into
Open
Conversation
… (S3, Identity Pool, User Pool Client ID, User Pool ID), and parse user email.
Adds client-side components to enable quick navigation into S3 storage prefixes from the UI and introduces server-side logic to relocate files and notify users. Why: - Improve UX by allowing links to open the storage browser at a specific folder/path (useful for linking from emails or other app pages). - Automate file organization and notifications when objects are processed, including error handling branches. What: - Adds a small navigation flow that stores a target S3 path in sessionStorage and initializes the storage browser to that path, plus a reusable link component for creating folder links. - Expands allowed storage prefixes so the browser can navigate to additional folders. - Implements two Lambda handlers to move/copy objects within S3 based on tags (including an error-branch naming scheme), delete originals, and send email notifications via SES; one variant also generates a presigned URL for direct access. - Sends email notifications containing links back to the application/storage browser so users can find moved files quickly. Benefits: - Faster, direct navigation from emails/pages to relevant S3 folders. - Automated post-processing workflow that organizes files and notifies stakeholders. - Improved traceability for error cases via dedicated error folder naming.
Adds a suite of serverless utilities to implement a tag-driven S3 ingestion and validation workflow. Introduces Lambda helpers to: - Validate and parse incoming filenames/tags and enforce expected table mappings. - Read CSVs, validate headers against database definitions, and generate validation reports. - Execute T-SQL and load CSV rows into the database with robust error handling and error-file generation. - Split CSVs by business unit into in-memory Excel files and upload them with proper S3 tagging. - Manage S3 object tags reliably (including safe stringification/truncation) and centralized S3 upload helper. - Relocate processed files within S3 and send notification emails with file links. - Retrieve secrets and (basic) Cognito user lookup utilities. This centralizes file ingestion logic, improves error reporting and diagnostics, supports tag-driven routing, and enables safe in-memory file generation and tagging for downstream processing.
Adds a GitHub Actions workflow that packages Lambda code into a single deployable artifact and deploys it to all functions automatically or on demand. Provides a single ZIP approach so shared modules can be reused across functions, uses a matrix to deploy 20 functions in parallel for faster rollouts, and authenticates via repository secrets. Includes a packaging job, parallel deploy jobs, and a final summary step, plus guidance for secret setup and function naming. This enables consistent, repeatable, and faster Lambda deployments when the function code changes.
Removes embedded, hard-coded Amplify/Cognito setup and re-enables the declarative auth definition. This cleans up the backend config by using the backend framework's auth helper, avoids leaking platform-specific credentials and settings in source, and makes phone-based login optional. Also removes the unused storage export from the backend composition to keep the backend registration minimal and focused. This was done to torubleshoot deployment issues to AWS Amplify.
…ploy workflow Replaces the previous matrix-based GitHub Actions workflow that deployed 20 separate Lambda targets in parallel with a simplified workflow that packages all Python files and deploys them as a single Lambda function. Simplifies CI by removing the parallel matrix and summary jobs, consolidating packaging, dependency install, AWS credential configuration, deployment and verification into one pipeline. Reduces complexity and maintenance overhead while preserving automated packaging and secure AWS authentication for deployments.
…ntegration Adds a new empty placeholder Lambda test file and updates a function docstring to include a note used for verifying GitHub-AWS integration. The docstring addition is informational only (a timestamped integration test note) and does not change runtime behavior. The new file serves as a minimal placeholder for CI/integration testing workflows.
Storage was defined in storage/resource.ts but was not imported or included in the backend definition. This caused Amplify builds to fail. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Updated amplify_outputs.json to point to hacienda-erp bucket - Configured storage paths for ConversionFiles, ConversionFileErrors, InitialUpload, InitialUploadErrors, TSQLFiles, and DataValidation - Removed amplify_outputs.json from .gitignore to preserve custom config 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- User Pool ID: us-east-1_GbDEpwd4X - User Pool Client ID: 4seq105rqkge73reoarel51ech - Identity Pool ID: us-east-1:5bf79d75-6a3e-4e00-ae61-ae283f593664 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TEST Environment (use this for testing): - User Pool: us-east-1_GKdOkUgkK (hacienda-erp-test) - User Pool Client: 4v7o38ldam1sa6e6ogdipht6vv - Identity Pool: us-east-1:0634f448-eace-44f7-878e-f36d8a47f706 - S3 Bucket: amplify-d2rr9r4nmj79me-va-storagebrowsertestbucket-lrgq2rrslniw PRODUCTION Environment: - User Pool: us-east-1_P8rgnt97e (hacienda-erp-production) - User Pool Client: ak27cunr34o5n448as0u1klja - Identity Pool: us-east-1:9abcff91-45fe-4154-b134-6ebda522a148 - S3 Bucket: hacienda-erp Default config (amplify_outputs.json) is set to TEST environment. To switch to production, copy amplify_outputs.production.json to amplify_outputs.json 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TEST Environment (amplify_outputs.json & amplify_outputs.test.json): - User Pool: us-east-1_iX92bMo7t (Amplify-managed) - Client ID: 1j24bgbmibu5tr8b8psc7ivjl - Identity Pool: us-east-1:e4aa252d-ae90-40d9-b7d7-e69903adc24c - S3 Bucket: amplify-d2rr9r4nmj79me-va-storagebrowsertestbucket-lrgq2rrslniw PRODUCTION Environment (amplify_outputs.production.json): - User Pool: us-east-1_P8rgnt97e (hacienda-erp-production) - Client ID: ak27cunr34o5n448as0u1klja - Identity Pool: us-east-1:9abcff91-45fe-4154-b134-6ebda522a148 - S3 Bucket: hacienda-erp Updated permission format to include entityidentity for proper access control. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Various changes to test strategy to tag files with user tags. Created Storagre Browser wrapping, etc. for this. Also added file folders to the displayed ones in the website.