Add Nix development environment and Docker builds#1
Open
onethirtyfive wants to merge 228 commits intomainfrom
Open
Add Nix development environment and Docker builds#1onethirtyfive wants to merge 228 commits intomainfrom
onethirtyfive wants to merge 228 commits intomainfrom
Conversation
onethirtyfive
force-pushed
the
feat/nix-docker-builds
branch
2 times, most recently
from
3adb2f0 to
a132949
Compare
…ing (TryGhost#25364) ref https://linear.app/ghost/issue/BER-2937/improve-notification-grouping-by-adding-date-criteria - removed the feature flag, and enabled the time grouping by default - fixed notifications grouping out of order when changing time windows - notifications now only group with consecutive ones of the same type <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Remove the notification-group flag and enforce 24h, sequential same‑type notification grouping to preserve chronological order. > > - **Notifications (ActivityPub)**: > - Remove `notification-group` feature flag and related checks; time-based grouping is now always on. > - Revise `groupNotifications`: > - Always group by 24h time buckets. > - Add sequence key to prevent grouping across type changes, preserving chronological order and grouping only consecutive same-type notifications. > - Keep replies/mentions ungrouped; update group keys for likes/reposts/follows. > - **Feature Flags**: > - Clear `FEATURE_FLAGS` list; remove flag usage from `Notifications.tsx`. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit af47e22. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
onethirtyfive
force-pushed
the
feat/nix-docker-builds
branch
from
40b5a53 to
3980c18
Compare
closes [GVA-583](https://linear.app/ghost/issue/GVA-583/create-migration-in-ghost-to-add-csd-member-count-to-emails) We need to know how many emails were sent using the CSD from the previous email in order to ramp up the sending from the CSD. See [spike PR](TryGhost#25303) for our test implementation of domain warmup using this field.
closes [GVA-587](https://linear.app/ghost/issue/GVA-587/create-a-migration-for-the-email-batches-table-to-say-whether-its-sent) We need to know whether an email batch should be sent from the fallback domain or the primary configured domain. See [spike PR](TryGhost#25303) for our test implementation.
…Ghost#25354) ref https://linear.app/ghost/issue/BER-2610 Adds test directory support to typescript configuration and typechecking command to admin.
…host#25355) ref https://linear.app/ghost/issue/BER-2610 Adds linting rule to not allow relative imports unless its a sibling or sub directory.
closes https://linear.app/ghost/issue/NY-708 - adds an `outbox` table - this table is used to provide durability and atomicity in an event-driven system - for events that we might want to take some action on, such as creation of a member, we can create a transaction that writes to both the members table and outbox table (among others) - The outbox table is then processed by some system(s) that take an action based on the event type. Where the outbox table differs from tables like `members_created_events` is that the outbox table is meant to be cleaned up over time, whereas `members_created_events` is a historical record
ref https://linear.app/ghost/issue/BER-2962 - a few discovery feed topics were low-volume and not bringing enough value, so we're removing them from the UI
no ref. - The word "Export" was redundant in the Labs/Migration export buttons. Also it wasn't following the same layout pattern as the import ones. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Shortens export button labels and switches the MigrationToolsExport actions to a responsive grid layout. > > - **UI (Settings › Migration tools)** > - **Layout**: Replace flex row with responsive grid (`grid-cols-1`, `md:grid-cols-2`, `lg:grid-cols-3`). > - **Copy**: Shorten button labels from `Export content & settings` → `Content & settings` and `Export post analytics` → `Post analytics`. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 1e774900fee500b81a2277c8421aed0587c0b499. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
ref https://linear.app/ghost/issue/BER-2610 Added `useUserPreferences` and `useEditUserPreferences` hook to encapsulate querying and editing the User's "preferences". Also adds testing infrastructure for hooks, including fixtures, factories and configuration.
no ref Caused the split helper to no longer include any empty strings (caused by sequential separators or separators at start/end).
…#25282) no ref This PR adds the ability to concat an array (passed as a single argument) to the existing concat helper. Enables syntax such as: ``` {{concat (split "hello world" separator=" ") separator="|"}} ``` New tests added! No changes to existing behavior unless someone was already passing arrays.
no ref Accidentally merged overlapping incompatible PRs (TryGhost#25282 and TryGhost#25283). Sorry, oops!
ref https://linear.app/ghost/issue/ONC-1248/ Adding query params to 'old' site urls (particularly .html) tends to break a link when outbound link tagging is enabled. Occasionally we need to add a new domain to the list.
ref https://ghost.slack.com/archives/C09K9BSGYQM/p1761678873410759 The motivation for this PR is to make Ghost's docker development setup easier to work with and reason about by following generally accepted conventions for file organization: - Moved main development Dockerfile to repo root for more convenient docker commands and a more conventional setup - Moved entire `.docker` directory to `docker` for a more conventional setup - Removed fragile `update-dockerfile.js` and `update-compose.js` scripts
onethirtyfive
force-pushed
the
feat/nix-docker-builds
branch
from
3980c18 to
ef2c03c
Compare
onethirtyfive
force-pushed
the
feat/nix-docker-builds
branch
7 times, most recently
from
a536ae4 to
3fc9a2e
Compare
ref https://linear.app/ghost/issue/BER-2915/integrate-settings-app-with-admin-shell The Tanstack/React Query dev tools makes it easier for us to introspect the query cache and debug issues with invalidation and mutations.
ref https://linear.app/ghost/issue/BER-2983/display-correct-member-count-in-sidebar Replaced hardcoded count with live data. The count syncs automatically with Ember changes through the existing bridge.
Changelog for v2.56.2 -> 2.56.3: - Updated i18n translations
ref https://linear.app/ghost/issue/NY-801 - Randomizes when the member welcome email job runs. Still every 5 minutes, but starts on a different minute/second - This will help avoid a "thundering herd" problem on app servers running multiple Ghost instances, and is the same pattern we use for other jobs like email analytics
ref https://linear.app/ghost/issue/NY-772/add-automated-emails-crud-api-endpoints The new `automated_emails` model should only be accessible by Administrators+ in Ghost via API. This adds the appropriate permissions to Ghost's DB to enforce these permissions in the API endpoints, which will be included in a subsequent commit.
ref https://linear.app/ghost/issue/NY-810/general-improvements-in-analytics-to-work-better-with-the-new-filter - This PR moves the Locations tab under the Web tab in site-wide analytics. The main reason is that Locations is very much related to Web traffic both in terms of data and functions (filtering). Now that we add more filtering options for web related data it makes sense to merge Locations under Web. - This change also allows "click-through" filtering — users can click on a Location, Source etc. to activate filtering by that value. With Locations on a separate tab, this experience becomes very scattered / impossible to do it in a nice way. - Quite a few visual details were off in Analytics in general which are all related to this change: spacings were too wide, some emphasis (icons) was missing on the sub navigation, keyboard navigation was missing for the filters component (for advanced users) etc. - Again, related to the above, the current NavBar component was not flexible enough to be used for the new filtering mechanism. The usage of CSS grid makes it much easier to use it for the new filtering UX. --------- Co-authored-by: Steve Larson <9larsons@gmail.com>
ref https://linear.app/ghost/issue/NY-772 This PR adds an API endpoint for the newly created`automated_emails` table. The endpoint will be used by the frontend settings UI to persist the user generated email content for welcome emails.
…eanup (TryGhost#25609) ref https://linear.app/ghost/issue/NY-821 - makes it so if welcome emails are configured and a member signs up via portal, a welcome email will send right away instead of waiting for the periodic job - checking `this.processing` prevents multiple instantiations; and because of how the batching in the outbox processing works, new members created while the outbox is processing will get picked up - we still have the periodic (5min) job that runs to process the outbox - this will be useful for retries and making sure entries don't get missed
no issue This tab has been removed from the analytics UI, so we no longer need this test. It was able to be merged with a failing test because the E2E tests are not currently taken into account by this repo's branch protection, which I'll fix in a separate PR. For now, this gets the E2E tests back into a passing state.
ref https://linear.app/ghost/issue/BER-3081/slice-1-the-search-result-shouldnt-be-cleared-on-keystroke Previously, when typing search queries in the search modal, there was a jarring flash where the UI would go blank while fetching results. This happened because the results state was cleared immediately when a new search started.
* added a way to merge e2e reports for React failed test runs separately, so that you can see traces and screenshots of the tests failed for Ember and React e2e test runs * added React E2E Tests Failed PR comment generator, now you will see one for React and one for Ember in PR
…#25621) ref https://linear.app/ghost/issue/PRO-1538/ - Setting highWaterMark to chunkSize so Node.js reads larger chunks from disk, reducing the number of Buffer.concat operations needed. - This is a Node.js internal optimization that doesn't change functional behavior. Existing tests already verify chunks are yielded correctly and multipart uploads work as expected. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Configures fs.createReadStream to use highWaterMark equal to the multipart chunk size in S3Storage’s chunk reader. > > - **Storage**: > - Update `ghost/core/core/server/adapters/storage/S3Storage.ts`: > - In `readFileInChunks`, set `fs.createReadStream(filePath, {highWaterMark: chunkSize})` to align read size with multipart chunking. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit c528110. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
* updated eslint playwright plugin
ref https://linear.app/ghost/issue/NY-824 - this decouples the processing of the outbox from the member welcome emails - outbox has a handler pattern so it knows what to do based on certain events (like `MemberCreated`) - this will allow us to reuse the member welcome email sending elsewhere too, like a "test email" in the UI
ref https://linear.app/ghost/issue/BER-2995 ref TryGhost#25388 Private sites should not federate content via ActivityPub, as the content is intended to be restricted to authenticated users only - Added `is_private` check to `isSocialWebEnabled()` calculated setting - Added `is_private` to dependents array for cache invalidation - Added event listener for `settings.is_private.edited` - Updated Network settings UI to show contextual error message when disabled
) ref https://linear.app/ghost/issue/ONC-1305 When a member's subscription was cancelled while an admin had the member page open, saving the member (e.g. changing a note) would send stale tier data to the API. The backend interpreted this as adding a complimentary tier, incorrectly changing the member's status from 'free' to 'comped' The fix removes tiers from the member serializer payload since tier changes are handled through dedicated API calls (add/remove complimentary), not through normal member saves
closes https://linear.app/ghost/issue/BER-3087/make-admin-forward-feature-flag-work-for-self-hosters Copies the admin forward build assets (a superset of the Ember admin assets) to the location expected by Ghost core and conditionally serves the correct entrypoint based on the existing feature flag. The development Dockerfile and the archive task has been updated to correctly copy the assets as well.
ref https://linear.app/ghost/issue/NY-833/refine-design-details-for-utm-filtering - Post analytics header Y gap was too large - Title attribute of values was not set in the filter dropdown so when it's truncated - The filter row spacing was 1px off related to the sidebar - Further generic minor design improvements
ref https://linear.app/ghost/issue/NY-812/ - added new tests for analytics filters (all behind the utmTracking labs flag)
ref TryGhost@eb5ef8e CI failed to check this job and it merged early - this is a patch to the failing tests. We're looking separately on the fix to CI.
…5626) ref https://linear.app/ghost/issue/NY-773 - Updates member welcome emails so that they fetch content from `automated_emails` instead of hard-coded - Includes logic to check if the automated email is active - Cleans up some of the boundaries between member welcome emails and the outbox; outbox doesn't need to worry about the mail configuration anymore
ref https://linear.app/ghost/issue/NY-827/ - added api_top_devices - added device as a session-level field User agent/device is a session-level metric, so it's been included in the session_data mv. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Adds a top devices endpoint and promotes device to a session-level field with filtering, updating pipes, fixtures, and tests. > > - **Tinybird endpoints**: > - `endpoints/api_top_devices.pipe`: new endpoint returning `device` with visit counts; integrates with `filtered_sessions` and supports pagination. > - **Pipes/data model**: > - `pipes/mv_hits.pipe`: extract `device` from payload and default to `unknown`; remove UA-derived device; propagate `device` downstream. > - `pipes/mv_session_data.pipe`: aggregate and expose session-level `device` via `argMin`. > - `pipes/filtered_sessions.pipe`: add session-level filter `device` and update description. > - **Fixtures & tests**: > - `fixtures/analytics_events.ndjson`: add `device` field values (e.g., `desktop`, `bot`). > - Add `tests/api_top_devices.yaml`; extend `tests/api_kpis.yaml`, `tests/api_top_locations.yaml`, `tests/api_top_pages.yaml` with device-filter cases. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 992fc00. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
ref https://linear.app/ghost/issue/NY-836/add-minimaleditor-component-to-admin-x-design-system In the member emails section of settings, we need to allow the user to edit the content of the welcome email. Initially we'll just include basic text formatting, but soon we'll want to add additional cards, like buttons for call-to-actions. Currently this modal uses the HtmlEditor, which already existed and was used in e.g. newsletter settings to edit the footer content. This worked initially, but the HtmlEditor outputs HTML, not Lexical, which breaks when we try to wire it up to the API (which expects lexical). This PR: - Creates a new `KoenigEditorBase` component with the common boilerplate needed to launch an instance of the editor - Updates `HtmlEditor` to be a thin wrapper around `KoenigEditorBase` to add the HTML output functionality - Exports the `KoenigEditorBase` from the design system so we can create custom editor instances for other use-cases, i.e. for member welcome emails
ref https://linear.app/ghost/issue/NY-827/ ref TryGhost#25634 - added api_top_devices - enabled endpoint in tinybird service - passed through devices option in ghost stats endpoints This is the front end changes that accompany the Tinybird changes (see ref) to enable the Devices filter/dropdown in the Analytics > Web view.
…5614) ref https://linear.app/ghost/issue/NY-829/add-ability-to-toggle-each-type-of-welcome-email-onoff The Member Welcome Emails settings UI was previously static with no backend integration. Toggle states were not persisted and emails could not be created or managed. This connects the UI to the new automated_emails API endpoint, enabling users to toggle welcome emails on/off with state persisting across page refreshes. Different default Lexical content is used for free vs paid member emails. This doesn't yet allow the content to be edited; this will be included in a subsequent comment.
ref https://linear.app/ghost/issue/NY-830/add-ability-to-edit-welcome-emails-subject-content-and-sender-info The member welcome emails UI in settings currently allows the user to toggle the emails on/off, but does not allow them to edit the content or any other fields. This adds the ability to edit the content, subject, and sender info of the welcome emails after enabling them.
ref https://linear.app/ghost/issue/DES-1256/thousands-separator-missing-from-percentage-format-in-analytics - Thousands separator was missing for percentage values in Analytics charts
* Added i18n wrapping for the email footer "This message was sent from..." * Changed the string to include the final permiod, matching existing string used in comments.
…t#25641) ref https://linear.app/ghost/issue/NY-816/ ref TryGhost@cc4eee9 - ported stats-filter component to the Post Analytics (posts) app - added tests Unfortunately, given the current structure of our React apps, we have limited options on how to treat shared components. Until we merge these apps into one (ideally into `admin`), we'll need to copy components like this and do dual maintenance. The ref'd commit added the new stats filter popover to the Analytics page. This commit brings that over to Posts.
Introduces Nix flake with hermetic development shell and reproducible multi-arch Docker image builds. Replaces Docker Buildx with Nix + Cachix for faster, deterministic CI builds. - Add flake.nix with devShell (Node.js, MySQL, Redis, Mailpit) - Add docker.nix for multi-stage Docker image builds - Add process-compose.yaml for native service orchestration - Add GitHub Actions workflow for x86_64/ARM64 builds with Cachix - Add push-to-cachix app for binary cache management Builds are hermetic (no network access), content-addressed, and bit-for-bit reproducible. Multi-arch support enables ARM cloud deployments with 20-40% cost savings (AWS Graviton vs x86).
onethirtyfive
force-pushed
the
feat/nix-docker-builds
branch
from
3fc9a2e to
f30256f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduces Nix flake with hermetic development shell and reproducible multi-arch Docker image builds. Replaces Docker Buildx with Nix + Cachix for faster, deterministic CI builds.
Builds are hermetic (no network access), content-addressed, and bit-for-bit reproducible. Multi-arch support enables ARM cloud deployments with 20-40% cost savings (AWS Graviton vs x86).