Enable Collectors in Cloud#26261
Draft
thll wants to merge 9 commits into
Draft
Conversation
Covers the wait-for-config activation flow: launches when config is already present, waits then launches on a CollectorsConfig ClusterConfigChangedEvent, ignores unrelated config changes, retries launch on MisfireException, and the shutdown paths (clean exit while waiting; stops the input after launch). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
+ add cloud visibility swagger annotations
In Cloud the ingest endpoint is server-provisioned and there is no persisted ingest input, so the settings page now: - renders the hostname/port as read-only (disabled) fields - hides the create-input checkbox, port-mismatch alert, and input status - uses cloud-appropriate help and getting-started copy - thresholds stay editable Also guard useCollectorInputIds/useCollectorInputDetails with `enabled: !AppConfig.isCloud()` so the cloud-hidden /collectors/config/inputs endpoint is never called. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Cloud collector ingest input runs in-memory with no persisted Input record, so messages carry a synthetic source input id. Special-case it in FormatReceivedBy to render a "Collector Ingest" link to the Collectors page instead of the "deleted input" fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously, Collectors were not usable in Graylog Cloud, because it requires the Collector Ingest Input to run and be managed by Administrators.
This PR adds a background service that starts an ephemeral Ingest input once the Collectors settings page has been saved for the first time, which at the moment serves as the bootstrapping trigger for Collectors both on premise as in the Cloud.
The implementation aims at re-using most of the on-premise code, e.g. the actual input is launched by the service but not registered in the usual way, so that it is not visible. For that to work reliably, an artificial input ID had to be added. We also use this ID to render the "received by" indicator in the Messages UI to link to the collectors page.