Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
1c514a0
feat(network): emit canonical moderation ban payload
osp54 Apr 26, 2026
ade8c23
build(network): consume protocol from Maven
osp54 Apr 26, 2026
a9efd6a
feat(network): migrate moderation transport DTOs
osp54 Apr 27, 2026
fd61b2b
refactor(network): use direct moderation DTOs
osp54 Apr 27, 2026
3e339d4
refactor(event): remove dead moderation wrappers
osp54 Apr 28, 2026
e819e98
feat(network): migrate discord transport DTOs
osp54 Apr 28, 2026
61e5d38
feat(network): migrate maps list transport DTOs
osp54 Apr 28, 2026
d15240e
fix(moderation): support IP-only protocol targets
osp54 Apr 29, 2026
23f8b99
feat(network): migrate maps remove transport DTOs
osp54 Apr 29, 2026
847c35b
feat(network): migrate heartbeat transport DTOs
osp54 Apr 29, 2026
115ac4a
feat(network): migrate chat broadcast DTOs
osp54 Apr 29, 2026
bae65ce
feat(network): migrate discord ingress DTOs
osp54 Apr 29, 2026
dbd05e6
feat(network): migrate private message DTOs
osp54 Apr 30, 2026
313c658
feat(network): finish phase5 transport DTO cutover
osp54 Apr 30, 2026
b85120f
feat(network): migrate maps load command DTO
osp54 Apr 30, 2026
a9799a9
refactor(network): drop dead legacy maps transport types
osp54 Apr 30, 2026
b432348
feat(network): migrate player session sync DTOs
osp54 May 1, 2026
8d8cc11
feat(network): migrate badge inventory and password reset command DTO…
osp54 May 1, 2026
b76d5b1
feat(network): migrate final chat command DTOs to V1
osp54 May 2, 2026
3e66238
refactor(network): align plugin with protocol enums
osp54 May 2, 2026
d27ca8c
build(ci): refresh Gradle dependencies in workflows
osp54 May 2, 2026
8f59525
test(network): stabilize Redis moderation integration assertions
osp54 May 2, 2026
a3830a2
fix(network): preserve heartbeat discord channel width
osp54 May 2, 2026
75b6579
feat(protocol): migrate to xcore-protocol 0.4.0 with canonical serial…
osp54 May 3, 2026
1e0a8ae
refactor(protocol): align plugin transport with canonical actor seman…
osp54 May 3, 2026
36be5f9
refactor(protocol): remove legacy transport fallbacks, finalize canon…
osp54 May 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ jobs:

- name: Build and test
if: github.event_name == 'pull_request' || env.XCORE_USERNAME == '' || env.XCORE_PASSWORD == ''
run: ./gradlew test shadowJar
run: ./gradlew --refresh-dependencies test shadowJar

- name: Build, test and publish snapshot
if: github.event_name != 'pull_request' && env.XCORE_USERNAME != '' && env.XCORE_PASSWORD != ''
run: ./gradlew test publishMavenPublicationToXcoreRepositorySnapshotsRepository -PxcorePublishVersion="$XCORE_PUBLISH_VERSION"
run: ./gradlew --refresh-dependencies test publishMavenPublicationToXcoreRepositorySnapshotsRepository -PxcorePublishVersion="$XCORE_PUBLISH_VERSION"
env:
ORG_GRADLE_PROJECT_xcoreRepositorySnapshotsUsername: ${{ secrets.XCORE_USERNAME }}
ORG_GRADLE_PROJECT_xcoreRepositorySnapshotsPassword: ${{ secrets.XCORE_PASSWORD }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
echo "XCORE_PUBLISH_VERSION=${VERSION}" >> "$GITHUB_ENV"

- name: Build, test and publish release
run: ./gradlew test publishMavenPublicationToXcoreRepositoryReleasesRepository -PxcorePublishVersion="$XCORE_PUBLISH_VERSION"
run: ./gradlew --refresh-dependencies test publishMavenPublicationToXcoreRepositoryReleasesRepository -PxcorePublishVersion="$XCORE_PUBLISH_VERSION"
env:
ORG_GRADLE_PROJECT_xcoreRepositoryReleasesUsername: ${{ secrets.XCORE_USERNAME }}
ORG_GRADLE_PROJECT_xcoreRepositoryReleasesPassword: ${{ secrets.XCORE_PASSWORD }}
Expand Down
3 changes: 3 additions & 0 deletions build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ val xcoreReleasesRepositoryUrl = providers.gradleProperty("xcoreMavenReleasesUrl
.orElse("https://maven.x-core.org/releases")

repositories {
maven { url = uri("https://maven.x-core.org/snapshots") }
maven { url = uri("https://maven.x-core.org/releases") }
mavenCentral()
anukeXpdustry()
maven(url = "https://oss.sonatype.org/content/repositories/snapshots")
Expand All @@ -62,6 +64,7 @@ dependencies {
compileOnly(toxopid.dependencies.mindustryCore)
compileOnly(toxopid.dependencies.arcCore)
compileOnly(toxopid.dependencies.mindustryHeadless)
implementation(libs.xcore.protocol.java)
implementation(libs.flubundle)
implementation(libs.cloud.mindustry)
implementation(libs.mongodb.sync)
Expand Down
119 changes: 119 additions & 0 deletions docs/adr/ADR-redis-to-protocol-first.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
# ADR: Move XCore transport contracts to a protocol-first model

## Status
Proposed

## Context
`XCore-plugin` and `XCore-discord-bot` currently share Redis-based message contracts for events, commands, and RPC-style request/response flows. Those contracts work in practice, but the protocol surface is split across multiple implementation-specific locations:

- Java transport route metadata and envelope construction in `XCore-plugin`
- Java transport event/request/response records in `TransportEvents`
- Python pydantic contract models and Redis bus logic in `XCore-discord-bot`
- Compatibility behavior encoded through aliases, legacy event names, and runtime fallback logic

This creates several problems:

1. The protocol exists as an accidental agreement between implementations instead of an explicit source of truth.
2. Cross-language compatibility depends on tolerant readers, duplicate field names, and historical knowledge.
3. Internal domain/storage models can leak into the wire format.
4. Route metadata and transport semantics are hard to evolve safely across repos.
5. Future consumers would have to reverse-engineer the protocol from application code.

## Decision
Adopt a **protocol-first** model and define a future shared repository named **`xcore-protocol`** as the canonical source of truth for XCore cross-process communication.

`xcore-protocol` will own:

- wire-level message schemas
- envelope definitions
- route/stream/RPC metadata
- message versioning and compatibility policy
- canonical fixtures/examples
- generated Java and Python protocol DTO/model artifacts
- thin handwritten validation/runtime support around generated artifacts
- cross-language compatibility tests

The first implementation step is **documentation-first** inside `XCore-plugin`, followed by a phased migration into the future `xcore-protocol` repository.

## Why `xcore-protocol`
`xcore-protocol` was chosen over names like `xcore-transport` or `xcore-contracts` because it best reflects the intended boundary:

- broader than raw schema files alone
- not permanently tied to Redis internals
- centered on the official language of communication between XCore components

## Scope Boundaries
`xcore-protocol` is intended to contain only **cross-process / cross-service wire protocol artifacts**.

It should include:

- event, command, and RPC message definitions
- envelope metadata definitions
- protocol validation helpers and fixtures
- route metadata and compatibility policy
- generated Java/Python DTOs and models derived from protocol specs
- thin Java/Python support libraries for parsing/building/validation around generated artifacts

It should not include:

- application business logic
- Discord UX or handlers
- Mongo repositories
- Mindustry runtime integration
- reconnect loops or app-specific worker orchestration
- general shared helper dumping grounds

## Contract Strategy
The immediate protocol redesign will:

1. Normalize canonical field names, time formats, and versioning rules.
2. Keep semantically distinct business messages separate.
3. Extract shared payload subtypes rather than merging unrelated messages by shape.
4. Move legacy aliases and historical event-name compatibility into dedicated compatibility adapters.
5. Stop treating internal application models as the public wire contract.
6. Generate Java and Python protocol model layers from canonical specs instead of maintaining duplicate hand-written wire DTOs in consumer repos.

## Rollout Strategy
Migration will start with the **moderation** contract family because it already crosses repository boundaries and shows the clearest compatibility pain.

Phases:

1. Documentation and target-state design in `XCore-plugin`
2. Bootstrap `xcore-protocol`
3. Migrate moderation contracts first
4. Migrate Discord linking/admin contracts
5. Migrate maps RPC contracts
6. Migrate chat/heartbeat/misc flows and clean legacy handling

## Consequences

### Positive
- One source of truth for cross-language protocol behavior
- Safer contract evolution with explicit review and compatibility checks
- Cleaner boundaries between domain models and wire models
- Better onboarding path for future consumers and future agents
- Clear governance for breaking vs additive changes
- Less DTO drift between Java and Python consumers through generated protocol artifacts

### Costs
- Requires initial design effort and documentation discipline
- Introduces a new repository and release process
- Needs explicit ownership and change governance
- Requires migration adapters during the transition period

## Alternatives Considered

### 1. Keep protocol ownership in `XCore-plugin`
Rejected as the target state because it keeps Python and future consumers secondary to a Java implementation repo.

### 2. Create a schema-only repository
Improves the current state but still leaves Java/Python wire DTOs and model layers duplicated and easier to drift.

### 3. Move the full ecosystem into a single monorepo
Rejected because the problem boundary is the shared protocol surface, not the full application estate. A full monorepo would impose much higher coordination cost than necessary.

## Acceptance Criteria For This Decision
- Documentation clearly defines target-state protocol ownership and boundaries.
- Documentation clearly defines generated protocol DTO/model ownership and consumer dependency direction.
- Future implementation work can proceed without re-deciding repo naming, scope, or migration direction.
- Moderation-first migration remains the agreed first rollout slice.
169 changes: 169 additions & 0 deletions docs/architecture/xcore-protocol-message-model.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# XCore Protocol Message Model

## Goal
Define the canonical message model for the future `xcore-protocol` repository, including envelope rules, payload conventions, naming, versioning, and compatibility handling.

## Core Principle
The public wire contract must not be an accidental serialization of internal application models. Protocol messages are explicit transport DTOs with stable meaning, and Java/Python protocol DTO/model layers should be generated from the canonical protocol definitions rather than hand-maintained independently in consumer repos.

## Message Categories

### Event
- broadcast or fan-out notification
- producer does not wait for a response
- usually replayable for observers depending on stream retention

### Command
- targeted instruction toward a specific server or logical target
- producer does not wait for a response
- typically non-replayable as a business action

### RPC Request / Response
- request expects a response
- request and response are linked by correlation metadata
- timeouts and error semantics are part of the protocol contract

## Envelope Model

### Long-term target
The protocol should converge on a unified envelope model with explicit metadata:

- `message_kind`
- `message_type`
- `message_version`
- `message_id`
- `correlation_id` (when needed)
- `causation_id` (recommended when derived from another message)
- `producer`
- `target` (for targeted messages)
- `created_at`
- `expires_at` (when relevant)
- `schema_ref`
- `content_type`
- `payload_json`

### Transition note
The current Redis field layout in `XCore-plugin` and `XCore-discord-bot` can be preserved through a migration window, but the target model must be documented explicitly now.

## Naming Rules

### Canonical policy
- Envelope fields use **snake_case**.
- Payload fields use **camelCase**.

### Examples
- Envelope: `message_type`, `created_at`, `correlation_id`
- Payload: `playerUuid`, `adminDiscordId`, `occurredAt`

### Non-goal
The protocol must not treat multiple spellings of the same canonical field as equal in the schema. Legacy spellings are compatibility concerns, not canonical contract design.

## Time Rules

### Envelope metadata
Use epoch milliseconds for transport metadata:
- `created_at`
- `expires_at`
- `responded_at`

### Payload business timestamps
Use ISO-8601 UTC for business timestamps unless the message family has a strong reason not to.

### Rationale
This keeps transport metadata simple for timeouts/retention and keeps business timestamps readable and consistent across languages.

## Message Identity And Versioning

### Canonical identity
Every message must have:
- `messageType`
- `messageVersion`

Recommended examples:
- `moderation.ban.created` / version `1`
- `discord.link.confirm.command` / version `1`
- `maps.list.request` / version `1`

### Rule
Breaking changes require a new message version. Do not change meaning in place.

## Generated Model Strategy

### Source of truth
Canonical schemas, shared subtypes, envelope definitions, and route manifests are the authored source of truth.

### Generated outputs
`xcore-protocol` should generate Java and Python protocol DTO/model artifacts from those canonical definitions.

### Consumer rule
Application repos should depend on generated protocol artifacts and keep only thin mapping/adaptation layers between internal models and wire models.

### Non-goal
Do not generate runtime worker loops, Redis connection management, or business orchestration from protocol definitions.

## Shared Payload Subtypes
To improve consistency without merging unrelated business messages, define reusable subtypes:

- `ActorRef`
- `PlayerRef`
- `ServerRef`
- `DiscordIdentityRef`
- `ExpirationInfo`
- `MapRef`
- `AuditContext`

These subtypes should be reused across schemas where they model the same concept.

## Contract Strategy

### Normalize now
- canonical field names
- canonical time formats
- message identity/versioning
- canonical route metadata

### Keep separate
Semantically distinct business messages should remain distinct even if they share many fields.

Examples that should remain separate:
- ban vs mute
- command vs event around Discord linking
- maps list vs maps remove RPC

### Use compatibility adapters
Legacy event names, duplicate spellings, and historical payload forms should move into explicit compatibility adapters.

## Compatibility Rules

### Canonical outbound rule
All new producers publish only the canonical schema form.

### Tolerant inbound rule
Consumers may temporarily accept historical forms through dedicated compatibility logic, but canonical parsing must remain strict.

### Legacy sunset rule
Compatibility shims must be documented with a deprecation window and test coverage.

## Route Manifest Philosophy
Each message definition should include or link to route metadata describing:

- stream/channel pattern
- message kind
- target scope
- TTL policy
- idempotency expectations
- replay expectations
- DLQ policy
- owner

The route manifest becomes the single source of truth for subscription/publish semantics and should feed generated route/metadata bindings exposed by the protocol repository.

## Immediate Families To Model First
- moderation
- Discord linking/admin changes
- maps RPC
- chat/heartbeat after the initial migration wave

## Success Criteria
- A future agent can implement or generate transport DTO/model layers without deciding naming, timing, or versioning policy on the fly.
- The model is strict enough to remove accidental drift but flexible enough to support compatibility adapters during migration.
Loading
Loading