Define push notification protocol so repositories can proactively notify aggregators of new/updated packages

[toderash]

Currently, aggregators discover packages by crawling repositories, which follows the current AspireCloud model and per the protocol design as documented in Draft PR #3.

This presents (at least?) three issues:

1. Crawl latency: new packages aren't discoverable until the next crawl cycle.
2. Crawler infrastructure: every aggregator must maintain its own crawl scheduler and handle crawl failures, rate limiting, and backpressure.
3. Ecosystem flexibility: the design assumes all aggregators are gated, and that repositories don't know where they're aggregated.

Proposed: repository-to-aggregator push notification

A simple push protocol would let repositories notify registered aggregators immediately when a package or release is created or updated. The aggregator then fetches the updated record on demand rather than on a schedule. This significantly reduces needless polling of infrequently-updated package repos.

Proposed endpoint on aggregators (POST /intake):

{
  "type": "package.updated",
  "did": "did:plc:ia6vk5krwkcka2nwuzs6l6lq",
  "repository": "https://example.fair.pm/packages/1234",
  "timestamp": "2026-04-26T12:00:00Z"
}

The notification is a hint only — it contains no authoritative data, just the DID and repository URL. The aggregator fetches the actual metadata from the repository to verify and index it. A malicious notification can't inject bad data; it can only cause the aggregator to fetch from the repository, which it would have done on the next crawl anyway. To ensure this, the repository URI MUST match the previous one, and the aggregator MUST contact the canonical repo URL.

Aggregator registration:

Repositories need to know which aggregators to notify. We may need more than one option to address different ecosystems. These include:

1. Repositories go through an application process for Aggregators to be listed (current assumption).
2. Repositories notify Aggregators via public API, requesting indexing, which can happen automatically based on set criteria.
3. Aggregators register a webhook URL with repositories they want notifications from.
4. Repositories notify all aggregators in the FAIR aggregator directory - this requires that a directory spec exists. Rather than a central registry, this may be designed as a federated protocol like ActivityPub or perhaps closer to our interests, ATproto (perhaps a better fit here).

Event types to define now:

• package.created — new package published
• package.updated — package metadata updated
• release.created — new release published
• package.deleted — package removed (tombstone)
• release.deleted — release removed

Relation to issue #48: This issue extends the scope of #48 from "aggregator-initiated discovery" to "repository-initiated notification." Both mechanisms need to exist. In addition to adding robustness, crawl support ensures that private aggregators can still reach the repo, even if they're behind a firewall that blocks the aggregator's API to external traffic.

[ascorbic]

I think ATProto would be a great solution here, and would work well with our plans for EmDash. A publisher (per #76) adds an #atproto_pds service to its DID document. This could be any PDS (Bluesky, npmx, tangled, Eurosky etc or self-hosted), or FAIR could provide them. When creating a package, a publisher creates a new pm.fair.package.profile entry, which contains the package metadata. Whenever there is a release, it creates a pm.fair.package.release entry. The PDS would then automatically emit #commit events for any of these changes which would be picked up by relays. This would also happen for other CRUD events. We'd get all of this for free from the existing atproto network, and existing atproto implementations handle all the aggregation. We'd just need to define lexicons for existing FAIR types. This also fits nicely into the extension model: these could be defined under each extension's own namespace and linked to the records.

This would be additive for aggregators: they could choose to subscribe to the atproto firehose, filtered by pm.fair.* record, and would automatically get the record updates, and new aggregators could backfill any existing records. All of these would be verified because they'd be signed by the publisher's signing key. They would effectively be acting as AppViews in the atproto jargon. This could be in addition to or instead of existing FAIR aggregation – each aggregator could choose.

This could also be the way to handle labellers as they use the same network and could discover packages and publish labels in the same way.

[toderash]

Yes, this opens up some interesting avenues. For FAIR, we've explicitly started on the far end of the spectrum from the firehose, but that's largely due to onboarding a legacy system where we need to protect provenance from a centralized repo to a decentralized one. For this reason, the curated Aggregator option needs to persist, but in a new ecosystem, that's an entirely different concern. #76 establishes a migration path for that, essentially codifying a rough design we've had in mind from the beginning - it just wasn't in the protocol from the discussions. There are some additional topics like that where decisions are still open, so it's not codified yet since it's pretty hard to step back once you've put it into the spec.