Skip to content

Notify on OpenSearch Sync Failures in Harvester #5849

@FuhuXia

Description

@FuhuXia

After each record update in the harvester, a sync step sends index updates to OpenSearch. If this step fails, the process logs an error and continues, leaving the database and OpenSearch out of sync. Although a nightly GitHub task attempts to reconcile them, such failures should trigger alerts so developers can investigate promptly rather than allowing issues to go unnoticed.

  • Log messages should clearly indicate when an OpenSearch sync fails.
  • Each failed record should generate a single log entry to quantify impact.
  • New Relic should send alerts to the Slack channel when failures exceed a defined threshold.

Metadata

Metadata

Assignees

No one assigned

    Labels

    O&MOperations and maintenance tasks for the Data.gov platformbugSoftware defect or bug

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status

    O&M Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions