diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 364d13e838c..34dcf17aee8 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -202,6 +202,7 @@
/packages/cyera @elastic/security-service-integrations
/packages/cylance @elastic/security-service-integrations
/packages/darktrace @elastic/security-service-integrations
+/packages/dataminr_pulse @elastic/security-external-integrations
/packages/ded @elastic/sec-applied-ml @elastic/kibana-management
/packages/dga @elastic/sec-applied-ml
/packages/digital_guardian @elastic/security-service-integrations
diff --git a/packages/dataminr_pulse/README.md b/packages/dataminr_pulse/README.md
new file mode 100644
index 00000000000..2cbb909540f
--- /dev/null
+++ b/packages/dataminr_pulse/README.md
@@ -0,0 +1,284 @@
+# Dataminr Pulse Integration User Guide
+
+# Overview
+
+### From Alerts to Action—Dataminr Pulse for Elastic Security
+
+Embed Dataminr Pulse real-time, actionable intelligence directly into Elastic Security. Transform the earliest external threat signals from over 1.1 million public, deep, and dark web sources into Elastic-native detections, enriched indices, and automated workflows.
+
+The Dataminr Pulse integration for Elastic Security seamlessly bridges the gap between external real-time data and your internal security operations. By leveraging the Elastic Agent with a CEL (Common Expression Language) input, the integration polls the Dataminr Pulse v4 API at configurable intervals—automatically managing OAuth token refresh and cursor-based pagination to ensure a continuous, resilient data flow.
+
+### Turn Signal Overload into Real-Time, AI-Powered Intelligence
+
+Stay ahead of the threat curve and be the first to see rapidly emerging and evolving threats, vulnerabilities, exploits, ransomware activity, third-party incidents, and more—often hours or days before traditional sources.
+
+**Unmatched Coverage, Precision, and Granularity**
+
+With Dataminr Pulse for Cyber Risk, security teams gain a critical time advantage. Dataminr processes more than 45 terabytes of daily public data, leveraging over 55 proprietary LLMs and 15 years of historic alerting information. With multimodal fusion AI, GenAI, and Agentic AI deeply embedded into the platform, security teams can:
+
+* **Dynamically detect and defend** digital assets beyond the perimeter.
+* **Unearth hidden threats** and close blind spots with advanced processing of text, images, video, and machine signals.
+* **Leverage Agentic AI-powered Intel Agents** to autonomously assemble adversary context, including TTPs, IOCs, CVEs, and MITRE ATT\&CK® mappings.
+* **Proactively prioritize and patch** fast-breaking vulnerabilities and exploits.
+
+
+**Accelerate Elastic Workflows with Actionable Context**
+
+* **Native ECS Mapping:** Ingested alerts are automatically mapped to the Elastic Common Schema (ECS) and stored in the logs-dataminr\_pulse.alerts-\* data stream for immediate correlation.
+* **Granular Entity Expansion:** The integration automatically expands alerts containing multiple discovered entities, such as vulnerabilities, threat actors, malware, IP addresses, and URLs into individual documents for deep, granular analysis.
+* **Integrated Detection & Response:** Populate Elastic Security signals and enrich indices to power advanced hunting, Kibana Dashboards, and Elastic Defend workflows.
+
+
+**Address Your Critical Use Cases with Dataminr**
+
+| Use Case | How Dataminr & Elastic Work Together |
+| :---- | :---- |
+| **Cyber Threat Intelligence** | Piece together attack context with crucial details about threat actors and malware directly within the Elastic Security Threat Intel view and custom dashboards specific to your Dataminr topics. |
+| **Vulnerability Prioritization** | See the earliest signals of PoC exploitation to prioritize patching within the Elastic Vulnerability Management framework. |
+| **Third-Party Risk** | Instantly identify and track supply chain attacks and vendor disruptions as they unfold in real-time. |
+| **Digital Risk Protection** | Spot credential dumps, phishing attempts, and brand impersonations involving your digital footprint. |
+| **Cyber-Physical Convergence** | Assess the complete blast radius of physical events and coordinate a unified response to converged threats. |
+
+# Dependencies
+
+Before installing, ensure the following requirements are met.
+
+## Dataminr Dependencies
+
+| Dependency | Requirement |
+| :---- | :---- |
+| Active Dataminr Pulse API account | Required |
+| Client ID | Required |
+| Client Secret | Required |
+| Dataminr Pulse API version | v4 |
+
+## Elastic Security Dependencies
+
+| Dependency | Requirement |
+| :---- | :---- |
+| Elastic Stack version | 8.13.0 or newer |
+| Kibana version | 8.13.0 or newer |
+| Elastic Agent version | 8.13.0 or newer |
+| Elastic subscription | Basic or higher |
+
+## Network Requirements
+
+The host running the Elastic Agent must have outbound HTTPS (port 443\) access to the following endpoints:
+
+| Destination | Purpose |
+| :---- | :---- |
+| api.dataminr.com | For authentication and to fetch alerts from Pulse API |
+| Fleet Server URL | Agent enrollment and policy management within the clients systems |
+
+**Note:** No inbound ports are required on the agent host.
+
+# Set up
+
+## Install the Integration
+
+1. Log in to elastic and navigate to the **Integrations** page under **Data Management**.
+2. Search for **Dataminr Pulse** in the integrations catalog.
+3. Click the **Dataminr Pulse** integration card, then click **Add Dataminr Pulse**.
+
+## Configure the Integration
+
+#### Dataminr Configuration
+
+To use this integration, you need a Dataminr Pulse API account with a valid Client ID and Client Secret. Contact your Dataminr account representative to obtain the API credentials.
+
+**Important:** Keep your Client Secret secure. It will be stored as a secret value in the Elastic Agent policy and will not be visible after saving.
+
+#### Elastic Configuration
+
+#### **Step 1: Configure and Deploy the Integration**
+
+After clicking **Add Dataminr Pulse** from the integrations page, fill in the configuration form.
+
+1. Browse to the data management and select integrations.
+2. Search ‘Dataminr Pulse’ to install the integration and follow the onscreen instructions.
+3. Optionally, configure the **Polling Interval**, and **Page Size**. The defaults work for most deployments.
+4. Select an existing **Agent policy** or create a new one.
+5. Click **Save and continue**, then **Save and deploy changes**.
+
+#### **Configuration Parameters**
+
+The table below describes all available configuration parameters.
+
+| Parameter | Description | Required | Default |
+| :---- | :---- | :---- | :---- |
+| Integration name | A descriptive name visible on the Elastic Agent policy. | No | dataminr\_pulse |
+| API URL | The full URL for the Dataminr Pulse alerts endpoint. | Yes | https://api.dataminr.com/pulse/v1/alerts |
+| Client ID | Your Dataminr API Client ID for OAuth token generation. | Yes | \- |
+| Client Secret | Your Dataminr API Client Secret for OAuth token generation. Stored securely. | Yes | \- |
+| Interval | How often the integration polls the Dataminr API for new alerts (e.g., 5m, 1m, 10m). | Yes | 5m |
+| Page Size | Maximum number of alerts returned per API request. The maximum allowed value is 100\. | Yes | 40 |
+| Tags | Custom tags applied to each ingested event. | No | forwarded, dataminr-pulse-alerts |
+| Preserve original event | When enabled, stores the raw API response in the event.original field. Useful for debugging. | No | false |
+| Enable request tracing | Logs full HTTP request/response details for debugging. Do not enable in production \- this logs credentials in plain text. | No | false |
+| Processors | Custom Elastic Agent processors in YAML format, applied before data is sent to Elasticsearch. | No | \- |
+
+#### **Configuration Performance Recommendations**
+
+* **Polling Interval**: Start with the default of 5m. Reduce to 1m only if you need near-real-time alert ingestion and your Dataminr API quota allows it.
+* **Page Size**: The default of 40 works well for most deployments. Increase up to 100 if you expect high alert volumes to reduce the number of API calls.
+* **Preserve original event**: Keep disabled in production. Enabling it roughly doubles the storage per document.
+
+#### **Step 2: Validate the Data Flow**
+
+After deploying the integration, verify that data is flowing correctly.
+
+##### **Fleet**
+
+1. Navigate to **Assets** \> **Fleet**.
+2. Under **Agents** locate your enrolled agent.
+3. Verify the agent status is **Healthy** (green).
+4. Click the agent name, then click the **Logs** tab.
+5. Look for log entries showing successful execution of CEL script within agent (Ex: “Unit state changed cel-default (STARTING-\>HEALTHY): Healthy”)
+
+##### **Discover**
+
+1. Navigate to **Discover** and create a session
+2. Set the **Index pattern** to logs-dataminr\_pulse.alerts\* under **Data view**
+3. Select the time to be the last hour, and confirm documents are appearing.
+
+##### **Index Management**
+
+1. Navigate to **Data Management** \> **Streams**
+2. Search for logs-dataminr\_pulse.alerts\*. Verify the index exists and the document count is increasing.
+
+##### **Dashboards**
+
+1. Navigate to **Dashboards**.
+2. Search for **Dataminr**. The integration includes pre-built dashboards for alert monitoring.
+3. Open a dashboard and verify it displays data.
+
+## Delete the Integration
+
+To remove the Dataminr Pulse integration from an agent policy:
+
+1. Navigate to **Assets** \> **Fleet \> Policies**
+2. Click the policy that contains the Dataminr Pulse integration.
+3. Locate the **Dataminr Pulse** integration entry and click the **Actions** menu (three dots), then select **Delete integration**.
+
+**Note:** Deleting the integration from the policy stops data collection but does not remove already-ingested data.
+
+## Reset Integration Assets
+
+If integration assets (dashboards, index templates, ingest pipelines) become corrupted or out of sync, you can reset them.
+
+1. Navigate to **Data Management** \> **Integrations**.
+2. Click **Dataminr Pulse**.
+3. Select the **Settings** tab.
+4. Click **Reinstall Dataminr Pulse**. This reinstalls dashboards, index templates, and ingest pipelines to their default state.
+
+# Data Mappings
+
+The integration maps Dataminr Pulse alert fields to Elastic Common Schema (ECS) fields. Custom fields are stored under the dataminr\_pulse namespace for reference.
+
+## ECS Field Mappings
+
+| Dataminr Pulse Field | ECS Field | Description |
+| :---- | :---- | :---- |
+| Alert timestamp | @timestamp | Event timestamp |
+| Alert headline | message | Single-sentence event summary |
+| Alert ID | event.id | Unique alert identifier |
+| Alert creation time | event.created | When the alert was created |
+| Alert priority (Alert, Urgent, Flash) | event.severity | Numeric severity (10, 20, 30\) |
+| Dataminr alert URL | event.url | Link to alert in Dataminr platform |
+| Dataminr alert location coordinates | geo.location | Coordinates of the Dataminr alert |
+| Dataminr alert location name | geo.name | address of the Dataminr alert |
+| Dataminr entity category | event.category | Categories \- Threat Actor, Vulnerability, Malware |
+| Threat actor name | threat.group.name | Threat actor name (MITRE ATT\&CK) |
+| Threat actor aliases | threat.group.alias | Threat actor alternative names |
+| Threat actor country of origin | threat.indicator.geo.country\_iso\_code | Country of Origin for threat actors |
+| CVE ID | vulnerability.id | CVE identifier |
+| CVSS score | vulnerability.score.base | CVSS base score |
+| Vulnerability description | vulnerability.description | Summary of the vulnerability |
+| Type to distinguish URL vs IP IOC | threat.enrichments\[\].indicator.type | Values \- ip4-ddr, ip6-addr, url |
+| URL | Threat.enrichments\[\].indicator.url.original | URL discovered to be related to the alert, as in the original form |
+| URL or IP addresses | threat.enrichments\[\].indicator.name | URL/IP addresses discovered to be related to the alert. |
+| IP Addresses | threat.enrichments\[\].indicator.ip | IP Addresses discovered to be related to the alert. |
+| Port | threat.enrichments\[\].indicator.port\[\] | Ports discovered to be associated with the IP addresses above |
+
+## Custom Dataminr Fields
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| dataminr\_pulse.categories.name | keyword | Alert topic categories |
+| dataminr\_pulse.companies.name | keyword | Affected company names |
+| dataminr\_pulse.sectors.name | keyword | Industry sectors |
+| dataminr\_pulse.source.href | keyword | URL to the public source post |
+| dataminr\_pulse.source.channels | keyword | Source channels (e.g., sensor) |
+| dataminr\_pulse.source.media.href | keyword | Media attachment URLs |
+| dataminr\_pulse.intel\_agents.summary | keyword | AI-generated critical context summary |
+| dataminr\_pulse.watchlists\_matched\_by\_type.name | keyword | Matched watchlist names |
+| dataminr\_pulse.alert\_type.name | keyword | Alert priority level (Alert, Urgent, Flash) |
+| dataminr\_pulse.live\_brief.summary | keyword | AI-generated event summary |
+| dataminr\_pulse.live\_brief.version | keyword | Live Brief version |
+| dataminr\_pulse.live\_brief.timestamp | date | Live Brief generation timestamp |
+| Dataminr\_pulse.threatactor | keyword | Threat actors discovered in the alert |
+| Dataminr\_pulse.threatactor.alias | keyword | Alternative names of threat actors discovered in the alert |
+| dataminr\_pulse.threatactor.country\_of\_origin | keyword | Country of origin for threat actors discovered in the alert |
+| dataminr\_pulse.vulnerability.name | keyword | Vulnerability identifiers (CVE IDs) |
+| Dataminr\_pulse.event.malware | keyword | Malwares discovered in the alert |
+| dataminr\_pulse.platforms | keyword | Operating systems that were discovered to be impacted because of the malwares |
+| Dataminr\_pulse.url | keyword | URLs discovered to be impacted |
+| Dataminr\_pulse.ip | keyword | IP address (for IP-type entities) |
+
+## Operational Fields
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| dataminr\_pulse.log.log\_type | keyword | Values: “alert-fetch” or “auth” |
+| dataminr\_pulse.log.api\_endpoint | keyword | Full API used to fetch alerts or for authentication |
+| dataminr\_pulse.log.http\_status\_code | keyword | HTTP response for the API call |
+| dataminr\_pulse.log.fetched\_alerts | long | Number of alerts fetched in the batch |
+| dataminr\_pulse.log.fetch\_timestamp | date | Timestamp when the Alert API call was made |
+| dataminr\_pulse.log.next\_cursor | keyword | Pagination cursor to be used in next batch |
+| dataminr\_pulse.log.status | keyword | If the iteration failed or succeded |
+
+# Troubleshooting
+
+## Enable Request Tracing
+
+Request tracing logs full HTTP request and response details, which is useful for diagnosing connectivity or authentication issues.
+
+**Important:** Request tracing logs credentials in plain text. Only enable it temporarily for debugging and disable it immediately after.
+
+1. Navigate to **Assets** \> **Fleet \> Policies**
+2. Click the policy that contains the Dataminr Pulse integration.
+3. Locate the **Dataminr Pulse** integration entry and click the **Actions** menu (three dots), then select **Edit integration**.
+4. Under advanced settings, set **Enable request tracing** to true.
+5. Click **Save and deploy changes**.
+6. To view traces, navigate to **Assets** \> **Fleet** \> **Agents**
+7. Click the agent, then click **Actions** \> **Request diagnostics**.
+8. Download the diagnostics bundle and examine the HTTP trace logs in the agent log files.
+
+### Common Errors
+
+| HTTP Status | Error | Explanation |
+| :---- | :---- | :---- |
+| 400 | Bad Request | The API URL is malformed or a request parameter is invalid. Verify the API URL and Base URL fields. |
+| 401 | Unauthorized | Authentication failed. Verify your Client ID and Client Secret are correct and the account is active. |
+| 403 | Forbidden | The API credentials do not have permission to access the requested resource. Contact your Dataminr account representative. |
+| 404 | Not Found | The API endpoint URL is incorrect. Ensure the API URL is set to https://api.dataminr.com/pulse/v1/alerts. |
+| 429 | Too Many Requests | API rate limit exceeded. Increase the Interval value or reduce the Page Size. |
+
+### No Data Appearing
+
+1. Check Agent Status: Navigate to Fleet \> Agents and verify the agent is Healthy.
+2. Check Agent Logs: Click the agent and review the Logs tab for error messages.
+3. Test Credentials: On the agent host, run:
+ 1. curl \-X POST [https://userauth.dataminr.com/auth/2/token](https://userauth.dataminr.com/auth/2/token) \-H "Content-Type: application/x-www-form-urlencoded" \-d "grant\_type=api\_key\&client\_id=YOUR\_ID\&client\_secret=YOUR\_SECRET"
+ 2. A successful response returns a JSON object with dmaToken and expire fields.
+4. Check Data Stream: In Dev Tools, run:
+ 1. GET logs-dataminr\_pulse.alerts-\*/\_count
+ 2. If the count is 0, the integration is not receiving data from the API. Review the agent logs for details.
+
+### Duplicate Alerts
+
+The integration uses document fingerprinting to prevent duplicates. If you observe duplicate documents:
+
+1. Verify the ingest pipeline is installed by running in Dev Tools:
+ 1. GET \_ingest/pipeline/logs-dataminr\_pulse.alerts-\*
+ 2. If missing, reset integration assets (see **Reset Integration Assets**).
\ No newline at end of file
diff --git a/packages/dataminr_pulse/_dev/build/build.yml b/packages/dataminr_pulse/_dev/build/build.yml
new file mode 100644
index 00000000000..e2b012548e0
--- /dev/null
+++ b/packages/dataminr_pulse/_dev/build/build.yml
@@ -0,0 +1,3 @@
+dependencies:
+ ecs:
+ reference: git@v8.11.0
diff --git a/packages/dataminr_pulse/_devtests/program.cel b/packages/dataminr_pulse/_devtests/program.cel
new file mode 100644
index 00000000000..1c623ab0fc6
--- /dev/null
+++ b/packages/dataminr_pulse/_devtests/program.cel
@@ -0,0 +1,223 @@
+timestamp(now).as(fetch_ts,
+ [408, 429, 500, 502, 503, 504, 401].as(retryable,
+ (state.access_token == "").as(no_token,
+ (int(state.expires_at) > 0).as(has_expiry,
+ (has_expiry ?
+ fetch_ts >= (timestamp("1970-01-01T00:00:00Z") +
+ duration(string(int(state.expires_at) - 300000) + "ms"))
+ : true
+ ).as(is_expired_or_unknown,
+ (no_token || is_expired_or_unknown).as(need_auth,
+ need_auth ?
+ post(
+ "https://userauth.dataminr.com/auth/2/token",
+ "application/x-www-form-urlencoded",
+ {
+ "grant_type": ["api_key"],
+ "client_id": [state.client_id],
+ "client_secret": [state.client_secret]
+ }.format_query()
+ ).as(auth_resp,
+ auth_resp.StatusCode == 200 ?
+ (size(auth_resp.Body) > 0 ? bytes(auth_resp.Body).decode_json() : {}).as(token_data,
+ has(token_data.dmaToken) && token_data.dmaToken != "" ?
+ {
+ "updated_state": state.with({
+ "access_token": string(token_data.dmaToken),
+ "expires_at": has(token_data.expire) ? int(token_data.expire) : 0
+ }),
+ "auth_event": {
+ "message": {
+ "log_type": "auth",
+ "status": "success",
+ "client_id": state.client_id,
+ "expires_at": has(token_data.expire) ? int(token_data.expire) : 0,
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ },
+ "should_fetch": true,
+ "auth_failed": false
+ }
+ :
+ {
+ "updated_state": state,
+ "auth_event": {
+ "message": {
+ "log_type": "auth",
+ "status": "failed",
+ "client_id": state.client_id,
+ "error_message": "No valid token received",
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts,
+ "response_body": string(auth_resp.Body)
+ }.encode_json()
+ },
+ "should_fetch": false,
+ "auth_failed": true
+ }
+ )
+ :
+ (size(auth_resp.Body) > 0 ? bytes(auth_resp.Body).decode_json() : {}).as(error_body,
+ retryable.exists(code, code == auth_resp.StatusCode).as(is_retryable,
+ (has(error_body.errors) && size(error_body.errors) > 0 ?
+ string(error_body.errors[0].message) :
+ "Authentication failed with HTTP " + string(auth_resp.StatusCode)
+ ).as(error_msg,
+ is_retryable ?
+ {
+ "updated_state": state,
+ "auth_event": {
+ "message": {
+ "log_type": "auth",
+ "status": "retry",
+ "client_id": state.client_id,
+ "error_message": error_msg,
+ "error_code": has(error_body.errors) && size(error_body.errors) > 0 ? int(error_body.errors[0].code) : 0,
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ },
+ "should_fetch": false,
+ "auth_failed": false,
+ "want_more": true
+ }
+ :
+ {
+ "updated_state": state,
+ "auth_event": {
+ "message": {
+ "log_type": "auth",
+ "status": "failed",
+ "client_id": state.client_id,
+ "error_message": error_msg,
+ "error_code": has(error_body.errors) && size(error_body.errors) > 0 ? int(error_body.errors[0].code) : 0,
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts,
+ "response_body": string(auth_resp.Body)
+ }.encode_json()
+ },
+ "should_fetch": false,
+ "auth_failed": true,
+ "want_more": false
+ }
+ )
+ )
+ )
+ )
+ :
+ {
+ "updated_state": state,
+ "auth_event": {},
+ "should_fetch": true,
+ "auth_failed": false
+ }
+ ).as(prep,
+ prep.should_fetch ?
+ (prep.updated_state.next_cursor != "" ?
+ prep.updated_state.base_url + prep.updated_state.next_cursor :
+ prep.updated_state.url + "?" + {"pageSize": [string(prep.updated_state.page_size)]}.format_query()
+ ).as(request_url,
+ get_request(request_url).with({
+ "Header": {
+ "Authorization": ["Bearer " + prep.updated_state.access_token],
+ "Accept": ["application/json"],
+ "x-application-name": ["elastic-siem-" + state.version]
+ }
+ }).do_request().as(api_resp,
+ api_resp.StatusCode == 200 ?
+ (size(api_resp.Body) > 0 ? bytes(api_resp.Body).decode_json() : {}).as(response_body,
+ prep.updated_state.with({
+ "events": (has(prep.auth_event.message) ? [prep.auth_event] : []) +
+ (has(response_body.alerts) && size(response_body.alerts) > 0 ?
+ response_body.alerts.map(alert, {
+ "message": alert.with({
+ "cursor_used": prep.updated_state.next_cursor != "" ? prep.updated_state.next_cursor : "initial",
+ "fetched_alerts": size(response_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }) : []
+ ) + [{
+ "message": {
+ "log_type": "alert-fetch",
+ "status": "success",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": prep.updated_state.page_size,
+ "next_cursor": has(response_body.nextPage) && response_body.nextPage != "" ? string(response_body.nextPage) : "",
+ "count_of_alerts": has(response_body.alerts) ? size(response_body.alerts) : 0,
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": prep.updated_state.next_cursor != "" ? prep.updated_state.next_cursor : "initial",
+ "fetched_alerts": has(response_body.alerts) ? size(response_body.alerts) : 0,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ }],
+ "next_cursor": has(response_body.nextPage) && response_body.nextPage != "" ? string(response_body.nextPage) : "",
+ "access_token": prep.updated_state.access_token,
+ "expires_at": prep.updated_state.expires_at,
+ "last_fetch_timestamp": fetch_ts,
+ "want_more": false
+ })
+ )
+ :
+ (size(api_resp.Body) > 0 ? bytes(api_resp.Body).decode_json() : {}).as(error_body,
+ retryable.exists(code, code == api_resp.StatusCode).as(is_retryable,
+ (has(error_body.error) ?
+ string(error_body.error) :
+ "API request failed with HTTP " + string(api_resp.StatusCode)
+ ).as(error_msg,
+ is_retryable ?
+ prep.updated_state.with({
+ "events": (has(prep.auth_event.message) ? [prep.auth_event] : []) + [{
+ "message": {
+ "log_type": "alert-fetch",
+ "status": "retry",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": prep.updated_state.page_size,
+ "error_message": error_msg,
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": prep.updated_state.next_cursor != "" ? prep.updated_state.next_cursor : "initial",
+ "fetched_alerts": 0,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ }],
+ "access_token": prep.updated_state.access_token,
+ "expires_at": prep.updated_state.expires_at,
+ "want_more": true
+ })
+ :
+ prep.updated_state.with({
+ "events": (has(prep.auth_event.message) ? [prep.auth_event] : []) + [{
+ "message": {
+ "log_type": "alert-fetch",
+ "status": "failed",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": prep.updated_state.page_size,
+ "error_message": error_msg,
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": prep.updated_state.next_cursor != "" ? prep.updated_state.next_cursor : "initial",
+ "fetched_alerts": 0,
+ "fetch_timestamp": fetch_ts,
+ "response_body": string(api_resp.Body)
+ }.encode_json()
+ }],
+ "access_token": prep.updated_state.access_token,
+ "expires_at": prep.updated_state.expires_at,
+ "next_cursor": api_resp.StatusCode == 400 || api_resp.StatusCode == 404 ? "" : prep.updated_state.next_cursor,
+ "want_more": false
+ })
+ )
+ )
+ )
+ )
+ )
+ :
+ prep.updated_state.with({
+ "events": [prep.auth_event],
+ "want_more": has(prep.want_more) ? prep.want_more : false
+ })
+ )
+ )
+ )
+ )
+ )
+ )
\ No newline at end of file
diff --git a/packages/dataminr_pulse/_devtests/state.json b/packages/dataminr_pulse/_devtests/state.json
new file mode 100644
index 00000000000..7dcfde6210f
--- /dev/null
+++ b/packages/dataminr_pulse/_devtests/state.json
@@ -0,0 +1,13 @@
+{
+ "url": "https://api.dataminr.com/pulse/v1/alerts",
+ "base_url": "https://api.dataminr.com/pulse",
+ "client_id": "0oaqa65xf8e6Ocdsl5d7",
+ "client_secret": "`",
+ "version": "0.11.0",
+ "page_size": 40,
+ "next_cursor": "",
+ "access_token": "",
+ "expires_at": 0,
+ "want_more": false,
+ "last_fetch_timestamp": 0
+}
diff --git a/packages/dataminr_pulse/changelog.yml b/packages/dataminr_pulse/changelog.yml
new file mode 100644
index 00000000000..88759141b24
--- /dev/null
+++ b/packages/dataminr_pulse/changelog.yml
@@ -0,0 +1,5 @@
+- version: "1.0.0"
+ changes:
+ - description: Initial release
+ type: enhancement
+ link: https://github.com/elastic/integrations/issues/1
diff --git a/packages/dataminr_pulse/config/kibana.yml b/packages/dataminr_pulse/config/kibana.yml
new file mode 100644
index 00000000000..1f2fa22ee3c
--- /dev/null
+++ b/packages/dataminr_pulse/config/kibana.yml
@@ -0,0 +1,29 @@
+server.host: "0.0.0.0"
+
+xpack.fleet.outputs:
+ - id: fleet-default-output
+ name: default
+ type: elasticsearch
+ hosts:
+ - http://elasticsearch:9200
+ is_default: true
+ is_default_monitoring: true
+
+xpack.fleet.agents.fleet_server.hosts:
+ - http://fleet-server:8220
+
+xpack.fleet.packages:
+ - name: fleet_server
+ version: latest
+
+xpack.fleet.agentPolicies:
+ - id: fleet-server-policy
+ name: Fleet Server Policy
+ is_default_fleet_server: true
+ package_policies:
+ - name: fleet_server-1
+ package:
+ name: fleet_server
+ - id: default-agent-policy
+ name: Default Agent Policy
+ is_default: true
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/docker-compose.yml b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/docker-compose.yml
new file mode 100644
index 00000000000..ef88b412ed7
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/docker-compose.yml
@@ -0,0 +1,9 @@
+version: "2.3"
+services:
+ elastic-siem-integration:
+ image: python:3.12-slim
+ volumes:
+ - ./files:/app
+ command: ["python", "/app/mock_server.py"]
+ ports:
+ - 8080
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/alerts_response.json b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/alerts_response.json
new file mode 100644
index 00000000000..e2992aa7752
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/alerts_response.json
@@ -0,0 +1,142 @@
+{
+ "alerts": [
+ {
+ "headline": "Test alert message",
+ "alertId": "5026527284588218000-1770894234190-2",
+ "publicPost": {
+ "timestamp": "2026-02-12T11:03:54.193Z",
+ "channels": ["sensor"],
+ "href": "https://r.dataminr.com/28IJuehlvBfbxqRBTBPK0r"
+ },
+ "listsMatched": [
+ {
+ "id": "4932401",
+ "name": "Cyber",
+ "topicIds": ["853036"],
+ "subType": "DIGITAL_RISK"
+ }
+ ],
+ "estimatedEventLocation": {
+ "name": "Mecklenburg County, NC, USA",
+ "coordinates": [35.263265, -80.854384],
+ "probabilityRadius": 0.1
+ },
+ "dataminrAlertUrl": "https://app.dataminr.com/#alertDetail/5/5026527284588218000-1770894234190-2",
+ "alertType": {
+ "name": "Alert"
+ },
+ "alertTopics": [
+ {
+ "id": "853036",
+ "name": "Outages & Service Disruptions - Electricity"
+ }
+ ],
+ "alertTimestamp": "2026-02-12T11:04:58.116Z",
+ "alertSectors": [
+ {
+ "name": "Utilities"
+ }
+ ],
+ "alertCompanies": [
+ {
+ "name": "Duke Energy Corporation"
+ }
+ ]
+ },
+ {
+ "headline": "Threat actor defaces example website",
+ "alertId": "7082842488761385731-1770893196295-1",
+ "publicPost": {
+ "timestamp": "2026-02-12T10:46:36.295Z",
+ "channels": ["chatter"],
+ "href": "https://r.dataminr.com/7M6ZZxnPyLakF8sKDRLSIQ",
+ "media": [
+ {
+ "type": "photo",
+ "href": "https://alertmedia.dataminr.com/test-image.png"
+ }
+ ]
+ },
+ "listsMatched": [
+ {
+ "id": "4932401",
+ "name": "Cyber",
+ "topicIds": ["963136", "962003"],
+ "subType": "DIGITAL_RISK"
+ }
+ ],
+ "estimatedEventLocation": {
+ "name": "Sendai, Miyagi, Japan",
+ "coordinates": [38.268195, 140.869418],
+ "probabilityRadius": 18.9
+ },
+ "dataminrAlertUrl": "https://app.dataminr.com/#alertDetail/5/7082842488761385731-1770893196295-1",
+ "alertType": {
+ "name": "Urgent"
+ },
+ "alertTopics": [
+ {
+ "id": "963136",
+ "name": "Threat Actors - Hacktivists"
+ },
+ {
+ "id": "962003",
+ "name": "Defacement"
+ }
+ ],
+ "alertTimestamp": "2026-02-12T10:47:40.147Z"
+ },
+ {
+ "headline": "Malware sample detected targeting financial systems",
+ "alertId": "9999888877776666-1770890000000-1",
+ "publicPost": {
+ "timestamp": "2026-02-12T09:00:00.000Z",
+ "channels": ["chatter"],
+ "href": "https://r.dataminr.com/test123"
+ },
+ "listsMatched": [
+ {
+ "id": "4932401",
+ "name": "Cyber",
+ "topicIds": ["963155"],
+ "subType": "DIGITAL_RISK"
+ }
+ ],
+ "dataminrAlertUrl": "https://app.dataminr.com/#alertDetail/5/9999888877776666-1770890000000-1",
+ "alertType": {
+ "name": "Flash"
+ },
+ "alertTopics": [
+ {
+ "id": "963155",
+ "name": "Data Exposure & Breaches"
+ }
+ ],
+ "alertTimestamp": "2026-02-12T09:01:00.000Z",
+ "alertSectors": [
+ {
+ "name": "Financials"
+ }
+ ],
+ "alertCompanies": [
+ {
+ "name": "Example Bank Corp"
+ }
+ ],
+ "metadata": {
+ "cyber": {
+ "hashValues": [
+ {
+ "type": "MD5",
+ "value": "d41d8cd98f00b204e9800998ecf8427e"
+ },
+ {
+ "type": "SHA256",
+ "value": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+ }
+ ]
+ }
+ }
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/mock_server.py b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/mock_server.py
new file mode 100644
index 00000000000..41629539bc3
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/deploy/docker/files/mock_server.py
@@ -0,0 +1,78 @@
+"""Mock Dataminr Pulse API server for elastic-package system tests."""
+
+import json
+import time
+from http.server import HTTPServer, BaseHTTPRequestHandler
+from urllib.parse import urlparse, parse_qs
+
+with open("/app/alerts_response.json", "r") as f:
+ ALERTS_RESPONSE = f.read()
+
+
+def make_token_response():
+ expire_ms = int((time.time() + 3600) * 1000)
+ return json.dumps({"dmaToken": "mock-test-token-abc123", "expire": expire_ms})
+
+
+class MockHandler(BaseHTTPRequestHandler):
+ def do_POST(self):
+ parsed = urlparse(self.path)
+ if parsed.path == "/auth/2/token":
+ self._handle_token()
+ else:
+ self._send_json(404, {"error": "Not found"})
+
+ def do_GET(self):
+ parsed = urlparse(self.path)
+ if parsed.path == "/pulse/v1/alerts":
+ self._handle_alerts()
+ else:
+ self._send_json(404, {"error": "Not found"})
+
+ def _handle_token(self):
+ content_length = int(self.headers.get("Content-Length", 0))
+ body = self.rfile.read(content_length).decode("utf-8")
+
+ content_type = self.headers.get("Content-Type", "")
+ if "application/x-www-form-urlencoded" not in content_type:
+ self._send_json(400, {"error": "Invalid content type"})
+ return
+
+ params = parse_qs(body)
+ grant_type = params.get("grant_type", [None])[0]
+ client_id = params.get("client_id", [None])[0]
+ client_secret = params.get("client_secret", [None])[0]
+
+ if grant_type != "api_key" or not client_id or not client_secret:
+ self._send_json(401, {"error": "Invalid credentials"})
+ return
+
+ self._send_raw(200, make_token_response())
+
+ def _handle_alerts(self):
+ auth_header = self.headers.get("Authorization", "")
+ if not auth_header.startswith("Bearer "):
+ self._send_json(401, {"error": "Unauthorized"})
+ return
+
+ self._send_raw(200, ALERTS_RESPONSE)
+
+ def _send_json(self, code, data):
+ self._send_raw(code, json.dumps(data))
+
+ def _send_raw(self, code, body_str):
+ body = body_str.encode()
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ self.wfile.write(body)
+
+ def log_message(self, format, *args):
+ print(f"[MockServer] {format % args}")
+
+
+if __name__ == "__main__":
+ server = HTTPServer(("0.0.0.0", 8080), MockHandler)
+ print("[MockServer] Starting Dataminr Pulse mock API on port 8080")
+ server.serve_forever()
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json
new file mode 100644
index 00000000000..3a011d6531a
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json
@@ -0,0 +1,13 @@
+{
+ "events": [
+ {
+ "message": "{\"headline\":\"Test alert message\",\"alertId\":\"5026527284588218000-1770894234190-2\",\"publicPost\":{\"timestamp\":\"2026-02-12T11:03:54.193Z\",\"channels\":[\"sensor\"],\"href\":\"https://r.dataminr.com/28IJuehlvBfbxqRBTBPK0r\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"853036\"],\"subType\":\"DIGITAL_RISK\"}],\"estimatedEventLocation\":{\"name\":\"Mecklenburg County, NC, USA\",\"coordinates\":[35.263265,-80.854384],\"probabilityRadius\":0.1},\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/5026527284588218000-1770894234190-2\",\"alertType\":{\"name\":\"Alert\"},\"alertTopics\":[{\"id\":\"853036\",\"name\":\"Outages & Service Disruptions - Electricity\"}],\"alertTimestamp\":\"2026-02-12T11:04:58.116Z\",\"alertSectors\":[{\"name\":\"Utilities\"}],\"alertCompanies\":[{\"name\":\"Duke Energy Corporation\"}],\"cursor_used\":\"initial\",\"fetched_alerts\":40,\"fetch_timestamp\":\"2026-02-12T11:05:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Threat actor defaces example website\",\"alertId\":\"7082842488761385731-1770893196295-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T10:46:36.295Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/7M6ZZxnPyLakF8sKDRLSIQ\",\"media\":[{\"type\":\"photo\",\"href\":\"https://alertmedia.dataminr.com/test-image.png\"}]},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963136\",\"962003\"],\"subType\":\"DIGITAL_RISK\"}],\"estimatedEventLocation\":{\"name\":\"Sendai, Miyagi, Japan\",\"coordinates\":[38.268195,140.869418],\"probabilityRadius\":18.9},\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/7082842488761385731-1770893196295-1\",\"alertType\":{\"name\":\"Urgent\"},\"alertTopics\":[{\"id\":\"963136\",\"name\":\"Threat Actors - Hacktivists\"},{\"id\":\"962003\",\"name\":\"Defacement\"}],\"alertTimestamp\":\"2026-02-12T10:47:40.147Z\",\"cursor_used\":\"/v1/alerts?lists=4932401&from=abc123\",\"fetched_alerts\":40,\"fetch_timestamp\":\"2026-02-12T10:48:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Malware sample detected targeting financial systems\",\"alertId\":\"9999888877776666-1770890000000-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T09:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/test123\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/9999888877776666-1770890000000-1\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Data Exposure & Breaches\"}],\"alertTimestamp\":\"2026-02-12T09:01:00.000Z\",\"alertSectors\":[{\"name\":\"Financials\"}],\"alertCompanies\":[{\"name\":\"Example Bank Corp\"}],\"metadata\":{\"cyber\":{\"hashValues\":[{\"type\":\"MD5\",\"value\":\"d41d8cd98f00b204e9800998ecf8427e\"},{\"type\":\"SHA256\",\"value\":\"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\"}]}},\"cursor_used\":\"initial\",\"fetched_alerts\":10,\"fetch_timestamp\":\"2026-02-12T09:02:00.000Z\"}"
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json-expected.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json-expected.json
new file mode 100644
index 00000000000..33e637f9814
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-alert-events.json-expected.json
@@ -0,0 +1,182 @@
+{
+ "expected": [
+ {
+ "@timestamp": "2026-02-12T11:04:58.116Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Alert"
+ },
+ "categories": {
+ "name": [
+ "Outages & Service Disruptions - Electricity"
+ ]
+ },
+ "companies": {
+ "name": [
+ "Duke Energy Corporation"
+ ]
+ },
+ "cursor_used": "initial",
+ "sectors": {
+ "name": [
+ "Utilities"
+ ]
+ },
+ "source": {
+ "channels": [
+ "sensor"
+ ],
+ "href": "https://r.dataminr.com/28IJuehlvBfbxqRBTBPK0r"
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "created": "2026-02-12T11:03:54.193Z",
+ "id": "5026527284588218000-1770894234190-2",
+ "kind": "alert",
+ "severity": 30,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/5026527284588218000-1770894234190-2"
+ },
+ "geo": {
+ "location": {
+ "lat": 35.263265,
+ "lon": -80.854384
+ },
+ "name": "Mecklenburg County, NC, USA"
+ },
+ "message": "Test alert message"
+ },
+ {
+ "@timestamp": "2026-02-12T10:47:40.147Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Urgent"
+ },
+ "categories": {
+ "name": [
+ "Threat Actors - Hacktivists",
+ "Defacement"
+ ]
+ },
+ "cursor_used": "/v1/alerts?lists=4932401&from=abc123",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/7M6ZZxnPyLakF8sKDRLSIQ",
+ "media": {
+ "href": [
+ "https://alertmedia.dataminr.com/test-image.png"
+ ]
+ }
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "created": "2026-02-12T10:46:36.295Z",
+ "id": "7082842488761385731-1770893196295-1",
+ "kind": "alert",
+ "severity": 50,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/7082842488761385731-1770893196295-1"
+ },
+ "geo": {
+ "location": {
+ "lat": 38.268195,
+ "lon": 140.869418
+ },
+ "name": "Sendai, Miyagi, Japan"
+ },
+ "message": "Threat actor defaces example website"
+ },
+ {
+ "@timestamp": "2026-02-12T09:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Data Exposure & Breaches"
+ ]
+ },
+ "companies": {
+ "name": [
+ "Example Bank Corp"
+ ]
+ },
+ "cursor_used": "initial",
+ "sectors": {
+ "name": [
+ "Financials"
+ ]
+ },
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/test123"
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "created": "2026-02-12T09:00:00.000Z",
+ "id": "9999888877776666-1770890000000-1",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/9999888877776666-1770890000000-1"
+ },
+ "file": {
+ "hash": {
+ "md5": "d41d8cd98f00b204e9800998ecf8427e",
+ "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+ }
+ },
+ "message": "Malware sample detected targeting financial systems",
+ "related": {
+ "hash": [
+ "d41d8cd98f00b204e9800998ecf8427e",
+ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+ ]
+ }
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-common-config.yml b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-common-config.yml
new file mode 100644
index 00000000000..2a0e7a960d8
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-common-config.yml
@@ -0,0 +1,2 @@
+dynamic_fields:
+ "_id": "^.*$"
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json
new file mode 100644
index 00000000000..8114b1c6d24
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json
@@ -0,0 +1,16 @@
+{
+ "events": [
+ {
+ "message": "{\"headline\":\"Critical RCE vulnerability exploited in the wild\",\"alertId\":\"1111222233334444-1770890000000-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T08:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/vuln1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/1111222233334444-1770890000000-1\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-12T08:01:00.000Z\",\"intelAgents\":[{\"summary\":\"CVE-2026-1234 is a critical remote code execution vulnerability in ExampleSoft product.\",\"timestamp\":\"2026-02-12T08:00:30.000Z\",\"version\":\"2.1\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-1234\",\"cve\":\"CVE-2026-1234\",\"cvss\":9.8,\"summary\":\"Remote code execution in ExampleSoft\",\"publishedDate\":\"2026-02-10T00:00:00.000Z\",\"epssScore\":0.95,\"exploitable\":true},{\"type\":\"threatActor\",\"name\":\"APT-TEST\",\"aliases\":[\"TestGroup\",\"TA-999\"],\"countryOfOrigin\":[\"RU\",\"CN\"]}]}],\"discovered_entity\":{\"type\":\"vulnerability\",\"name\":\"CVE-2026-1234\",\"cve\":\"CVE-2026-1234\",\"cvss\":9.8,\"summary\":\"Remote code execution in ExampleSoft\",\"publishedDate\":\"2026-02-10T00:00:00.000Z\",\"epssScore\":0.95,\"exploitable\":true},\"discoveredentity_type\":\"vulnerability\",\"discoveredentity_value\":\"CVE-2026-1234\",\"entityType\":\"vulnerability\",\"is_main\":\"yes\",\"cursor_used\":\"initial\",\"fetched_alerts\":5,\"fetch_timestamp\":\"2026-02-12T08:02:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Critical RCE vulnerability exploited in the wild\",\"alertId\":\"1111222233334444-1770890000000-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T08:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/vuln1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/1111222233334444-1770890000000-1\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-12T08:01:00.000Z\",\"intelAgents\":[{\"summary\":\"CVE-2026-1234 is a critical remote code execution vulnerability in ExampleSoft product.\",\"timestamp\":\"2026-02-12T08:00:30.000Z\",\"version\":\"2.1\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-1234\",\"cve\":\"CVE-2026-1234\",\"cvss\":9.8,\"summary\":\"Remote code execution in ExampleSoft\",\"publishedDate\":\"2026-02-10T00:00:00.000Z\",\"epssScore\":0.95,\"exploitable\":true},{\"type\":\"threatActor\",\"name\":\"APT-TEST\",\"aliases\":[\"TestGroup\",\"TA-999\"],\"countryOfOrigin\":[\"RU\",\"CN\"]}]}],\"discovered_entity\":{\"type\":\"threatActor\",\"name\":\"APT-TEST\",\"aliases\":[\"TestGroup\",\"TA-999\"],\"countryOfOrigin\":[\"RU\",\"CN\"]},\"discoveredentity_type\":\"threatActor\",\"discoveredentity_value\":\"APT-TEST\",\"entityType\":\"threatActor\",\"is_main\":\"no\",\"cursor_used\":\"initial\",\"fetched_alerts\":5,\"fetch_timestamp\":\"2026-02-12T08:02:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Suspicious IP scanning critical infrastructure\",\"alertId\":\"5555666677778888-1770895000000-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T09:30:00.000Z\",\"channels\":[\"sensor\"],\"href\":\"https://r.dataminr.com/ip1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963200\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/5555666677778888-1770895000000-1\",\"alertType\":{\"name\":\"Urgent\"},\"alertTopics\":[{\"id\":\"963200\",\"name\":\"Network Scanning\"}],\"alertTimestamp\":\"2026-02-12T09:31:00.000Z\",\"intelAgents\":[{\"summary\":\"Multiple IPs observed scanning.\",\"timestamp\":\"2026-02-12T09:30:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"ipAddress\",\"ip\":\"192[.]168[.]1[.]100\",\"ports\":[443,8080]},{\"type\":\"url\",\"name\":\"hxxps://evil[.]example[.]com/payload\"}]}],\"discoveredentity_type\":\"url-ipAddress\",\"entityType\":\"url-ipAddress\",\"is_main\":\"yes\",\"cursor_used\":\"/v1/alerts?from=xyz789\",\"fetched_alerts\":3,\"fetch_timestamp\":\"2026-02-12T09:32:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"New ransomware variant targeting enterprise systems\",\"alertId\":\"9999000011112222-1770900000000-1\",\"publicPost\":{\"timestamp\":\"2026-02-12T10:00:00.000Z\",\"channels\":[\"darkweb\"],\"href\":\"https://r.dataminr.com/malware1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963300\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/9999000011112222-1770900000000-1\",\"alertType\":{\"name\":\"Alert\"},\"alertTopics\":[{\"id\":\"963300\",\"name\":\"Malware Analysis\"}],\"alertTimestamp\":\"2026-02-12T10:01:00.000Z\",\"intelAgents\":[{\"summary\":\"DarkVortex ransomware targets Windows and Linux systems.\",\"timestamp\":\"2026-02-12T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"malware\",\"name\":\"DarkVortex\",\"affectedOperatingSystems\":[\"win\",\"elf\"]}]}],\"discovered_entity\":{\"type\":\"malware\",\"name\":\"DarkVortex\",\"affectedOperatingSystems\":[\"win\",\"elf\"]},\"discoveredentity_type\":\"malware\",\"discoveredentity_value\":\"DarkVortex\",\"entityType\":\"malware\",\"is_main\":\"yes\",\"cursor_used\":\"initial\",\"fetched_alerts\":2,\"fetch_timestamp\":\"2026-02-12T10:02:00.000Z\"}"
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json-expected.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json-expected.json
new file mode 100644
index 00000000000..03c64208c22
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-entity-events.json-expected.json
@@ -0,0 +1,325 @@
+{
+ "expected": [
+ {
+ "@timestamp": "2026-02-12T08:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {
+ "cve": "CVE-2026-1234",
+ "summary": "Remote code execution in ExampleSoft",
+ "value": "CVE-2026-1234"
+ },
+ "intel_agents": {
+ "summary": "CVE-2026-1234 is a critical remote code execution vulnerability in ExampleSoft product."
+ },
+ "is_main": "yes",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/vuln1"
+ },
+ "threatactor": {
+ "alias": [
+ "TestGroup",
+ "TA-999"
+ ],
+ "country_of_origin": [
+ "RU",
+ "CN"
+ ],
+ "name": [
+ "APT-TEST"
+ ]
+ },
+ "vulnerability": {
+ "name": [
+ "CVE-2026-1234"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "vulnerability"
+ ],
+ "created": "2026-02-12T08:00:00.000Z",
+ "id": "1111222233334444-1770890000000-1",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/1111222233334444-1770890000000-1"
+ },
+ "message": "Critical RCE vulnerability exploited in the wild",
+ "vulnerability": {
+ "description": "Remote code execution in ExampleSoft",
+ "id": "CVE-2026-1234",
+ "score": {
+ "base": 9.8
+ }
+ }
+ },
+ {
+ "@timestamp": "2026-02-12T08:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {
+ "aliases": [
+ "TestGroup",
+ "TA-999"
+ ],
+ "country_of_origin": [
+ "RU",
+ "CN"
+ ],
+ "value": "APT-TEST"
+ },
+ "intel_agents": {
+ "summary": "CVE-2026-1234 is a critical remote code execution vulnerability in ExampleSoft product."
+ },
+ "is_main": "no",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/vuln1"
+ },
+ "threatactor": {
+ "alias": [
+ "TestGroup",
+ "TA-999"
+ ],
+ "country_of_origin": [
+ "RU",
+ "CN"
+ ],
+ "name": [
+ "APT-TEST"
+ ]
+ },
+ "vulnerability": {
+ "name": [
+ "CVE-2026-1234"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "threat"
+ ],
+ "created": "2026-02-12T08:00:00.000Z",
+ "id": "1111222233334444-1770890000000-1",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "indicator"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/1111222233334444-1770890000000-1"
+ },
+ "message": "Critical RCE vulnerability exploited in the wild",
+ "threat": {
+ "framework": "MITRE ATT&CK",
+ "group": {
+ "alias": [
+ "TestGroup",
+ "TA-999"
+ ],
+ "name": "APT-TEST"
+ },
+ "indicator": {
+ "geo": {
+ "country_iso_code": [
+ "RU",
+ "CN"
+ ]
+ }
+ }
+ }
+ },
+ {
+ "@timestamp": "2026-02-12T09:31:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Urgent"
+ },
+ "categories": {
+ "name": [
+ "Network Scanning"
+ ]
+ },
+ "cursor_used": "/v1/alerts?from=xyz789",
+ "discovered_entity": {},
+ "intel_agents": {
+ "summary": "Multiple IPs observed scanning."
+ },
+ "ip": [
+ "192.168.1.100"
+ ],
+ "is_main": "yes",
+ "source": {
+ "channels": [
+ "sensor"
+ ],
+ "href": "https://r.dataminr.com/ip1"
+ },
+ "url": [
+ "hxxps://evil.example.com/payload"
+ ],
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "threat"
+ ],
+ "created": "2026-02-12T09:30:00.000Z",
+ "id": "5555666677778888-1770895000000-1",
+ "kind": "alert",
+ "severity": 50,
+ "type": [
+ "indicator"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/5555666677778888-1770895000000-1"
+ },
+ "message": "Suspicious IP scanning critical infrastructure",
+ "related": {
+ "ip": [
+ "192.168.1.100"
+ ]
+ },
+ "threat": {
+ "enrichments": [
+ {
+ "indicator": {
+ "ip": "192.168.1.100",
+ "name": "192[.]168[.]1[.]100",
+ "port": [
+ 443,
+ 8080
+ ],
+ "type": "ipv4-addr"
+ }
+ },
+ {
+ "indicator": {
+ "name": "hxxps://evil[.]example[.]com/payload",
+ "type": "url",
+ "url": {
+ "original": "hxxps://evil.example.com/payload"
+ }
+ }
+ }
+ ]
+ }
+ },
+ {
+ "@timestamp": "2026-02-12T10:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Alert"
+ },
+ "categories": {
+ "name": [
+ "Malware Analysis"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {
+ "affected_operating_systems": [
+ "win",
+ "elf"
+ ],
+ "value": "DarkVortex"
+ },
+ "intel_agents": {
+ "summary": "DarkVortex ransomware targets Windows and Linux systems."
+ },
+ "is_main": "yes",
+ "malware": [
+ "DarkVortex"
+ ],
+ "platforms": [
+ "Windows",
+ "Linux"
+ ],
+ "source": {
+ "channels": [
+ "darkweb"
+ ],
+ "href": "https://r.dataminr.com/malware1"
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "malware"
+ ],
+ "created": "2026-02-12T10:00:00.000Z",
+ "id": "9999000011112222-1770900000000-1",
+ "kind": "alert",
+ "severity": 30,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/9999000011112222-1770900000000-1"
+ },
+ "message": "New ransomware variant targeting enterprise systems",
+ "threat": {
+ "software": {
+ "name": "DarkVortex",
+ "platforms": [
+ "Windows",
+ "Linux"
+ ],
+ "type": "Malware"
+ }
+ }
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json
new file mode 100644
index 00000000000..0fea42f368a
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json
@@ -0,0 +1,16 @@
+{
+ "events": [
+ {
+ "message": "{\"headline\":\"Multi-entity alert for fingerprint testing\",\"alertId\":\"FINGERPRINT-TEST-ALERT-001\",\"publicPost\":{\"timestamp\":\"2026-02-20T10:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/fp1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-20T10:01:00.000Z\",\"intelAgents\":[{\"summary\":\"Test summary.\",\"timestamp\":\"2026-02-20T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-0001\"},{\"type\":\"threatActor\",\"name\":\"APT-FP-TEST\"},{\"type\":\"ipAddress\",\"ip\":\"10[.]0[.]0[.]1\"},{\"type\":\"url\",\"name\":\"hxxps://evil[.]test[.]com\"}]}],\"discovered_entity\":{\"type\":\"vulnerability\",\"name\":\"CVE-2026-0001\"},\"discoveredentity_type\":\"vulnerability\",\"discoveredentity_value\":\"CVE-2026-0001\",\"entityType\":\"vulnerability\",\"is_main\":\"yes\",\"cursor_used\":\"initial\",\"fetched_alerts\":1,\"fetch_timestamp\":\"2026-02-20T10:02:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Multi-entity alert for fingerprint testing\",\"alertId\":\"FINGERPRINT-TEST-ALERT-001\",\"publicPost\":{\"timestamp\":\"2026-02-20T10:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/fp1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-20T10:01:00.000Z\",\"intelAgents\":[{\"summary\":\"Test summary.\",\"timestamp\":\"2026-02-20T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-0001\"},{\"type\":\"threatActor\",\"name\":\"APT-FP-TEST\"},{\"type\":\"ipAddress\",\"ip\":\"10[.]0[.]0[.]1\"},{\"type\":\"url\",\"name\":\"hxxps://evil[.]test[.]com\"}]}],\"discovered_entity\":{\"type\":\"threatActor\",\"name\":\"APT-FP-TEST\"},\"discoveredentity_type\":\"threatActor\",\"discoveredentity_value\":\"APT-FP-TEST\",\"entityType\":\"threatActor\",\"is_main\":\"no\",\"cursor_used\":\"initial\",\"fetched_alerts\":1,\"fetch_timestamp\":\"2026-02-20T10:02:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Multi-entity alert for fingerprint testing\",\"alertId\":\"FINGERPRINT-TEST-ALERT-001\",\"publicPost\":{\"timestamp\":\"2026-02-20T10:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/fp1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-20T10:01:00.000Z\",\"intelAgents\":[{\"summary\":\"Test summary.\",\"timestamp\":\"2026-02-20T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-0001\"},{\"type\":\"threatActor\",\"name\":\"APT-FP-TEST\"},{\"type\":\"ipAddress\",\"ip\":\"10[.]0[.]0[.]1\"},{\"type\":\"url\",\"name\":\"hxxps://evil[.]test[.]com\"}]}],\"discoveredentity_type\":\"url-ipAddress\",\"entityType\":\"url-ipAddress\",\"is_main\":\"no\",\"cursor_used\":\"initial\",\"fetched_alerts\":1,\"fetch_timestamp\":\"2026-02-20T10:02:00.000Z\"}"
+ },
+ {
+ "message": "{\"headline\":\"Multi-entity alert for fingerprint testing\",\"alertId\":\"FINGERPRINT-TEST-ALERT-001\",\"publicPost\":{\"timestamp\":\"2026-02-20T10:00:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/fp1\"},\"listsMatched\":[{\"id\":\"4932401\",\"name\":\"Cyber\",\"topicIds\":[\"963155\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTopics\":[{\"id\":\"963155\",\"name\":\"Vulnerability Disclosure\"}],\"alertTimestamp\":\"2026-02-20T10:01:00.000Z\",\"intelAgents\":[{\"summary\":\"Test summary.\",\"timestamp\":\"2026-02-20T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-0001\"},{\"type\":\"threatActor\",\"name\":\"APT-FP-TEST\"},{\"type\":\"ipAddress\",\"ip\":\"10[.]0[.]0[.]1\"},{\"type\":\"url\",\"name\":\"hxxps://evil[.]test[.]com\"}]}],\"discoveredentity_type\":\"consolidated\",\"entityType\":\"consolidated\",\"is_main\":\"no\",\"cursor_used\":\"initial\",\"fetched_alerts\":1,\"fetch_timestamp\":\"2026-02-20T10:02:00.000Z\"}"
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json-expected.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json-expected.json
new file mode 100644
index 00000000000..8fe97ae9cd4
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-fingerprint-events.json-expected.json
@@ -0,0 +1,309 @@
+{
+ "expected": [
+ {
+ "@timestamp": "2026-02-20T10:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {
+ "value": "CVE-2026-0001"
+ },
+ "intel_agents": {
+ "summary": "Test summary."
+ },
+ "ip": [
+ "10.0.0.1"
+ ],
+ "is_main": "yes",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/fp1"
+ },
+ "threatactor": {
+ "name": [
+ "APT-FP-TEST"
+ ]
+ },
+ "url": [
+ "hxxps://evil.test.com"
+ ],
+ "vulnerability": {
+ "name": [
+ "CVE-2026-0001"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "vulnerability"
+ ],
+ "created": "2026-02-20T10:00:00.000Z",
+ "id": "FINGERPRINT-TEST-ALERT-001",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001"
+ },
+ "message": "Multi-entity alert for fingerprint testing",
+ "related": {
+ "ip": [
+ "10.0.0.1"
+ ]
+ },
+ "vulnerability": {
+ "id": "CVE-2026-0001"
+ }
+ },
+ {
+ "@timestamp": "2026-02-20T10:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {
+ "value": "APT-FP-TEST"
+ },
+ "intel_agents": {
+ "summary": "Test summary."
+ },
+ "ip": [
+ "10.0.0.1"
+ ],
+ "is_main": "no",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/fp1"
+ },
+ "threatactor": {
+ "name": [
+ "APT-FP-TEST"
+ ]
+ },
+ "url": [
+ "hxxps://evil.test.com"
+ ],
+ "vulnerability": {
+ "name": [
+ "CVE-2026-0001"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "threat"
+ ],
+ "created": "2026-02-20T10:00:00.000Z",
+ "id": "FINGERPRINT-TEST-ALERT-001",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "indicator"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001"
+ },
+ "message": "Multi-entity alert for fingerprint testing",
+ "related": {
+ "ip": [
+ "10.0.0.1"
+ ]
+ },
+ "threat": {
+ "framework": "MITRE ATT&CK",
+ "group": {
+ "name": "APT-FP-TEST"
+ }
+ }
+ },
+ {
+ "@timestamp": "2026-02-20T10:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {},
+ "intel_agents": {
+ "summary": "Test summary."
+ },
+ "ip": [
+ "10.0.0.1"
+ ],
+ "is_main": "no",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/fp1"
+ },
+ "threatactor": {
+ "name": [
+ "APT-FP-TEST"
+ ]
+ },
+ "url": [
+ "hxxps://evil.test.com"
+ ],
+ "vulnerability": {
+ "name": [
+ "CVE-2026-0001"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "threat"
+ ],
+ "created": "2026-02-20T10:00:00.000Z",
+ "id": "FINGERPRINT-TEST-ALERT-001",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "indicator"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001"
+ },
+ "message": "Multi-entity alert for fingerprint testing",
+ "related": {
+ "ip": [
+ "10.0.0.1"
+ ]
+ },
+ "threat": {
+ "enrichments": [
+ {
+ "indicator": {
+ "ip": "10.0.0.1",
+ "name": "10[.]0[.]0[.]1",
+ "type": "ipv4-addr"
+ }
+ },
+ {
+ "indicator": {
+ "name": "hxxps://evil[.]test[.]com",
+ "type": "url",
+ "url": {
+ "original": "hxxps://evil.test.com"
+ }
+ }
+ }
+ ]
+ }
+ },
+ {
+ "@timestamp": "2026-02-20T10:01:00.000Z",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Flash"
+ },
+ "categories": {
+ "name": [
+ "Vulnerability Disclosure"
+ ]
+ },
+ "cursor_used": "initial",
+ "discovered_entity": {},
+ "intel_agents": {
+ "summary": "Test summary."
+ },
+ "ip": [
+ "10.0.0.1"
+ ],
+ "is_main": "no",
+ "source": {
+ "channels": [
+ "chatter"
+ ],
+ "href": "https://r.dataminr.com/fp1"
+ },
+ "threatactor": {
+ "name": [
+ "APT-FP-TEST"
+ ]
+ },
+ "url": [
+ "hxxps://evil.test.com"
+ ],
+ "vulnerability": {
+ "name": [
+ "CVE-2026-0001"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "created": "2026-02-20T10:00:00.000Z",
+ "id": "FINGERPRINT-TEST-ALERT-001",
+ "kind": "alert",
+ "severity": 90,
+ "type": [
+ "info"
+ ],
+ "url": "https://app.dataminr.com/#alertDetail/5/FINGERPRINT-TEST-ALERT-001"
+ },
+ "message": "Multi-entity alert for fingerprint testing",
+ "related": {
+ "ip": [
+ "10.0.0.1"
+ ]
+ }
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json
new file mode 100644
index 00000000000..2566840bb80
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json
@@ -0,0 +1,16 @@
+{
+ "events": [
+ {
+ "message": "{\"log_type\":\"auth\",\"status\":\"success\",\"client_id\":\"test-client-id\",\"expires_at\":1739361900000,\"http_status_code\":200,\"fetch_timestamp\":\"2026-02-12T11:05:00.000Z\"}"
+ },
+ {
+ "message": "{\"log_type\":\"auth\",\"status\":\"failed\",\"client_id\":\"test-client-id\",\"http_status_code\":401,\"error_message\":\"Authentication failed with HTTP 401\",\"error_code\":401,\"fetch_timestamp\":\"2026-02-12T11:05:00.000Z\",\"response_body\":\"{\\\"errors\\\":[{\\\"code\\\":401,\\\"message\\\":\\\"Authentication failed with HTTP 401\\\"}]}\"}"
+ },
+ {
+ "message": "{\"log_type\":\"alert-fetch\",\"status\":\"success\",\"api_endpoint\":\"https://api.dataminr.com/pulse/v1/alerts?pageSize=40\",\"input_alert_fetch_count\":40,\"next_cursor\":\"/v1/alerts?from=abc123\",\"count_of_alerts\":40,\"http_status_code\":200,\"cursor_used\":\"initial\",\"fetched_alerts\":40,\"fetch_timestamp\":\"2026-02-12T11:05:00.000Z\"}"
+ },
+ {
+ "message": "{\"log_type\":\"alert-fetch\",\"status\":\"failed\",\"api_endpoint\":\"https://api.dataminr.com/pulse/v1/alerts?pageSize=40\",\"input_alert_fetch_count\":40,\"http_status_code\":403,\"error_message\":\"API request failed with HTTP 403\",\"cursor_used\":\"initial\",\"fetched_alerts\":0,\"fetch_timestamp\":\"2026-02-12T11:05:00.000Z\",\"response_body\":\"{\\\"error\\\":\\\"Forbidden\\\"}\"}"
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json-expected.json b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json-expected.json
new file mode 100644
index 00000000000..8fd954fe273
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/pipeline/test-log-events.json-expected.json
@@ -0,0 +1,116 @@
+{
+ "expected": [
+ {
+ "@timestamp": "2026-02-12T11:05:00.000Z",
+ "dataminr_pulse": {
+ "log": {
+ "client_id": "test-client-id",
+ "expires_at": "2025-02-12T12:05:00.000Z",
+ "fetch_timestamp": "2026-02-12T11:05:00.000Z",
+ "http_status_code": 200,
+ "log_type": "auth",
+ "status": "success"
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "authentication"
+ ],
+ "kind": "event",
+ "type": [
+ "info"
+ ]
+ }
+ },
+ {
+ "@timestamp": "2026-02-12T11:05:00.000Z",
+ "dataminr_pulse": {
+ "log": {
+ "client_id": "test-client-id",
+ "error_code": 401,
+ "error_message": "Authentication failed with HTTP 401",
+ "fetch_timestamp": "2026-02-12T11:05:00.000Z",
+ "http_status_code": 401,
+ "log_type": "auth",
+ "response_body": "{\"errors\":[{\"code\":401,\"message\":\"Authentication failed with HTTP 401\"}]}",
+ "status": "failed"
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "authentication"
+ ],
+ "kind": "event",
+ "type": [
+ "info"
+ ]
+ },
+ "message": "Authentication failed with HTTP 401"
+ },
+ {
+ "@timestamp": "2026-02-12T11:05:00.000Z",
+ "dataminr_pulse": {
+ "log": {
+ "api_endpoint": "https://api.dataminr.com/pulse/v1/alerts?pageSize=40",
+ "count_of_alerts": 40,
+ "fetch_timestamp": "2026-02-12T11:05:00.000Z",
+ "fetched_alerts": 40,
+ "http_status_code": 200,
+ "input_alert_fetch_count": 40,
+ "log_type": "alert-fetch",
+ "next_cursor": "/v1/alerts?from=abc123",
+ "status": "success"
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "kind": "event",
+ "outcome": "success",
+ "type": [
+ "access"
+ ]
+ }
+ },
+ {
+ "@timestamp": "2026-02-12T11:05:00.000Z",
+ "dataminr_pulse": {
+ "log": {
+ "api_endpoint": "https://api.dataminr.com/pulse/v1/alerts?pageSize=40",
+ "error_message": "API request failed with HTTP 403",
+ "fetch_timestamp": "2026-02-12T11:05:00.000Z",
+ "fetched_alerts": 0,
+ "http_status_code": 403,
+ "input_alert_fetch_count": 40,
+ "log_type": "alert-fetch",
+ "response_body": "{\"error\":\"Forbidden\"}",
+ "status": "failed"
+ }
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "category": [
+ "web"
+ ],
+ "kind": "event",
+ "outcome": "failure",
+ "type": [
+ "error"
+ ]
+ },
+ "message": "API request failed with HTTP 403"
+ }
+ ]
+}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.expected b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.expected
new file mode 100644
index 00000000000..6564aac98a2
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.expected
@@ -0,0 +1,64 @@
+inputs:
+ - data_stream:
+ namespace: ep
+ meta:
+ package:
+ name: dataminr_pulse
+ name: test-custom-config-elastic-siem-integration
+ streams:
+ - config_version: 2
+ data_stream:
+ dataset: dataminr_pulse.alerts
+ interval: 10m
+ max_executions: 10
+ processors:
+ - add_fields:
+ fields:
+ custom_field: custom_value
+ target: ""
+ program: "timestamp(now).as(fetch_ts,\n [408, 429, 500, 502, 503, 504].as(retryable_codes,\n (state.access_token == \"\" || \n (int(state.expires_at) > 0 && \n fetch_ts >= timestamp(\"1970-01-01T00:00:00Z\") + duration(string(int(state.expires_at) - int(state.token_expiry_buffer_ms)) + \"ms\"))\n ).as(need_auth,\n need_auth ?\n post(\n state.auth_url,\n \"application/x-www-form-urlencoded\",\n {\n \"grant_type\": [\"api_key\"],\n \"client_id\": [state.client_id],\n \"client_secret\": [state.client_secret]\n }.format_query()\n ).as(auth_resp,\n (size(auth_resp.Body) > 0 ? bytes(auth_resp.Body).decode_json() : {}).as(auth_body,\n auth_resp.StatusCode == 200 && has(auth_body.dmaToken) && auth_body.dmaToken != \"\" ?\n {\n \"new_state\": state.with({\n \"access_token\": string(auth_body.dmaToken),\n \"expires_at\": has(auth_body.expire) ? int(auth_body.expire) : 0\n }),\n \"auth_event\": {\n \"message\": {\n \"log_type\": \"auth\",\n \"status\": \"success\",\n \"client_id\": state.client_id,\n \"expires_at\": has(auth_body.expire) ? int(auth_body.expire) : 0,\n \"http_status_code\": auth_resp.StatusCode,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n },\n \"should_fetch\": true,\n \"auth_failed\": false\n }\n :\n retryable_codes.exists(c, c == auth_resp.StatusCode).as(is_retryable,\n (has(auth_body.errors) && size(auth_body.errors) > 0 ?\n string(auth_body.errors[0].message) :\n auth_resp.StatusCode == 200 ? \"No valid token received\" : \"Authentication failed with HTTP \" + string(auth_resp.StatusCode)\n ).as(error_message,\n (has(auth_body.errors) && size(auth_body.errors) > 0 ? int(auth_body.errors[0].code) : 0).as(error_code,\n {\n \"new_state\": state,\n \"auth_event\": {\n \"message\": (is_retryable ?\n {\n \"log_type\": \"auth\",\n \"status\": \"retry\",\n \"client_id\": state.client_id,\n \"error_message\": error_message,\n \"error_code\": error_code,\n \"http_status_code\": auth_resp.StatusCode,\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetch_timestamp\": fetch_ts\n }\n :\n {\n \"log_type\": \"auth\",\n \"status\": \"failed\",\n \"client_id\": state.client_id,\n \"error_message\": error_message,\n \"error_code\": error_code,\n \"http_status_code\": auth_resp.StatusCode,\n \"fetch_timestamp\": fetch_ts,\n \"response_body\": string(auth_resp.Body)\n }\n ).encode_json()\n },\n \"should_fetch\": false,\n \"auth_failed\": !is_retryable,\n \"want_more\": is_retryable\n }\n )\n )\n )\n )\n )\n :\n {\n \"new_state\": state,\n \"auth_event\": {},\n \"should_fetch\": true,\n \"auth_failed\": false\n }\n ).as(auth_result,\n auth_result.should_fetch ?\n (auth_result.new_state.next_cursor != \"\" ?\n auth_result.new_state.base_url + auth_result.new_state.next_cursor :\n auth_result.new_state.url + \"?\" + {\"pageSize\": [string(state.page_size)]}.format_query()\n ).as(request_url,\n get_request(request_url).with({\n \"Header\": {\n \"Authorization\": [\"Bearer \" + auth_result.new_state.access_token],\n \"Accept\": [\"application/json\"],\n \"x-application-name\": [\"connector_elastic_siem\"],\n \"x-application-version\": [state.version]\n }\n }).do_request().as(api_resp,\n (size(api_resp.Body) > 0 ? bytes(api_resp.Body).decode_json() : {}).as(api_body,\n api_resp.StatusCode == 200 ?\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) +\n (has(api_body.alerts) && size(api_body.alerts) > 0 ?\n api_body.alerts.map(alert,\n has(alert.intelAgents) && size(alert.intelAgents) > 0 ?\n alert.intelAgents.map(agent,\n has(agent.discoveredEntities) && size(agent.discoveredEntities) > 0 ?\n agent.discoveredEntities[0].as(first_entity,\n agent.discoveredEntities.filter(entity, \n entity.type == \"vulnerability\" || entity.type == \"threatActor\" || entity.type == \"malware\"\n ).as(individual_entities,\n agent.discoveredEntities.exists(e, e.type == \"ipAddress\" || e.type == \"url\").as(has_ip_url,\n agent.discoveredEntities.exists(e, \n e.type != \"vulnerability\" && e.type != \"threatActor\" && e.type != \"malware\" && \n e.type != \"ipAddress\" && e.type != \"url\"\n ).as(has_other,\n (\n individual_entities.map(entity,\n {\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"discovered_entity\": entity,\n \"entityType\": entity.type,\n \"is_main\": (entity.type == first_entity.type && entity.name == first_entity.name) ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }\n ) +\n (has_ip_url ? [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"url-ipAddress\",\n \"is_main\": (first_entity.type == \"ipAddress\" || first_entity.type == \"url\") ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }] : []) +\n (has_other ? [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"consolidated\",\n \"is_main\": (first_entity.type != \"vulnerability\" && first_entity.type != \"threatActor\" &&\n first_entity.type != \"malware\" && first_entity.type != \"ipAddress\" &&\n first_entity.type != \"url\") ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }] : [])\n )\n )\n )\n )\n )\n :\n [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"no-entities\",\n \"is_main\": \"yes\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }]\n ).flatten()\n :\n [{\n \"message\": alert.with({\n \"entityType\": \"no-intelAgents\",\n \"is_main\": \"yes\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }]\n ).flatten() : []\n ) + [{\n \"message\": {\n \"log_type\": \"alert-fetch\",\n \"status\": \"success\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"next_cursor\": has(api_body.nextPage) && api_body.nextPage != \"\" ? string(api_body.nextPage) : \"\",\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": has(api_body.alerts) ? size(api_body.alerts) : 0,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n }],\n \"next_cursor\": has(api_body.nextPage) && api_body.nextPage != \"\" ? string(api_body.nextPage) : \"\",\n \"access_token\": auth_result.new_state.access_token,\n \"expires_at\": auth_result.new_state.expires_at,\n \"last_fetch_timestamp\": fetch_ts,\n \"retry_count\": 0,\n \"want_more\": false\n })\n :\n api_resp.StatusCode == 401 ?\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{\n \"message\": {\n \"log_type\": \"alert-fetch\",\n \"status\": \"retry\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": \"Authentication failed - token will be regenerated\",\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n }],\n \"access_token\": \"\",\n \"expires_at\": 0,\n \"retry_count\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"want_more\": true\n })\n :\n retryable_codes.exists(c, c == api_resp.StatusCode).as(is_retryable,\n (has(api_body.error) ? string(api_body.error) : \"API request failed with HTTP \" + string(api_resp.StatusCode)).as(error_message,\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{\n \"message\": (is_retryable ?\n {\n \"log_type\": \"alert-fetch\",\n \"status\": \"retry\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": error_message,\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts\n }\n :\n {\n \"log_type\": \"alert-fetch\",\n \"status\": \"failed\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": error_message,\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts,\n \"response_body\": string(api_resp.Body)\n }\n ).encode_json()\n }],\n \"access_token\": auth_result.new_state.access_token,\n \"expires_at\": auth_result.new_state.expires_at,\n \"next_cursor\": (api_resp.StatusCode == 400 || api_resp.StatusCode == 404) ? \"\" : auth_result.new_state.next_cursor,\n \"retry_count\": is_retryable ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,\n \"want_more\": is_retryable\n })\n )\n )\n )\n )\n )\n :\n auth_result.new_state.with({\n \"events\": [auth_result.auth_event],\n \"retry_count\": (has(auth_result.want_more) && auth_result.want_more) ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,\n \"want_more\": has(auth_result.want_more) ? auth_result.want_more : false\n })\n )\n )\n)\n"
+ redact:
+ fields:
+ - client_id
+ - client_secret
+ - access_token
+ resource.retry.max_attempts: 3
+ resource.retry.wait_max: 30s
+ resource.retry.wait_min: 10s
+ resource.tracer.filename: ../../logs/cel/http-request-trace-*.ndjson
+ resource.tracer.maxbackups: 5
+ resource.url: https://custom.dataminr.com/pulse/v1/alerts
+ state:
+ access_token: ""
+ auth_url: https://custom.dataminr.com/auth/2/token
+ base_url: https://custom.dataminr.com/pulse
+ client_id: custom-client-id
+ client_secret: ${SECRET_0}
+ expires_at: 0
+ last_fetch_timestamp: 0
+ next_cursor: ""
+ page_size: 40
+ retry_count: 0
+ token_expiry_buffer_ms: 60000
+ url: https://custom.dataminr.com/pulse/v1/alerts
+ version: ""
+ tags:
+ - preserve_original_event
+ type: cel
+ use_output: default
+output_permissions:
+ default:
+ _elastic_agent_checks:
+ cluster:
+ - monitor
+ _elastic_agent_monitoring:
+ indices: []
+ uuid-for-permissions-on-related-indices:
+ indices:
+ - names:
+ - logs-dataminr_pulse.alerts-ep
+ privileges:
+ - auto_configure
+ - create_doc
+secret_references:
+ - {}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.yml b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.yml
new file mode 100644
index 00000000000..9e0805d97fe
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-custom-config.yml
@@ -0,0 +1,22 @@
+vars: ~
+data_stream:
+ vars:
+ url: https://custom.dataminr.com/pulse/v1/alerts
+ base_url: https://custom.dataminr.com/pulse
+ auth_url: https://custom.dataminr.com/auth/2/token
+ token_expiry_buffer_ms: 60000
+ client_id: custom-client-id
+ client_secret: custom-client-secret
+ interval: 10m
+ page_size: "40"
+ enable_request_tracer: true
+ tags:
+ - forwarded
+ - dataminr-pulse-alerts
+ - custom-tag
+ preserve_original_event: true
+ processors: |-
+ - add_fields:
+ target: ''
+ fields:
+ custom_field: custom_value
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.expected b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.expected
new file mode 100644
index 00000000000..b0e60401ade
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.expected
@@ -0,0 +1,56 @@
+inputs:
+ - data_stream:
+ namespace: ep
+ meta:
+ package:
+ name: dataminr_pulse
+ name: test-default-config-elastic-siem-integration
+ streams:
+ - config_version: 2
+ data_stream:
+ dataset: dataminr_pulse.alerts
+ interval: 5m
+ max_executions: 10
+ program: "timestamp(now).as(fetch_ts,\n [408, 429, 500, 502, 503, 504].as(retryable_codes,\n (state.access_token == \"\" || \n (int(state.expires_at) > 0 && \n fetch_ts >= timestamp(\"1970-01-01T00:00:00Z\") + duration(string(int(state.expires_at) - int(state.token_expiry_buffer_ms)) + \"ms\"))\n ).as(need_auth,\n need_auth ?\n post(\n state.auth_url,\n \"application/x-www-form-urlencoded\",\n {\n \"grant_type\": [\"api_key\"],\n \"client_id\": [state.client_id],\n \"client_secret\": [state.client_secret]\n }.format_query()\n ).as(auth_resp,\n (size(auth_resp.Body) > 0 ? bytes(auth_resp.Body).decode_json() : {}).as(auth_body,\n auth_resp.StatusCode == 200 && has(auth_body.dmaToken) && auth_body.dmaToken != \"\" ?\n {\n \"new_state\": state.with({\n \"access_token\": string(auth_body.dmaToken),\n \"expires_at\": has(auth_body.expire) ? int(auth_body.expire) : 0\n }),\n \"auth_event\": {\n \"message\": {\n \"log_type\": \"auth\",\n \"status\": \"success\",\n \"client_id\": state.client_id,\n \"expires_at\": has(auth_body.expire) ? int(auth_body.expire) : 0,\n \"http_status_code\": auth_resp.StatusCode,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n },\n \"should_fetch\": true,\n \"auth_failed\": false\n }\n :\n retryable_codes.exists(c, c == auth_resp.StatusCode).as(is_retryable,\n (has(auth_body.errors) && size(auth_body.errors) > 0 ?\n string(auth_body.errors[0].message) :\n auth_resp.StatusCode == 200 ? \"No valid token received\" : \"Authentication failed with HTTP \" + string(auth_resp.StatusCode)\n ).as(error_message,\n (has(auth_body.errors) && size(auth_body.errors) > 0 ? int(auth_body.errors[0].code) : 0).as(error_code,\n {\n \"new_state\": state,\n \"auth_event\": {\n \"message\": (is_retryable ?\n {\n \"log_type\": \"auth\",\n \"status\": \"retry\",\n \"client_id\": state.client_id,\n \"error_message\": error_message,\n \"error_code\": error_code,\n \"http_status_code\": auth_resp.StatusCode,\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetch_timestamp\": fetch_ts\n }\n :\n {\n \"log_type\": \"auth\",\n \"status\": \"failed\",\n \"client_id\": state.client_id,\n \"error_message\": error_message,\n \"error_code\": error_code,\n \"http_status_code\": auth_resp.StatusCode,\n \"fetch_timestamp\": fetch_ts,\n \"response_body\": string(auth_resp.Body)\n }\n ).encode_json()\n },\n \"should_fetch\": false,\n \"auth_failed\": !is_retryable,\n \"want_more\": is_retryable\n }\n )\n )\n )\n )\n )\n :\n {\n \"new_state\": state,\n \"auth_event\": {},\n \"should_fetch\": true,\n \"auth_failed\": false\n }\n ).as(auth_result,\n auth_result.should_fetch ?\n (auth_result.new_state.next_cursor != \"\" ?\n auth_result.new_state.base_url + auth_result.new_state.next_cursor :\n auth_result.new_state.url + \"?\" + {\"pageSize\": [string(state.page_size)]}.format_query()\n ).as(request_url,\n get_request(request_url).with({\n \"Header\": {\n \"Authorization\": [\"Bearer \" + auth_result.new_state.access_token],\n \"Accept\": [\"application/json\"],\n \"x-application-name\": [\"connector_elastic_siem\"],\n \"x-application-version\": [state.version]\n }\n }).do_request().as(api_resp,\n (size(api_resp.Body) > 0 ? bytes(api_resp.Body).decode_json() : {}).as(api_body,\n api_resp.StatusCode == 200 ?\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) +\n (has(api_body.alerts) && size(api_body.alerts) > 0 ?\n api_body.alerts.map(alert,\n has(alert.intelAgents) && size(alert.intelAgents) > 0 ?\n alert.intelAgents.map(agent,\n has(agent.discoveredEntities) && size(agent.discoveredEntities) > 0 ?\n agent.discoveredEntities[0].as(first_entity,\n agent.discoveredEntities.filter(entity, \n entity.type == \"vulnerability\" || entity.type == \"threatActor\" || entity.type == \"malware\"\n ).as(individual_entities,\n agent.discoveredEntities.exists(e, e.type == \"ipAddress\" || e.type == \"url\").as(has_ip_url,\n agent.discoveredEntities.exists(e, \n e.type != \"vulnerability\" && e.type != \"threatActor\" && e.type != \"malware\" && \n e.type != \"ipAddress\" && e.type != \"url\"\n ).as(has_other,\n (\n individual_entities.map(entity,\n {\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"discovered_entity\": entity,\n \"entityType\": entity.type,\n \"is_main\": (entity.type == first_entity.type && entity.name == first_entity.name) ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }\n ) +\n (has_ip_url ? [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"url-ipAddress\",\n \"is_main\": (first_entity.type == \"ipAddress\" || first_entity.type == \"url\") ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }] : []) +\n (has_other ? [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"consolidated\",\n \"is_main\": (first_entity.type != \"vulnerability\" && first_entity.type != \"threatActor\" &&\n first_entity.type != \"malware\" && first_entity.type != \"ipAddress\" &&\n first_entity.type != \"url\") ? \"yes\" : \"no\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }] : [])\n )\n )\n )\n )\n )\n :\n [{\n \"message\": alert.with({\n \"intelAgents\": alert.intelAgents,\n \"entityType\": \"no-entities\",\n \"is_main\": \"yes\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }]\n ).flatten()\n :\n [{\n \"message\": alert.with({\n \"entityType\": \"no-intelAgents\",\n \"is_main\": \"yes\",\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": size(api_body.alerts),\n \"fetch_timestamp\": fetch_ts\n }).encode_json()\n }]\n ).flatten() : []\n ) + [{\n \"message\": {\n \"log_type\": \"alert-fetch\",\n \"status\": \"success\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"next_cursor\": has(api_body.nextPage) && api_body.nextPage != \"\" ? string(api_body.nextPage) : \"\",\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": has(api_body.alerts) ? size(api_body.alerts) : 0,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n }],\n \"next_cursor\": has(api_body.nextPage) && api_body.nextPage != \"\" ? string(api_body.nextPage) : \"\",\n \"access_token\": auth_result.new_state.access_token,\n \"expires_at\": auth_result.new_state.expires_at,\n \"last_fetch_timestamp\": fetch_ts,\n \"retry_count\": 0,\n \"want_more\": false\n })\n :\n api_resp.StatusCode == 401 ?\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{\n \"message\": {\n \"log_type\": \"alert-fetch\",\n \"status\": \"retry\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": \"Authentication failed - token will be regenerated\",\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts\n }.encode_json()\n }],\n \"access_token\": \"\",\n \"expires_at\": 0,\n \"retry_count\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"want_more\": true\n })\n :\n retryable_codes.exists(c, c == api_resp.StatusCode).as(is_retryable,\n (has(api_body.error) ? string(api_body.error) : \"API request failed with HTTP \" + string(api_resp.StatusCode)).as(error_message,\n auth_result.new_state.with({\n \"events\": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{\n \"message\": (is_retryable ?\n {\n \"log_type\": \"alert-fetch\",\n \"status\": \"retry\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": error_message,\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"retry_number\": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts\n }\n :\n {\n \"log_type\": \"alert-fetch\",\n \"status\": \"failed\",\n \"api_endpoint\": request_url,\n \"input_alert_fetch_count\": int(state.page_size),\n \"error_message\": error_message,\n \"http_status_code\": api_resp.StatusCode,\n \"cursor_used\": auth_result.new_state.next_cursor != \"\" ? auth_result.new_state.next_cursor : \"initial\",\n \"fetched_alerts\": 0,\n \"fetch_timestamp\": fetch_ts,\n \"response_body\": string(api_resp.Body)\n }\n ).encode_json()\n }],\n \"access_token\": auth_result.new_state.access_token,\n \"expires_at\": auth_result.new_state.expires_at,\n \"next_cursor\": (api_resp.StatusCode == 400 || api_resp.StatusCode == 404) ? \"\" : auth_result.new_state.next_cursor,\n \"retry_count\": is_retryable ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,\n \"want_more\": is_retryable\n })\n )\n )\n )\n )\n )\n :\n auth_result.new_state.with({\n \"events\": [auth_result.auth_event],\n \"retry_count\": (has(auth_result.want_more) && auth_result.want_more) ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,\n \"want_more\": has(auth_result.want_more) ? auth_result.want_more : false\n })\n )\n )\n)\n"
+ redact:
+ fields:
+ - client_id
+ - client_secret
+ - access_token
+ resource.retry.max_attempts: 3
+ resource.retry.wait_max: 30s
+ resource.retry.wait_min: 10s
+ resource.url: https://api.dataminr.com/pulse/v1/alerts
+ state:
+ access_token: ""
+ auth_url: https://userauth.dataminr.com/auth/2/token
+ base_url: https://api.dataminr.com/pulse
+ client_id: test-client-id
+ client_secret: ${SECRET_0}
+ expires_at: 0
+ last_fetch_timestamp: 0
+ next_cursor: ""
+ page_size: 40
+ retry_count: 0
+ token_expiry_buffer_ms: 300000
+ url: https://api.dataminr.com/pulse/v1/alerts
+ version: ""
+ tags: null
+ type: cel
+ use_output: default
+output_permissions:
+ default:
+ _elastic_agent_checks:
+ cluster:
+ - monitor
+ _elastic_agent_monitoring:
+ indices: []
+ uuid-for-permissions-on-related-indices:
+ indices:
+ - names:
+ - logs-dataminr_pulse.alerts-ep
+ privileges:
+ - auto_configure
+ - create_doc
+secret_references:
+ - {}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.yml b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.yml
new file mode 100644
index 00000000000..0fa8a5de6c5
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/policy/test-default-config.yml
@@ -0,0 +1,5 @@
+vars: ~
+data_stream:
+ vars:
+ client_id: test-client-id
+ client_secret: test-client-secret
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/scripts/pipeline_simulate.txt b/packages/dataminr_pulse/data_stream/alerts/_dev/test/scripts/pipeline_simulate.txt
new file mode 100644
index 00000000000..307ef69a9cb
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/scripts/pipeline_simulate.txt
@@ -0,0 +1,51 @@
+# Use the existing managed stack.
+use_stack -profile ${CONFIG_PROFILES}/${PROFILE}
+! stderr .
+
+# Install the data stream's pipeline.
+install_pipelines -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT}
+! stderr .
+stdout '^installed pipelines in '${DATA_STREAM}' with nonce'
+
+# Simulate a basic alert event and verify ECS mappings.
+simulate -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT} default data-alert.json
+! stderr .
+stdout '"@timestamp": "2026-01-15T10:00:00.000Z"'
+stdout '"message": "Test vulnerability alert"'
+stdout '"severity": 90'
+stdout '"kind": "alert"'
+stdout '"id": "test-alert-001"'
+
+# Simulate an entity event and verify discovered_entity mapping.
+simulate -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT} default data-entity.json
+! stderr .
+stdout '"discovered_entity"'
+stdout '"cve": "CVE-2026-9999"'
+stdout '"value": "CVE-2026-9999"'
+stdout '"is_main": "yes"'
+
+# Simulate an event with geo coordinates and verify lat/lon mapping.
+simulate -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT} default data-geo.json
+! stderr .
+stdout '"lat": 40.7128'
+stdout '"lon": -74.006'
+stdout '"name": "New York, NY, USA"'
+
+# Simulate an event without geo coordinates and verify no geo fields.
+simulate -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT} default data-no-geo.json
+! stderr .
+! stdout '"geo"'
+
+# Uninstall pipelines.
+uninstall_pipelines -profile ${CONFIG_PROFILES}/${PROFILE} ${DATA_STREAM_ROOT}
+! stderr .
+stdout '^uninstalled pipelines in '${DATA_STREAM}
+
+-- data-alert.json --
+{"message": "{\"headline\":\"Test vulnerability alert\",\"alertId\":\"test-alert-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTimestamp\":\"2026-01-15T10:00:00.000Z\",\"publicPost\":{\"timestamp\":\"2026-01-15T09:59:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/test1\"},\"alertTopics\":[{\"id\":\"100\",\"name\":\"Vulnerability Disclosure\"}],\"listsMatched\":[{\"id\":\"1\",\"name\":\"Cyber\",\"topicIds\":[\"100\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/test-alert-001\"}"}
+-- data-entity.json --
+{"message": "{\"headline\":\"Entity event with discovered entity\",\"alertId\":\"test-entity-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTimestamp\":\"2026-01-15T10:00:00.000Z\",\"publicPost\":{\"timestamp\":\"2026-01-15T09:59:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/test2\"},\"alertTopics\":[{\"id\":\"100\",\"name\":\"Vulnerability Disclosure\"}],\"listsMatched\":[{\"id\":\"1\",\"name\":\"Cyber\",\"topicIds\":[\"100\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/test-entity-001\",\"discovered_entity\":{\"type\":\"vulnerability\",\"name\":\"CVE-2026-9999\",\"cve\":\"CVE-2026-9999\",\"cvss\":8.5,\"summary\":\"Test vulnerability\"},\"discoveredentity_type\":\"vulnerability\",\"discoveredentity_value\":\"CVE-2026-9999\",\"entityType\":\"vulnerability\",\"is_main\":\"yes\",\"intelAgents\":[{\"summary\":\"Test summary\",\"timestamp\":\"2026-01-15T10:00:30.000Z\",\"version\":\"1.0\",\"discoveredEntities\":[{\"type\":\"vulnerability\",\"name\":\"CVE-2026-9999\",\"cve\":\"CVE-2026-9999\",\"cvss\":8.5,\"summary\":\"Test vulnerability\"}]}]}"}
+-- data-geo.json --
+{"message": "{\"headline\":\"Geo test alert\",\"alertId\":\"test-geo-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTimestamp\":\"2026-01-15T10:00:00.000Z\",\"publicPost\":{\"timestamp\":\"2026-01-15T09:59:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/geo1\"},\"alertTopics\":[{\"id\":\"100\",\"name\":\"Test\"}],\"listsMatched\":[{\"id\":\"1\",\"name\":\"Cyber\",\"topicIds\":[\"100\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/test-geo-001\",\"estimatedEventLocation\":{\"name\":\"New York, NY, USA\",\"coordinates\":[40.7128,-74.006],\"probabilityRadius\":0.5}}"}
+-- data-no-geo.json --
+{"message": "{\"headline\":\"No geo alert\",\"alertId\":\"test-nogeo-001\",\"alertType\":{\"name\":\"Flash\"},\"alertTimestamp\":\"2026-01-15T10:00:00.000Z\",\"publicPost\":{\"timestamp\":\"2026-01-15T09:59:00.000Z\",\"channels\":[\"chatter\"],\"href\":\"https://r.dataminr.com/nogeo1\"},\"alertTopics\":[{\"id\":\"100\",\"name\":\"Test\"}],\"listsMatched\":[{\"id\":\"1\",\"name\":\"Cyber\",\"topicIds\":[\"100\"],\"subType\":\"DIGITAL_RISK\"}],\"dataminrAlertUrl\":\"https://app.dataminr.com/#alertDetail/5/test-nogeo-001\"}"}
diff --git a/packages/dataminr_pulse/data_stream/alerts/_dev/test/system/test-default-config.yml b/packages/dataminr_pulse/data_stream/alerts/_dev/test/system/test-default-config.yml
new file mode 100644
index 00000000000..4bc0bea2a0b
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/_dev/test/system/test-default-config.yml
@@ -0,0 +1,12 @@
+vars: ~
+data_stream:
+ vars:
+ url: http://{{Hostname}}:{{Port}}/pulse/v1/alerts
+ base_url: http://{{Hostname}}:{{Port}}/pulse
+ auth_url: http://{{Hostname}}:{{Port}}/auth/2/token
+ client_id: test-client-id
+ client_secret: test-client-secret
+ interval: 5m
+ page_size: "40"
+assert:
+ hit_count: 5
diff --git a/packages/dataminr_pulse/data_stream/alerts/agent/stream/stream.yml.hbs b/packages/dataminr_pulse/data_stream/alerts/agent/stream/stream.yml.hbs
new file mode 100644
index 00000000000..4e319db08b5
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/agent/stream/stream.yml.hbs
@@ -0,0 +1,331 @@
+config_version: 2
+interval: {{interval}}
+max_executions: 10
+{{#if enable_request_tracer}}
+resource.tracer.filename: "../../logs/cel/http-request-trace-*.ndjson"
+resource.tracer.maxbackups: 5
+{{/if}}
+{{#if proxy_url}}
+resource.proxy_url: {{proxy_url}}
+{{/if}}
+{{#if ssl}}
+resource.ssl: {{ssl}}
+{{/if}}
+{{#if http_client_timeout}}
+resource.timeout: {{http_client_timeout}}
+{{/if}}
+resource.url: {{url}}
+resource.retry.max_attempts: 3
+resource.retry.wait_min: 10s
+resource.retry.wait_max: 30s
+state:
+ url: {{url}}
+ base_url: {{base_url}}
+ auth_url: {{auth_url}}
+ token_expiry_buffer_ms: {{token_expiry_buffer_ms}}
+ client_id: {{client_id}}
+ client_secret: {{client_secret}}
+ version: "{{version}}"
+ page_size: {{page_size}}
+ next_cursor: ""
+ access_token: ""
+ expires_at: 0
+ last_fetch_timestamp: 0
+ retry_count: 0
+redact:
+ fields:
+ - client_id
+ - client_secret
+ - access_token
+program: |
+ timestamp(now).as(fetch_ts,
+ [408, 429, 500, 502, 503, 504].as(retryable_codes,
+ (state.access_token == "" ||
+ (int(state.expires_at) > 0 &&
+ fetch_ts >= timestamp("1970-01-01T00:00:00Z") + duration(string(int(state.expires_at) - int(state.token_expiry_buffer_ms)) + "ms"))
+ ).as(need_auth,
+ need_auth ?
+ post(
+ state.auth_url,
+ "application/x-www-form-urlencoded",
+ {
+ "grant_type": ["api_key"],
+ "client_id": [state.client_id],
+ "client_secret": [state.client_secret]
+ }.format_query()
+ ).as(auth_resp,
+ (size(auth_resp.Body) > 0 ? bytes(auth_resp.Body).decode_json() : {}).as(auth_body,
+ auth_resp.StatusCode == 200 && has(auth_body.dmaToken) && auth_body.dmaToken != "" ?
+ {
+ "new_state": state.with({
+ "access_token": string(auth_body.dmaToken),
+ "expires_at": has(auth_body.expire) ? int(auth_body.expire) : 0
+ }),
+ "auth_event": {
+ "message": {
+ "log_type": "auth",
+ "status": "success",
+ "client_id": state.client_id,
+ "expires_at": has(auth_body.expire) ? int(auth_body.expire) : 0,
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ },
+ "should_fetch": true,
+ "auth_failed": false
+ }
+ :
+ retryable_codes.exists(c, c == auth_resp.StatusCode).as(is_retryable,
+ (has(auth_body.errors) && size(auth_body.errors) > 0 ?
+ string(auth_body.errors[0].message) :
+ auth_resp.StatusCode == 200 ? "No valid token received" : "Authentication failed with HTTP " + string(auth_resp.StatusCode)
+ ).as(error_message,
+ (has(auth_body.errors) && size(auth_body.errors) > 0 ? int(auth_body.errors[0].code) : 0).as(error_code,
+ {
+ "new_state": state,
+ "auth_event": {
+ "message": (is_retryable ?
+ {
+ "log_type": "auth",
+ "status": "retry",
+ "client_id": state.client_id,
+ "error_message": error_message,
+ "error_code": error_code,
+ "http_status_code": auth_resp.StatusCode,
+ "retry_number": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,
+ "fetch_timestamp": fetch_ts
+ }
+ :
+ {
+ "log_type": "auth",
+ "status": "failed",
+ "client_id": state.client_id,
+ "error_message": error_message,
+ "error_code": error_code,
+ "http_status_code": auth_resp.StatusCode,
+ "fetch_timestamp": fetch_ts,
+ "response_body": string(auth_resp.Body)
+ }
+ ).encode_json()
+ },
+ "should_fetch": false,
+ "auth_failed": !is_retryable,
+ "want_more": is_retryable
+ }
+ )
+ )
+ )
+ )
+ )
+ :
+ {
+ "new_state": state,
+ "auth_event": {},
+ "should_fetch": true,
+ "auth_failed": false
+ }
+ ).as(auth_result,
+ auth_result.should_fetch ?
+ (auth_result.new_state.next_cursor != "" ?
+ auth_result.new_state.base_url + auth_result.new_state.next_cursor :
+ auth_result.new_state.url + "?" + {"pageSize": [string(state.page_size)]}.format_query()
+ ).as(request_url,
+ get_request(request_url).with({
+ "Header": {
+ "Authorization": ["Bearer " + auth_result.new_state.access_token],
+ "Accept": ["application/json"],
+ "x-application-name": ["connector_elastic_siem"],
+ "x-application-version": [state.version]
+ }
+ }).do_request().as(api_resp,
+ (size(api_resp.Body) > 0 ? bytes(api_resp.Body).decode_json() : {}).as(api_body,
+ api_resp.StatusCode == 200 ?
+ auth_result.new_state.with({
+ "events": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) +
+ (has(api_body.alerts) && size(api_body.alerts) > 0 ?
+ api_body.alerts.map(alert,
+ has(alert.intelAgents) && size(alert.intelAgents) > 0 ?
+ alert.intelAgents.map(agent,
+ has(agent.discoveredEntities) && size(agent.discoveredEntities) > 0 ?
+ agent.discoveredEntities[0].as(first_entity,
+ agent.discoveredEntities.filter(entity,
+ entity.type == "vulnerability" || entity.type == "threatActor" || entity.type == "malware"
+ ).as(individual_entities,
+ agent.discoveredEntities.exists(e, e.type == "ipAddress" || e.type == "url").as(has_ip_url,
+ agent.discoveredEntities.exists(e,
+ e.type != "vulnerability" && e.type != "threatActor" && e.type != "malware" &&
+ e.type != "ipAddress" && e.type != "url"
+ ).as(has_other,
+ (
+ individual_entities.map(entity,
+ {
+ "message": alert.with({
+ "intelAgents": alert.intelAgents,
+ "discovered_entity": entity,
+ "entityType": entity.type,
+ "is_main": (entity.type == first_entity.type && entity.name == first_entity.name) ? "yes" : "no",
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": size(api_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }
+ ) +
+ (has_ip_url ? [{
+ "message": alert.with({
+ "intelAgents": alert.intelAgents,
+ "entityType": "url-ipAddress",
+ "is_main": (first_entity.type == "ipAddress" || first_entity.type == "url") ? "yes" : "no",
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": size(api_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }] : []) +
+ (has_other ? [{
+ "message": alert.with({
+ "intelAgents": alert.intelAgents,
+ "entityType": "consolidated",
+ "is_main": (first_entity.type != "vulnerability" && first_entity.type != "threatActor" &&
+ first_entity.type != "malware" && first_entity.type != "ipAddress" &&
+ first_entity.type != "url") ? "yes" : "no",
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": size(api_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }] : [])
+ )
+ )
+ )
+ )
+ )
+ :
+ [{
+ "message": alert.with({
+ "intelAgents": alert.intelAgents,
+ "entityType": "no-entities",
+ "is_main": "yes",
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": size(api_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }]
+ ).flatten()
+ :
+ [{
+ "message": alert.with({
+ "entityType": "no-intelAgents",
+ "is_main": "yes",
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": size(api_body.alerts),
+ "fetch_timestamp": fetch_ts
+ }).encode_json()
+ }]
+ ).flatten() : []
+ ) + [{
+ "message": {
+ "log_type": "alert-fetch",
+ "status": "success",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": int(state.page_size),
+ "next_cursor": has(api_body.nextPage) && api_body.nextPage != "" ? string(api_body.nextPage) : "",
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": has(api_body.alerts) ? size(api_body.alerts) : 0,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ }],
+ "next_cursor": has(api_body.nextPage) && api_body.nextPage != "" ? string(api_body.nextPage) : "",
+ "access_token": auth_result.new_state.access_token,
+ "expires_at": auth_result.new_state.expires_at,
+ "last_fetch_timestamp": fetch_ts,
+ "retry_count": 0,
+ "want_more": has(api_body.nextPage) && api_body.nextPage != ""
+ })
+ :
+ api_resp.StatusCode == 401 ?
+ auth_result.new_state.with({
+ "events": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{
+ "message": {
+ "log_type": "alert-fetch",
+ "status": "retry",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": int(state.page_size),
+ "error_message": "Authentication failed - token will be regenerated",
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "retry_number": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,
+ "fetched_alerts": 0,
+ "fetch_timestamp": fetch_ts
+ }.encode_json()
+ }],
+ "access_token": "",
+ "expires_at": 0,
+ "retry_count": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,
+ "want_more": true
+ })
+ :
+ retryable_codes.exists(c, c == api_resp.StatusCode).as(is_retryable,
+ (has(api_body.error) ? string(api_body.error) : "API request failed with HTTP " + string(api_resp.StatusCode)).as(error_message,
+ auth_result.new_state.with({
+ "events": (has(auth_result.auth_event.message) ? [auth_result.auth_event] : []) + [{
+ "message": (is_retryable ?
+ {
+ "log_type": "alert-fetch",
+ "status": "retry",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": int(state.page_size),
+ "error_message": error_message,
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "retry_number": (has(state.retry_count) ? int(state.retry_count) : 0) + 1,
+ "fetched_alerts": 0,
+ "fetch_timestamp": fetch_ts
+ }
+ :
+ {
+ "log_type": "alert-fetch",
+ "status": "failed",
+ "api_endpoint": request_url,
+ "input_alert_fetch_count": int(state.page_size),
+ "error_message": error_message,
+ "http_status_code": api_resp.StatusCode,
+ "cursor_used": auth_result.new_state.next_cursor != "" ? auth_result.new_state.next_cursor : "initial",
+ "fetched_alerts": 0,
+ "fetch_timestamp": fetch_ts,
+ "response_body": string(api_resp.Body)
+ }
+ ).encode_json()
+ }],
+ "access_token": auth_result.new_state.access_token,
+ "expires_at": auth_result.new_state.expires_at,
+ "next_cursor": (api_resp.StatusCode == 400 || api_resp.StatusCode == 404) ? "" : auth_result.new_state.next_cursor,
+ "retry_count": is_retryable ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,
+ "want_more": is_retryable
+ })
+ )
+ )
+ )
+ )
+ )
+ :
+ auth_result.new_state.with({
+ "events": [auth_result.auth_event],
+ "retry_count": (has(auth_result.want_more) && auth_result.want_more) ? (has(state.retry_count) ? int(state.retry_count) : 0) + 1 : 0,
+ "want_more": has(auth_result.want_more) ? auth_result.want_more : false
+ })
+ )
+ )
+ )
+tags:
+{{#if preserve_original_event}}
+ - preserve_original_event
+{{/if}}
+{{#each tags as |tag|}}
+ - {{tag}}
+{{/each}}
+{{#contains "forwarded" tags}}
+publisher_pipeline.disable_host: true
+{{/contains}}
+{{#if processors}}
+processors:
+{{processors}}
+{{/if}}
\ No newline at end of file
diff --git a/packages/dataminr_pulse/data_stream/alerts/elasticsearch/ingest_pipeline/default.yml b/packages/dataminr_pulse/data_stream/alerts/elasticsearch/ingest_pipeline/default.yml
new file mode 100644
index 00000000000..f9ba9efc4d5
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/elasticsearch/ingest_pipeline/default.yml
@@ -0,0 +1,1064 @@
+---
+description: Pipeline for processing Dataminr Pulse alerts with ECS mapping and request logging
+processors:
+ # Set ECS version and initial event fields
+ - set:
+ field: ecs.version
+ tag: set_ecs_version
+ value: '8.11.0'
+ - set:
+ field: event.kind
+ tag: set_event_kind_default
+ value: event
+ - append:
+ field: event.category
+ tag: append_web_into_event_category
+ value: web
+ allow_duplicates: false
+ - append:
+ field: event.type
+ tag: append_info_into_event_type
+ value: info
+ allow_duplicates: false
+ - rename:
+ field: message
+ tag: rename_message_to_event_original
+ target_field: event.original
+ if: ctx.event?.original == null
+ ignore_missing: true
+
+ # Parse JSON first to determine document type
+ - json:
+ field: event.original
+ tag: json_event_original
+ target_field: json
+
+ # Handle master documents
+ - set:
+ field: dataminr_pulse.is_main
+ copy_from: json.is_main
+ ignore_empty_value: true
+ if: ctx.json?.is_main != null
+ tag: set_is_main
+
+ # Handle log documents (auth or alert-fetch)
+ - set:
+ field: _temp.is_log
+ value: true
+ if: ctx.json?.log_type != null
+
+ # Process auth logs
+ - set:
+ field: event.kind
+ value: event
+ if: ctx.json?.log_type == 'auth'
+ override: true
+ - set:
+ field: event.category
+ value: []
+ if: ctx.json?.log_type == 'auth'
+ override: true
+ - append:
+ field: event.category
+ value: authentication
+ allow_duplicates: false
+ if: ctx.json?.log_type == 'auth'
+ - set:
+ field: event.type
+ value: []
+ if: ctx.json?.log_type == 'auth'
+ override: true
+ - append:
+ field: event.type
+ value: info
+ allow_duplicates: false
+ if: ctx.json?.log_type == 'auth'
+
+ # Process alert-fetch logs
+ - set:
+ field: event.kind
+ value: event
+ if: ctx.json?.log_type == 'alert-fetch'
+ override: true
+ - set:
+ field: event.category
+ value: []
+ if: ctx.json?.log_type == 'alert-fetch'
+ override: true
+ - append:
+ field: event.category
+ value: web
+ allow_duplicates: false
+ if: ctx.json?.log_type == 'alert-fetch'
+ - set:
+ field: event.type
+ value: []
+ if: ctx.json?.log_type == 'alert-fetch'
+ override: true
+ - append:
+ field: event.type
+ value: access
+ allow_duplicates: false
+ if: ctx.json?.log_type == 'alert-fetch' && ctx.json?.status == 'success'
+ - append:
+ field: event.type
+ value: error
+ allow_duplicates: false
+ if: ctx.json?.log_type == 'alert-fetch' && ctx.json?.status == 'failed'
+ - set:
+ field: event.outcome
+ value: success
+ if: ctx.json?.log_type == 'alert-fetch' && ctx.json?.status == 'success'
+ - set:
+ field: event.outcome
+ value: failure
+ if: ctx.json?.log_type == 'alert-fetch' && ctx.json?.status == 'failed'
+
+ # Map log fields to dataminr_pulse.log for all logs
+ - rename:
+ field: json
+ target_field: dataminr_pulse.log
+ tag: rename_json_to_log
+ if: ctx._temp?.is_log == true
+ ignore_missing: true
+
+ # Remove cursor_used from log object as it's an alert-level field
+ - remove:
+ field: dataminr_pulse.log.cursor_used
+ tag: remove_cursor_from_log
+ ignore_missing: true
+ if: ctx._temp?.is_log == true
+
+ # Map error message for logs (use message field, not error.message to avoid pipeline_error flag)
+ - set:
+ field: message
+ copy_from: dataminr_pulse.log.error_message
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.log?.error_message != null
+
+ # Map fetch_timestamp for logs
+ - date:
+ field: dataminr_pulse.log.fetch_timestamp
+ tag: date_log_fetch_timestamp
+ target_field: '@timestamp'
+ formats:
+ - ISO8601
+ if: ctx.dataminr_pulse?.log?.fetch_timestamp != null
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ # Convert expires_at from epoch millis to date
+ - date:
+ field: dataminr_pulse.log.expires_at
+ tag: date_log_expires_at
+ target_field: dataminr_pulse.log.expires_at
+ formats:
+ - epoch_millis
+ if: ctx.dataminr_pulse?.log?.expires_at != null
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ # Custom pipeline for logs
+ - pipeline:
+ name: logs-dataminr_pulse.alerts@custom
+ if: ctx._temp?.is_log == true
+ ignore_missing_pipeline: true
+ ignore_failure: true
+
+ # Process alert documents
+ # Extract alert-level metadata fields
+ - set:
+ field: dataminr_pulse.cursor_used
+ tag: set_cursor_used
+ copy_from: json.cursor_used
+ ignore_empty_value: true
+ if: ctx._temp?.is_log != true && ctx.json?.cursor_used != null
+
+ # Remove metadata fields from json object after extraction
+ - remove:
+ field:
+ - json.cursor_used
+ - json.fetched_alerts
+ - json.fetch_timestamp
+ tag: remove_extracted_metadata
+ ignore_missing: true
+ if: ctx._temp?.is_log != true
+
+ # Store entityType for tracking document type
+ - set:
+ field: dataminr_pulse.entity_type
+ tag: set_entity_type_field
+ copy_from: json.entityType
+ ignore_empty_value: true
+ if: ctx.json?.entityType != null
+
+ # Extract all entity types from intelAgents in a single pass
+ - script:
+ lang: painless
+ tag: extract_all_entities
+ if: ctx.json?.intelAgents != null && ctx.dataminr_pulse?.entity_type != null
+ source: |
+ def threatActorNames = [];
+ def threatActorAliases = [];
+ def threatActorCountries = [];
+ def vulnerabilityNames = [];
+ def malwareList = [];
+ def platformsSet = new LinkedHashSet();
+ def urlList = [];
+ def ipList = [];
+ if (ctx.json.intelAgents instanceof List) {
+ for (intelAgent in ctx.json.intelAgents) {
+ if (intelAgent.discoveredEntities != null && intelAgent.discoveredEntities instanceof List) {
+ for (entity in intelAgent.discoveredEntities) {
+ if (entity.type == 'threatActor' && entity.name != null && entity.name != '') {
+ if (!threatActorNames.contains(entity.name)) {
+ threatActorNames.add(entity.name);
+ }
+ if (entity.aliases != null && entity.aliases instanceof List) {
+ for (alias in entity.aliases) {
+ if (!threatActorAliases.contains(alias)) {
+ threatActorAliases.add(alias);
+ }
+ }
+ }
+ if (entity.countryOfOrigin != null) {
+ if (entity.countryOfOrigin instanceof List) {
+ for (country in entity.countryOfOrigin) {
+ if (!threatActorCountries.contains(country)) {
+ threatActorCountries.add(country);
+ }
+ }
+ } else if (entity.countryOfOrigin instanceof String) {
+ if (!threatActorCountries.contains(entity.countryOfOrigin)) {
+ threatActorCountries.add(entity.countryOfOrigin);
+ }
+ }
+ }
+ } else if (entity.type == 'vulnerability') {
+ def vulnName = null;
+ if (entity.cve != null && entity.cve != '') {
+ vulnName = entity.cve;
+ } else if (entity.name != null && entity.name != '') {
+ vulnName = entity.name;
+ } else if (entity.id != null && entity.id != '') {
+ vulnName = entity.id;
+ }
+ if (vulnName != null && !vulnerabilityNames.contains(vulnName)) {
+ vulnerabilityNames.add(vulnName);
+ }
+ } else if (entity.type == 'malware' && entity.name != null && entity.name != '') {
+ if (!malwareList.contains(entity.name)) {
+ malwareList.add(entity.name);
+ }
+ if (entity.affectedOperatingSystems != null && entity.affectedOperatingSystems instanceof List) {
+ for (os in entity.affectedOperatingSystems) {
+ def normalizedOS = os;
+ if (os == 'win') {
+ normalizedOS = 'Windows';
+ } else if (os == 'elf') {
+ normalizedOS = 'Linux';
+ }
+ platformsSet.add(normalizedOS);
+ }
+ }
+ } else if (entity.type == 'url' && entity.name != null && entity.name != '') {
+ def refangedUrl = entity.name.replace('[.]', '.');
+ if (!urlList.contains(refangedUrl)) {
+ urlList.add(refangedUrl);
+ }
+ } else if (entity.type == 'ipAddress' && entity.ip != null && entity.ip != '') {
+ def refangedIp = entity.ip.replace('[.]', '.');
+ if (!ipList.contains(refangedIp)) {
+ ipList.add(refangedIp);
+ }
+ }
+ }
+ }
+ }
+ }
+ if (ctx.dataminr_pulse == null) { ctx.dataminr_pulse = [:]; }
+ if (threatActorNames.size() > 0) {
+ if (ctx.dataminr_pulse.threatactor == null) { ctx.dataminr_pulse.threatactor = [:]; }
+ ctx.dataminr_pulse.threatactor.name = threatActorNames;
+ if (threatActorAliases.size() > 0) {
+ ctx.dataminr_pulse.threatactor.alias = threatActorAliases;
+ }
+ if (threatActorCountries.size() > 0) {
+ ctx.dataminr_pulse.threatactor.country_of_origin = threatActorCountries;
+ }
+ }
+ if (vulnerabilityNames.size() > 0) {
+ if (ctx.dataminr_pulse.vulnerability == null) { ctx.dataminr_pulse.vulnerability = [:]; }
+ ctx.dataminr_pulse.vulnerability.name = vulnerabilityNames;
+ }
+ if (malwareList.size() > 0) {
+ ctx.dataminr_pulse.malware = malwareList;
+ }
+ if (platformsSet.size() > 0) {
+ ctx.dataminr_pulse.platforms = new ArrayList(platformsSet);
+ }
+ if (urlList.size() > 0) {
+ ctx.dataminr_pulse.url = urlList;
+ }
+ if (ipList.size() > 0) {
+ ctx.dataminr_pulse.ip = ipList;
+ if (ctx.related == null) { ctx.related = [:]; }
+ if (ctx.related.ip == null) { ctx.related.ip = []; }
+ for (ip in ipList) {
+ if (!ctx.related.ip.contains(ip)) { ctx.related.ip.add(ip); }
+ }
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ # URL-IP address handling
+ - script:
+ lang: painless
+ tag: build_threat_enrichments_for_url_ip
+ if: ctx.dataminr_pulse?.entity_type == 'url-ipAddress' && ctx.json?.intelAgents != null
+ source: |
+ def enrichments = [];
+
+ if (ctx.json.intelAgents instanceof List) {
+ for (intelAgent in ctx.json.intelAgents) {
+ if (intelAgent.discoveredEntities != null && intelAgent.discoveredEntities instanceof List) {
+ for (entity in intelAgent.discoveredEntities) {
+
+ // Handle URL entities
+ if (entity.type == 'url' && entity.name != null && entity.name != '') {
+ def enrichment = [:];
+ enrichment.indicator = [:];
+ enrichment.indicator.type = 'url';
+ enrichment.indicator.name = entity.name; // defanged original
+
+ def refangedUrl = entity.name.replace('[.]', '.');
+ if (enrichment.indicator.url == null) { enrichment.indicator.url = [:]; }
+ enrichment.indicator.url.original = refangedUrl;
+
+ enrichments.add(enrichment);
+ }
+
+ // Handle IP entities
+ if (entity.type == 'ipAddress' && entity.ip != null && entity.ip != '') {
+ def enrichment = [:];
+ enrichment.indicator = [:];
+ enrichment.indicator.name = entity.ip; // defanged original
+
+ def refangedIp = entity.ip.replace('[.]', '.');
+ enrichment.indicator.ip = refangedIp;
+
+ // Detect IPv4 vs IPv6
+ if (refangedIp.contains(':')) {
+ enrichment.indicator.type = 'ipv6-addr';
+ } else {
+ enrichment.indicator.type = 'ipv4-addr';
+ }
+
+ // Add ports if available
+ if (entity.ports != null && entity.ports instanceof List && entity.ports.size() > 0) {
+ enrichment.indicator.port = entity.ports;
+ }
+
+ enrichments.add(enrichment);
+ }
+ }
+ }
+ }
+ }
+
+ if (enrichments.size() > 0) {
+ if (ctx.threat == null) { ctx.threat = [:]; }
+ ctx.threat.enrichments = enrichments;
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ - remove:
+ field: event.category
+ if: ctx.dataminr_pulse?.entity_type == 'url-ipAddress'
+ ignore_missing: true
+
+ - append:
+ field: event.category
+ value: threat
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.entity_type == 'url-ipAddress'
+
+ - remove:
+ field: event.type
+ if: ctx.dataminr_pulse?.entity_type == 'url-ipAddress'
+ ignore_missing: true
+
+ - append:
+ field: event.type
+ value: indicator
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.entity_type == 'url-ipAddress'
+
+ # Generate base fingerprint from alert ID and timestamp
+ - fingerprint:
+ fields:
+ - json.alertId
+ - json.alertTimestamp
+ tag: fingerprint_alert_base
+ target_field: _temp.base_fingerprint
+ ignore_missing: true
+ if: ctx.json?.alertId != null
+
+ # For alerts WITH discovered_entity, add entity info to make it unique
+ - fingerprint:
+ fields:
+ - _temp.base_fingerprint
+ - json.discovered_entity.type
+ - json.discovered_entity.ip
+ - json.discovered_entity.name
+ tag: fingerprint_with_entity
+ target_field: _id
+ ignore_missing: true
+ if: ctx.json?.discovered_entity != null
+
+ # For alerts WITHOUT discovered_entity, add entityType to make fingerprint unique per split
+ - fingerprint:
+ fields:
+ - _temp.base_fingerprint
+ - json.entityType
+ tag: fingerprint_without_entity
+ target_field: _id
+ ignore_missing: true
+ if: ctx.json?.discovered_entity == null && ctx._temp?.base_fingerprint != null
+
+ # Remove entityType from json after fingerprint generation
+ - remove:
+ field:
+ - json.entityType
+ tag: remove_entity_metadata_fields
+ ignore_missing: true
+
+ # Map discovered_entity to dataminr_pulse namespace (single object, not array)
+ - rename:
+ field: json.discovered_entity
+ tag: rename_discovered_entity
+ target_field: dataminr_pulse.discovered_entity
+ ignore_missing: true
+ if: ctx.json?.discovered_entity != null
+
+ # Map simplified discovered_entity type and value from connector-provided flat fields
+ - set:
+ field: dataminr_pulse.discovered_entity.type
+ tag: set_discovered_entity_type
+ copy_from: json.discoveredentity_type
+ ignore_empty_value: true
+ if: ctx.json?.discoveredentity_type != null
+
+ - set:
+ field: dataminr_pulse.discovered_entity.value
+ tag: set_discovered_entity_value
+ copy_from: json.discoveredentity_value
+ ignore_empty_value: true
+ if: ctx.json?.discoveredentity_value != null
+
+ - remove:
+ field:
+ - json.discoveredentity_type
+ - json.discoveredentity_value
+ tag: remove_discovered_entity_flat_fields
+ ignore_missing: true
+
+ # Rename camelCase sub-fields from API to snake_case
+ - rename:
+ field: dataminr_pulse.discovered_entity.geoIpMapping
+ target_field: dataminr_pulse.discovered_entity.geo_ip_mapping
+ tag: rename_geo_ip_mapping
+ ignore_missing: true
+ - rename:
+ field: dataminr_pulse.discovered_entity.CAL
+ target_field: dataminr_pulse.discovered_entity.cyber_asset_lifecycle
+ tag: rename_cal
+ ignore_missing: true
+ - rename:
+ field: dataminr_pulse.discovered_entity.cyber_asset_lifecycle.impactFactors
+ target_field: dataminr_pulse.discovered_entity.cyber_asset_lifecycle.impact_factors
+ tag: rename_impact_factors
+ ignore_missing: true
+ - rename:
+ field: dataminr_pulse.discovered_entity.knownExploitedDate
+ target_field: dataminr_pulse.discovered_entity.known_exploited_date
+ tag: rename_known_exploited_date
+ ignore_missing: true
+ - rename:
+ field: dataminr_pulse.discovered_entity.affectedOperatingSystems
+ target_field: dataminr_pulse.discovered_entity.affected_operating_systems
+ tag: rename_affected_operating_systems
+ ignore_missing: true
+ - rename:
+ field: dataminr_pulse.discovered_entity.countryOfOrigin
+ target_field: dataminr_pulse.discovered_entity.country_of_origin
+ tag: rename_country_of_origin
+ ignore_missing: true
+
+ # Vulnerability ECS Mapping
+
+ - set:
+ field: vulnerability.id
+ copy_from: dataminr_pulse.discovered_entity.name
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+
+ - convert:
+ field: dataminr_pulse.discovered_entity.cvss
+ target_field: vulnerability.score.base
+ type: float
+ ignore_missing: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Failed to convert cvss to float: {{{_ingest.on_failure_message}}}'
+
+ - set:
+ field: vulnerability.description
+ copy_from: dataminr_pulse.discovered_entity.summary
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+
+ - remove:
+ field: event.category
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+ ignore_missing: true
+
+ - append:
+ field: event.category
+ value: vulnerability
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+
+ - remove:
+ field: event.type
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+ ignore_missing: true
+
+ - append:
+ field: event.type
+ value: info
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'vulnerability'
+
+ # Threat Actor ECS Mapping
+
+ - set:
+ field: threat.group.name
+ copy_from: dataminr_pulse.discovered_entity.name
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+
+ - foreach:
+ field: dataminr_pulse.discovered_entity.aliases
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor' && ctx.dataminr_pulse?.discovered_entity?.aliases instanceof List
+ processor:
+ append:
+ field: threat.group.alias
+ value: '{{{_ingest._value}}}'
+ allow_duplicates: false
+
+ - set:
+ field: threat.indicator.geo.country_iso_code
+ copy_from: dataminr_pulse.threatactor.country_of_origin
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor' && ctx.dataminr_pulse?.threatactor?.country_of_origin != null
+
+ - set:
+ field: threat.framework
+ value: "MITRE ATT&CK"
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+
+ - remove:
+ field: event.category
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+ ignore_missing: true
+
+ - append:
+ field: event.category
+ value: threat
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+
+ - remove:
+ field: event.type
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+ ignore_missing: true
+
+ - append:
+ field: event.type
+ value: indicator
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'threatActor'
+
+ # Malware ECS mapping
+ - set:
+ field: threat.software.name
+ copy_from: dataminr_pulse.discovered_entity.name
+ ignore_empty_value: true
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+
+ - set:
+ field: threat.software.type
+ value: "Malware"
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+
+ # Normalize and map platforms for malware
+ - script:
+ lang: painless
+ tag: normalize_malware_platforms
+ if: ctx.dataminr_pulse?.discovered_entity?.affected_operating_systems != null && ctx.dataminr_pulse?.entity_type == 'malware'
+ source: |
+ def platforms = ctx.dataminr_pulse.discovered_entity.affected_operating_systems;
+ if (platforms instanceof List && platforms.size() > 0) {
+ def normalizedPlatforms = [];
+ for (os in platforms) {
+ if (os == 'win') {
+ normalizedPlatforms.add('Windows');
+ } else if (os == 'elf') {
+ normalizedPlatforms.add('Linux');
+ } else {
+ normalizedPlatforms.add(os);
+ }
+ }
+ if (normalizedPlatforms.size() > 0) {
+ if (ctx.threat == null) { ctx.threat = [:]; }
+ if (ctx.threat.software == null) { ctx.threat.software = [:]; }
+ ctx.threat.software.platforms = normalizedPlatforms;
+ }
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ - remove:
+ field: event.category
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+ ignore_missing: true
+
+ - append:
+ field: event.category
+ value: malware
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+
+ - remove:
+ field: event.type
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+ ignore_missing: true
+
+ - append:
+ field: event.type
+ value: info
+ allow_duplicates: false
+ if: ctx.dataminr_pulse?.discovered_entity?.type == 'malware'
+
+
+ # Map base alert fields
+ - set:
+ field: event.id
+ tag: set_event_id
+ copy_from: json.alertId
+ ignore_empty_value: true
+ if: ctx._temp?.is_log != true
+
+ # Map alertTimestamp to @timestamp
+ - date:
+ field: json.alertTimestamp
+ tag: date_alertTimestamp
+ target_field: '@timestamp'
+ formats:
+ - ISO8601
+ - epoch_millis
+ if: ctx._temp?.is_log != true && ctx.json?.alertTimestamp != null && ctx.json.alertTimestamp != ''
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ # Map dataminrAlertUrl to event.url
+ - set:
+ field: event.url
+ tag: set_event_url
+ copy_from: json.dataminrAlertUrl
+ ignore_empty_value: true
+ if: ctx._temp?.is_log != true
+
+ # Map alertType.name to event.severity and store type
+ - script:
+ lang: painless
+ tag: script_map_alert_severity
+ if: ctx.json?.alertType?.name != null
+ source: |
+ def alertType = ctx.json.alertType.name;
+ if (alertType == 'Flash') {
+ ctx.event.severity = 90;
+ } else if (alertType == 'Urgent') {
+ ctx.event.severity = 50;
+ } else {
+ ctx.event.severity = 30;
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ # Set event.kind to alert for actual Dataminr alert documents
+ - set:
+ field: event.kind
+ tag: set_event_kind_alert
+ value: alert
+ override: true
+ if: ctx.json?.alertType?.name != null
+
+ # Rename alertType name to custom field
+ - rename:
+ field: json.alertType.name
+ tag: rename_alertType
+ target_field: dataminr_pulse.alert_type.name
+ ignore_missing: true
+
+ # Map headline to message field
+ - set:
+ field: message
+ tag: set_message_from_headline
+ copy_from: json.headline
+ ignore_empty_value: true
+ if: ctx._temp?.is_log != true
+
+ - date:
+ field: json.publicPost.timestamp
+ tag: date_publicPost_timestamp
+ target_field: event.created
+ formats:
+ - ISO8601
+ - epoch_millis
+ if: ctx.json?.publicPost?.timestamp != null && ctx.json.publicPost.timestamp != ''
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ - rename:
+ field: json.publicPost.href
+ tag: rename_publicPost_href
+ target_field: dataminr_pulse.source.href
+ if: ctx.json?.publicPost?.href != null && ctx.json.publicPost.href != ''
+ ignore_missing: true
+
+ - rename:
+ field: json.publicPost.channels
+ tag: rename_publicPost_channels
+ target_field: dataminr_pulse.source.channels
+ if: ctx.json?.publicPost?.channels != null && ctx.json.publicPost.channels != ''
+ ignore_missing: true
+
+ - foreach:
+ field: json.publicPost.media
+ if: ctx.json?.publicPost?.media != null
+ tag: foreach_media_add_href
+ processor:
+ append:
+ field: dataminr_pulse.source.media.href
+ tag: add_media_href
+ value: '{{{_ingest._value.href}}}'
+ allow_duplicates: false
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ - foreach:
+ field: json.alertTopics
+ if: ctx.json?.alertTopics != null
+ tag: foreach_topic_add_name
+ processor:
+ append:
+ field: dataminr_pulse.categories.name
+ tag: add_topic_name
+ value: '{{{_ingest._value.name}}}'
+ allow_duplicates: false
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ - foreach:
+ field: json.alertCompanies
+ if: ctx.json?.alertCompanies != null
+ tag: foreach_company_add_name
+ processor:
+ append:
+ field: dataminr_pulse.companies.name
+ tag: add_company_name
+ value: '{{{_ingest._value.name}}}'
+ allow_duplicates: false
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ - foreach:
+ field: json.alertSectors
+ if: ctx.json?.alertSectors != null
+ tag: foreach_sector_add_name
+ processor:
+ append:
+ field: dataminr_pulse.sectors.name
+ tag: add_sector_name
+ value: '{{{_ingest._value.name}}}'
+ allow_duplicates: false
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ # Store live brief
+ - rename:
+ field: json.liveBrief.0.summary
+ tag: rename_live_brief_summary
+ target_field: dataminr_pulse.live_brief.summary
+ ignore_missing: true
+
+ - rename:
+ field: json.liveBrief.0.version
+ tag: rename_live_brief_version
+ target_field: dataminr_pulse.live_brief.version
+ ignore_missing: true
+
+ - rename:
+ field: json.liveBrief.0.timestamp
+ tag: rename_live_brief_timestamp
+ target_field: dataminr_pulse.live_brief.timestamp
+ ignore_missing: true
+
+ # Map EstimatedEventLocation.coordinates to geo.location
+ - script:
+ lang: painless
+ tag: script_map_geo_coordinates
+ if: ctx.json?.estimatedEventLocation?.coordinates != null
+ source: |
+ def coords = ctx.json.estimatedEventLocation.coordinates;
+ if (coords instanceof List && coords.size() == 2) {
+ if (ctx.geo == null) { ctx.geo = [:]; }
+ if (ctx.geo.location == null) { ctx.geo.location = [:]; }
+ ctx.geo.location.lat = coords[0];
+ ctx.geo.location.lon = coords[1];
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ # Store location name
+ - set:
+ field: geo.name
+ tag: set_location_name
+ copy_from: json.estimatedEventLocation.name
+ ignore_empty_value: true
+
+ - script:
+ lang: painless
+ tag: convert_intelAgents_summary
+ if: ctx.json?.intelAgents instanceof List && ctx.json.intelAgents.size() > 0 && ctx.json.intelAgents[0].summary != null
+ source: |
+ def summary = ctx.json.intelAgents[0].summary;
+ if (summary instanceof String) {
+ if (ctx.dataminr_pulse == null) { ctx.dataminr_pulse = [:]; }
+ if (ctx.dataminr_pulse.intel_agents == null) { ctx.dataminr_pulse.intel_agents = [:]; }
+ ctx.dataminr_pulse.intel_agents.summary = summary;
+ } else if (summary instanceof List) {
+ def parts = [];
+ for (item in summary) {
+ if (item instanceof Map) {
+ def title = item.containsKey('title') ? item.title : '';
+ def content = '';
+ if (item.containsKey('content') && item.content instanceof List) {
+ content = String.join(' ', item.content);
+ }
+ if (title != '' && content != '') {
+ parts.add(title + ': ' + content);
+ } else if (content != '') {
+ parts.add(content);
+ } else if (title != '') {
+ parts.add(title);
+ }
+ } else if (item instanceof String) {
+ parts.add(item);
+ }
+ }
+ if (parts.size() > 0) {
+ if (ctx.dataminr_pulse == null) { ctx.dataminr_pulse = [:]; }
+ if (ctx.dataminr_pulse.intel_agents == null) { ctx.dataminr_pulse.intel_agents = [:]; }
+ ctx.dataminr_pulse.intel_agents.summary = String.join(' | ', parts);
+ }
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
+
+ - foreach:
+ field: json.listsMatched
+ if: ctx.json?.listsMatched != null
+ tag: foreach_listmatched_add_name
+ processor:
+ append:
+ field: dataminr_pulse.watchlists_matched_by_type.name
+ tag: add_watchlist_name
+ value: '{{{_ingest._value.name}}}'
+ allow_duplicates: false
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+
+ # Map metadata.cyber.hashValues to file.hash fields
+ - script:
+ lang: painless
+ tag: script_map_hash_values
+ if: ctx.json?.metadata?.cyber?.hashValues != null
+ source: |
+ def hashes = ctx.json.metadata.cyber.hashValues;
+ if (hashes instanceof List && hashes.size() > 0) {
+ if (ctx.file == null) { ctx.file = [:]; }
+ if (ctx.file.hash == null) { ctx.file.hash = [:]; }
+ if (ctx.related == null) { ctx.related = [:]; }
+ if (ctx.related.hash == null) { ctx.related.hash = []; }
+
+ for (hash in hashes) {
+ if (hash.type != null && hash.value != null) {
+ def hashType = hash.type.toLowerCase();
+ if (hashType == 'md5') {
+ ctx.file.hash.md5 = hash.value;
+ if (!ctx.related.hash.contains(hash.value)) {
+ ctx.related.hash.add(hash.value);
+ }
+ } else if (hashType == 'sha1') {
+ ctx.file.hash.sha1 = hash.value;
+ if (!ctx.related.hash.contains(hash.value)) {
+ ctx.related.hash.add(hash.value);
+ }
+ } else if (hashType == 'sha256') {
+ ctx.file.hash.sha256 = hash.value;
+ if (!ctx.related.hash.contains(hash.value)) {
+ ctx.related.hash.add(hash.value);
+ }
+ } else if (hashType == 'sha512') {
+ ctx.file.hash.sha512 = hash.value;
+ if (!ctx.related.hash.contains(hash.value)) {
+ ctx.related.hash.add(hash.value);
+ }
+ }
+ }
+ }
+ }
+ on_failure:
+ - append:
+ field: error.message
+ value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
+ # Preserve intelAgents under vendor namespace
+ - rename:
+ field: json.intelAgents
+ target_field: dataminr_pulse.intel_agents_raw
+ ignore_missing: true
+ tag: rename_intel_agents_raw
+ - remove:
+ field: event.original
+ tag: remove_event_original
+ if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event'))
+ ignore_failure: true
+ ignore_missing: true
+
+ # Clean up temporary processing fields
+ - remove:
+ field:
+ - json
+ - _temp
+ tag: remove_temp_fields
+ ignore_missing: true
+
+ # Drop null/empty values
+ - script:
+ tag: script_drop_null_values
+ lang: painless
+ description: Drops null/empty values recursively
+ source: |
+ boolean drop(Object o) {
+ if (o == null || o == '') {
+ return true;
+ } else if (o instanceof Map) {
+ ((Map) o).values().removeIf(v -> drop(v));
+ return (((Map) o).size() == 0);
+ } else if (o instanceof List) {
+ ((List) o).removeIf(v -> drop(v));
+ return (((List) o).size() == 0);
+ }
+ return false;
+ }
+ drop(ctx);
+
+ # Remove unwanted fields from discovered_entity
+ - remove:
+ field:
+ - dataminr_pulse.discovered_entity.cvss
+ - dataminr_pulse.discovered_entity.epssScore
+ - dataminr_pulse.discovered_entity.exploitable
+ - dataminr_pulse.discovered_entity.exploitPocLinks
+ - dataminr_pulse.discovered_entity.known_exploited_date
+ - dataminr_pulse.discovered_entity.name
+ - dataminr_pulse.discovered_entity.products
+ - dataminr_pulse.discovered_entity.publishedDate
+ - dataminr_pulse.discovered_entity.type
+ - dataminr_pulse.entity_type
+ - dataminr_pulse.intel_agents_raw
+ tag: remove_unwanted_fields
+ ignore_missing: true
+
+on_failure:
+ - set:
+ field: event.kind
+ tag: set_pipeline_error_to_event_kind
+ value: pipeline_error
+ - append:
+ field: error.message
+ value: >-
+ Processor {{{_ingest.on_failure_processor_type}}}
+ with tag '{{{_ingest.on_failure_processor_tag}}}'
+ in pipeline {{{_ingest.pipeline}}}
+ failed with message: {{{_ingest.on_failure_message}}}
\ No newline at end of file
diff --git a/packages/dataminr_pulse/data_stream/alerts/fields/ecs.yml b/packages/dataminr_pulse/data_stream/alerts/fields/ecs.yml
new file mode 100644
index 00000000000..1577c50303d
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/fields/ecs.yml
@@ -0,0 +1,77 @@
+- external: ecs
+ name: data_stream.type
+- external: ecs
+ name: data_stream.dataset
+- external: ecs
+ name: data_stream.namespace
+- external: ecs
+ name: '@timestamp'
+- external: ecs
+ name: message
+- external: ecs
+ name: event.id
+- external: ecs
+ name: event.kind
+- external: ecs
+ name: event.category
+- external: ecs
+ name: event.type
+- external: ecs
+ name: event.severity
+- external: ecs
+ name: event.url
+- external: ecs
+ name: event.created
+- external: ecs
+ name: event.original
+- external: ecs
+ name: error.message
+- external: ecs
+ name: file.hash.md5
+- external: ecs
+ name: file.hash.sha256
+- external: ecs
+ name: file.hash.sha1
+- external: ecs
+ name: file.hash.sha512
+- external: ecs
+ name: related.hash
+- external: ecs
+ name: related.ip
+- external: ecs
+ name: vulnerability.id
+- external: ecs
+ name: vulnerability.description
+- external: ecs
+ name: vulnerability.score.base
+- external: ecs
+ name: vulnerability.scanner.vendor
+- external: ecs
+ name: threat.framework
+- external: ecs
+ name: threat.group.name
+- external: ecs
+ name: threat.group.alias
+- external: ecs
+ name: threat.indicator.geo.country_iso_code
+- external: ecs
+ name: threat.indicator.name
+- external: ecs
+ name: threat.indicator.type
+- external: ecs
+ name: threat.indicator.ip
+- external: ecs
+ name: threat.indicator.port
+- external: ecs
+ name: threat.indicator.url.original
+- external: ecs
+ name: threat.software.name
+- external: ecs
+ name: threat.software.type
+- external: ecs
+ name: threat.software.platforms
+- external: ecs
+ name: threat.enrichments
+- name: input.type
+ type: keyword
+ description: Type of input that generated the event.
\ No newline at end of file
diff --git a/packages/dataminr_pulse/data_stream/alerts/fields/fields.yml b/packages/dataminr_pulse/data_stream/alerts/fields/fields.yml
new file mode 100644
index 00000000000..991ec51468c
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/fields/fields.yml
@@ -0,0 +1,262 @@
+---
+# data_stream/alerts/fields/fields.yml
+
+# Geo location fields (not ECS-reusable at top level)
+- name: geo
+ type: group
+ description: Estimated event location
+ fields:
+ - name: location
+ type: geo_point
+ description: Geo-point location (lat/lon)
+ - name: name
+ type: keyword
+ description: Location name
+
+- name: dataminr_pulse
+ type: group
+ fields:
+ - name: cursor_used
+ type: keyword
+ description: Pagination cursor used to fetch this alert
+ - name: alert_type
+ type: group
+ fields:
+ - name: name
+ type: keyword
+ description: Alert priority level (Alert, Urgent, Flash)
+ - name: categories
+ type: group
+ fields:
+ - name: name
+ type: keyword
+ description: Topic name
+ - name: companies
+ type: group
+ fields:
+ - name: name
+ type: keyword
+ description: Company name
+ - name: sectors
+ type: group
+ fields:
+ - name: name
+ type: keyword
+ description: Sector name
+ - name: source
+ type: group
+ fields:
+ - name: href
+ type: keyword
+ description: URL to public post
+ - name: channels
+ type: keyword
+ description: Source channels
+ - name: media
+ type: group
+ description: Media attachments
+ fields:
+ - name: href
+ type: keyword
+ description: Media attachment URL
+ - name: intel_agents
+ type: group
+ description: AI-generated critical context
+ fields:
+ - name: summary
+ type: keyword
+ description: Intel Agent summary
+ - name: is_main
+ type: keyword
+ description: Indicates if this is the master document for this alert_id (yes/no)
+ - name: live_brief
+ type: group
+ fields:
+ - name: summary
+ type: keyword
+ description: Live Brief summary
+ - name: version
+ type: keyword
+ description: Live Brief version
+ - name: timestamp
+ type: date
+ description: Live Brief timestamp
+ - name: watchlists_matched_by_type
+ type: group
+ fields:
+ - name: name
+ type: keyword
+ description: Watchlist name
+ - name: log
+ type: group
+ description: Operational log fields for auth and alert-fetch events
+ fields:
+ - name: log_type
+ type: keyword
+ description: Type of log entry (auth or alert-fetch)
+ - name: status
+ type: keyword
+ description: Status of operation (success or failed)
+ - name: http_status_code
+ type: long
+ description: HTTP response status code
+ - name: error_message
+ type: text
+ description: Error message for failed operations
+ - name: client_id
+ type: keyword
+ description: Client ID used for authentication
+ - name: retry_number
+ type: long
+ description: Retry attempt number (0 for no retries)
+ - name: expires_at
+ type: date
+ description: Token expiration timestamp in milliseconds
+ - name: api_endpoint
+ type: keyword
+ description: Full API endpoint URL used for alert fetching
+ - name: input_alert_fetch_count
+ type: long
+ description: Number of alerts requested (page size)
+ - name: next_cursor
+ type: keyword
+ description: Next cursor received from API for pagination
+ - name: count_of_alerts
+ type: long
+ description: Number of alerts returned in the response
+ - name: fetched_alerts
+ type: long
+ description: Number of alerts fetched in the batch
+ - name: fetch_timestamp
+ type: date
+ description: Timestamp when the API call was made
+ - name: response_body
+ type: text
+ description: Raw response body for failed requests
+ - name: error_code
+ type: long
+ description: Error code from API error response
+
+ - name: discovered_entity
+ type: group
+ description: Discovered entity from the alert (when alerts are expanded by entities)
+ fields:
+ - name: type
+ type: keyword
+ description: Entity type (e.g., ipAddress, vulnerability, threatActor)
+ - name: value
+ type: keyword
+ description: Entity identifier value
+
+ # IP Address entity fields
+ - name: ip
+ type: ip
+ description: IP address (for ipAddress type entities)
+ - name: ports
+ type: long
+ description: Associated ports
+ - name: geo_ip_mapping
+ type: group
+ description: Geographic IP mapping information
+ fields:
+ - name: city
+ type: keyword
+ description: City location
+ - name: country
+ type: keyword
+ description: Country code
+ - name: region
+ type: keyword
+ description: Region code
+ - name: cyber_asset_lifecycle
+ type: group
+ description: Cyber Asset Lifecycle information
+ fields:
+ - name: status
+ type: keyword
+ description: Status (e.g., Unassigned)
+ - name: impact_factors
+ type: keyword
+ description: Impact factors
+ - name: classifiers
+ type: keyword
+ description: Classifiers
+
+ # Threat actor entity fields
+ - name: aliases
+ type: keyword
+ description: Threat actor aliases
+
+ # Vulnerability entity fields
+ - name: cve
+ type: keyword
+ description: CVE identifier for the vulnerability
+ - name: summary
+ type: text
+ description: Detailed summary of the vulnerability
+ - name: known_exploited_date
+ type: date
+ description: Date when vulnerability was known to be exploited
+
+ # Malware entity fields
+ - name: affected_operating_systems
+ type: keyword
+ description: Affected operating systems for malware entities
+
+ # Threat actor additional fields
+ - name: country_of_origin
+ type: keyword
+ description: Countries of origin for threat actor entities
+
+ - name: entity_type
+ type: keyword
+ description: Entity type from the alert (e.g., vulnerability, threatActor, url-ipAddress)
+
+ - name: ip
+ type: ip
+ description: Extracted IP addresses from discovered entities
+
+ - name: url
+ type: keyword
+ description: Extracted URLs from discovered entities (refanged)
+
+ - name: malware
+ type: keyword
+ description: Extracted malware names from discovered entities
+
+ - name: platforms
+ type: keyword
+ description: Affected operating systems/platforms from malware entities
+
+ - name: threatactor
+ type: group
+ description: All threat actors discovered in the alert (custom field for correlation)
+ fields:
+ - name: name
+ type: keyword
+ description: Threat actor name
+ - name: alias
+ type: keyword
+ description: Threat actor aliases
+ - name: country_of_origin
+ type: keyword
+ description: Threat actor countries of origin
+
+ - name: vulnerability
+ type: group
+ description: All vulnerabilities discovered in the alert (custom field for correlation)
+ fields:
+ - name: name
+ type: keyword
+ description: Vulnerability identifiers (CVE IDs)
+
+# Intel agents raw data (preserved from API response)
+- name: intelAgents
+ type: flattened
+ description: Raw Intel Agents data from the Dataminr Pulse API response
+
+# Cloud provider metadata (added by Elastic Agent when running on AWS)
+- name: aws.tags
+ type: flattened
+ description: AWS resource tags from the host environment
+
diff --git a/packages/dataminr_pulse/data_stream/alerts/manifest.yml b/packages/dataminr_pulse/data_stream/alerts/manifest.yml
new file mode 100644
index 00000000000..a53f45504fd
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/manifest.yml
@@ -0,0 +1,121 @@
+title: Dataminr Pulse Alerts
+type: logs
+streams:
+ - input: cel
+ title: Dataminr Pulse Alerts
+ description: Collect alerts from Dataminr Pulse API with automatic OAuth token refresh
+ template_path: stream.yml.hbs
+ vars:
+ - name: url
+ type: text
+ title: API URL
+ description: The base URL for the Dataminr Pulse API
+ default: https://api.dataminr.com/pulse/v1/alerts
+ required: true
+ show_user: true
+ - name: base_url
+ type: text
+ title: Base URL
+ description: The base URL without the endpoint path (e.g., https://api.dataminr.com/pulse)
+ default: https://api.dataminr.com/pulse
+ required: true
+ show_user: false
+ - name: auth_url
+ type: text
+ title: Authentication URL
+ description: The OAuth token endpoint URL
+ default: https://userauth.dataminr.com/auth/2/token
+ required: true
+ secret: false
+ show_user: false
+ - name: token_expiry_buffer_ms
+ type: integer
+ title: Token Expiry Buffer (milliseconds)
+ description: Buffer time in milliseconds before token expiry to trigger refresh (default 300000 = 5 minutes)
+ default: 300000
+ required: true
+ show_user: false
+ secret: false
+ - name: client_id
+ type: text
+ title: Client ID
+ description: Client ID from Dataminr for token generation
+ required: true
+ show_user: true
+ secret: false
+ - name: client_secret
+ type: password
+ title: Client Secret
+ description: Client secret from Dataminr for token generation
+ required: true
+ show_user: true
+ secret: true
+ - name: interval
+ type: select
+ title: Interval
+ description: Interval between API requests
+ default: 5m
+ required: true
+ multi: false
+ show_user: true
+ options:
+ - value: 30s
+ text: "30 seconds"
+ - value: 1m
+ text: "1 minute"
+ - value: 3m
+ text: "3 minute"
+ - value: 5m
+ text: "5 minutes"
+ - value: 10m
+ text: "10 minutes"
+ - value: 15m
+ text: "15 minute"
+ - name: page_size
+ type: select
+ title: Page Size
+ description: Maximum number of alerts to return per request (default 40, max 100)
+ default: "40"
+ required: true
+ multi: false
+ show_user: true
+ options:
+ - value: "10"
+ text: "10"
+ - value: "20"
+ text: "20"
+ - value: "30"
+ text: "30"
+ - value: "40"
+ text: "40"
+ - value: "50"
+ text: "50"
+ - value: "60"
+ text: "60"
+ - value: "70"
+ text: "70"
+ - value: "80"
+ text: "80"
+ - value: "90"
+ text: "90"
+ - value: "100"
+ text: "100"
+ - name: enable_request_tracer
+ type: bool
+ title: Enable request tracing
+ default: false
+ required: false
+ show_user: false
+ - name: preserve_original_event
+ type: bool
+ title: Preserve original event
+ description: Preserves a raw copy of the original event, added to the field `event.original`
+ default: false
+ required: false
+ show_user: true
+ - name: processors
+ type: yaml
+ title: Processors
+ multi: false
+ required: false
+ show_user: false
\ No newline at end of file
diff --git a/packages/dataminr_pulse/data_stream/alerts/sample_event.json b/packages/dataminr_pulse/data_stream/alerts/sample_event.json
new file mode 100644
index 00000000000..3b64bc8eb0f
--- /dev/null
+++ b/packages/dataminr_pulse/data_stream/alerts/sample_event.json
@@ -0,0 +1,63 @@
+{
+ "@timestamp": "2026-02-12T11:04:58.116Z",
+ "data_stream": {
+ "type": "logs",
+ "dataset": "dataminr_pulse.alerts",
+ "namespace": "default"
+ },
+ "ecs": {
+ "version": "8.11.0"
+ },
+ "event": {
+ "id": "5026527284588218001-1770894234193-1",
+ "kind": "event",
+ "category": [
+ "web"
+ ],
+ "type": [
+ "info"
+ ],
+ "severity": 30,
+ "url": "https://app.dataminr.com/#alertDetail/5/5026527284588218000-1770894234190-2",
+ "created": "2026-02-12T11:03:54.193Z"
+ },
+ "message": "Test alert message",
+ "dataminr_pulse": {
+ "alert_type": {
+ "name": "Alert"
+ },
+ "categories": {
+ "name": [
+ "Outages & Service Disruptions - Electricity"
+ ]
+ },
+ "companies": {
+ "name": [
+ "Duke Energy Corporation"
+ ]
+ },
+ "sectors": {
+ "name": [
+ "Utilities"
+ ]
+ },
+ "source": {
+ "href": "https://r.dataminr.com/28IJuehlvBfbxqRBTBPK0r",
+ "channels": [
+ "sensor"
+ ]
+ },
+ "watchlists_matched_by_type": {
+ "name": [
+ "Cyber"
+ ]
+ }
+ },
+ "geo": {
+ "location": {
+ "lat": 35.2632650,
+ "lon": -80.8543840
+ },
+ "name": "Mecklenburg County, NC, USA"
+ }
+}
diff --git a/packages/dataminr_pulse/docker-compose.yml b/packages/dataminr_pulse/docker-compose.yml
new file mode 100644
index 00000000000..4ec20f846aa
--- /dev/null
+++ b/packages/dataminr_pulse/docker-compose.yml
@@ -0,0 +1,190 @@
+services:
+ setup:
+ image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
+ container_name: setup
+ depends_on:
+ elasticsearch:
+ condition: service_healthy
+ volumes:
+ - ./setup:/setup
+ user: "0"
+ command: >
+ bash -c '
+ echo "Setting kibana_system password...";
+ until curl -s -u "elastic:${ELASTIC_PASSWORD}" http://elasticsearch:9200 | grep -q "cluster_name"; do sleep 2; done;
+ curl -s -X POST -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" http://elasticsearch:9200/_security/user/kibana_system/_password -d "{\"password\":\"${KIBANA_PASSWORD}\"}" | grep -q "^{}$$";
+ echo "Creating Fleet Server service token...";
+ response=$$(curl -s -X POST -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" http://elasticsearch:9200/_security/service/elastic/fleet-server/credential/token/fleet-token-1 2>&1);
+ token=$$(echo $$response | sed "s/.*\"value\":\"\([^\"]*\)\".*/\1/");
+ if [ -z "$$token" ] || [ "$$token" = "$$response" ]; then
+ echo "Token may already exist, trying to get it with a new name...";
+ response=$$(curl -s -X POST -u "elastic:${ELASTIC_PASSWORD}" -H "Content-Type: application/json" http://elasticsearch:9200/_security/service/elastic/fleet-server/credential/token/fleet-token-$$(date +%s));
+ token=$$(echo $$response | sed "s/.*\"value\":\"\([^\"]*\)\".*/\1/");
+ fi;
+ printf "%s" "$$token" > /setup/fleet_service_token;
+ echo "Setup complete. Service token saved.";
+ '
+ networks:
+ - elastic
+ healthcheck:
+ test: ["CMD-SHELL", "test -f /setup/fleet_service_token"]
+ interval: 5s
+ timeout: 5s
+ retries: 30
+
+ elasticsearch:
+ image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
+ container_name: elasticsearch
+ hostname: elasticsearch
+ environment:
+ - node.name=elasticsearch
+ - cluster.name=${CLUSTER_NAME}
+ - discovery.type=single-node
+ - ELASTIC_PASSWORD=${ELASTIC_PASSWORD}
+ - bootstrap.memory_lock=true
+ - xpack.security.enabled=true
+ - xpack.security.http.ssl.enabled=false
+ - xpack.security.transport.ssl.enabled=false
+ - xpack.license.self_generated.type=${LICENSE}
+ - "ES_JAVA_OPTS=-Xms1g -Xmx1g"
+ ulimits:
+ memlock:
+ soft: -1
+ hard: -1
+ mem_limit: ${ES_MEM_LIMIT}
+ volumes:
+ - ./esdata:/usr/share/elasticsearch/data
+ ports:
+ - ${ES_PORT}:9200
+ healthcheck:
+ test:
+ [
+ "CMD-SHELL",
+ "curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:9200/_cluster/health | grep -qE '\"status\":\"(green|yellow)\"'",
+ ]
+ interval: 10s
+ timeout: 10s
+ retries: 30
+ networks:
+ - elastic
+
+ kibana:
+ image: docker.elastic.co/kibana/kibana:${STACK_VERSION}
+ container_name: kibana
+ hostname: kibana
+ depends_on:
+ setup:
+ condition: service_completed_successfully
+ elasticsearch:
+ condition: service_healthy
+ environment:
+ - SERVERNAME=kibana
+ - ELASTICSEARCH_HOSTS=http://elasticsearch:9200
+ - ELASTICSEARCH_USERNAME=kibana_system
+ - ELASTICSEARCH_PASSWORD=${KIBANA_PASSWORD}
+ - XPACK_SECURITY_ENCRYPTIONKEY=${KIBANA_ENCRYPTION_KEY}
+ - XPACK_ENCRYPTEDSAVEDOBJECTS_ENCRYPTIONKEY=${KIBANA_SAVED_OBJECTS_KEY}
+ - XPACK_REPORTING_ENCRYPTIONKEY=${KIBANA_REPORTING_KEY}
+ volumes:
+ - ./config/kibana.yml:/usr/share/kibana/config/kibana.yml:ro
+ mem_limit: ${KIBANA_MEM_LIMIT}
+ ports:
+ - ${KIBANA_PORT}:5601
+ healthcheck:
+ test:
+ [
+ "CMD-SHELL",
+ "curl -s -u elastic:${ELASTIC_PASSWORD} http://localhost:5601/api/status | grep -q '\"overall\"'",
+ ]
+ interval: 10s
+ timeout: 10s
+ retries: 60
+ networks:
+ - elastic
+
+ fleet-server:
+ image: docker.elastic.co/elastic-agent/elastic-agent:${STACK_VERSION}
+ container_name: fleet-server
+ hostname: fleet-server
+ depends_on:
+ kibana:
+ condition: service_healthy
+ volumes:
+ - ./setup:/setup:ro
+ environment:
+ - FLEET_SERVER_ENABLE=true
+ - FLEET_SERVER_POLICY_ID=fleet-server-policy
+ - FLEET_SERVER_ELASTICSEARCH_HOST=http://elasticsearch:9200
+ - FLEET_SERVER_SERVICE_TOKEN_PATH=/setup/fleet_service_token
+ - FLEET_URL=http://fleet-server:8220
+ - FLEET_SERVER_HOST=0.0.0.0
+ - FLEET_SERVER_PORT=8220
+ - FLEET_SERVER_INSECURE_HTTP=true
+ - KIBANA_FLEET_SETUP=true
+ - KIBANA_HOST=http://kibana:5601
+ - KIBANA_USERNAME=elastic
+ - KIBANA_PASSWORD=${ELASTIC_PASSWORD}
+ ports:
+ - "8220:8220"
+ healthcheck:
+ test:
+ [
+ "CMD-SHELL",
+ "curl -s http://localhost:8220/api/status | grep -q 'HEALTHY'",
+ ]
+ interval: 15s
+ timeout: 10s
+ retries: 30
+ mem_limit: ${AGENT_MEM_LIMIT}
+ networks:
+ - elastic
+
+ elastic-agent:
+ image: docker.elastic.co/elastic-agent/elastic-agent:${STACK_VERSION}
+ container_name: dataminr-agent
+ hostname: dataminr-agent
+ depends_on:
+ fleet-server:
+ condition: service_healthy
+ environment:
+ - FLEET_ENROLL=true
+ - FLEET_URL=http://fleet-server:8220
+ - FLEET_INSECURE=true
+ - KIBANA_HOST=http://kibana:5601
+ - ELASTICSEARCH_HOST=http://elasticsearch:9200
+ - ELASTICSEARCH_USERNAME=elastic
+ - ELASTICSEARCH_PASSWORD=${ELASTIC_PASSWORD}
+ - KIBANA_USERNAME=elastic
+ - KIBANA_PASSWORD=${ELASTIC_PASSWORD}
+ volumes:
+ - ./data/elastic-agent-state:/usr/share/elastic-agent/state
+ mem_limit: ${AGENT_MEM_LIMIT}
+ networks:
+ - elastic
+
+# mock-dataminr-api:
+# image: python:3.12-slim
+# container_name: mock-dataminr-api
+# hostname: mock-dataminr-api
+# profiles:
+# - mock
+# volumes:
+# - ./data_stream/alerts/_dev/deploy/docker/files:/app:ro
+# command: ["python", "/app/mock_server.py"]
+# ports:
+# - "8080:8080"
+# healthcheck:
+# test:
+# [
+# "CMD-SHELL",
+# "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8080/')\" 2>/dev/null || true",
+# ]
+# interval: 10s
+# timeout: 5s
+# retries: 5
+# networks:
+# - elastic
+
+networks:
+ elastic:
+ driver: bridge
diff --git a/packages/dataminr_pulse/docs/README.md b/packages/dataminr_pulse/docs/README.md
new file mode 100644
index 00000000000..2cbb909540f
--- /dev/null
+++ b/packages/dataminr_pulse/docs/README.md
@@ -0,0 +1,284 @@
+# Dataminr Pulse Integration User Guide
+
+# Overview
+
+### From Alerts to Action—Dataminr Pulse for Elastic Security
+
+Embed Dataminr Pulse real-time, actionable intelligence directly into Elastic Security. Transform the earliest external threat signals from over 1.1 million public, deep, and dark web sources into Elastic-native detections, enriched indices, and automated workflows.
+
+The Dataminr Pulse integration for Elastic Security seamlessly bridges the gap between external real-time data and your internal security operations. By leveraging the Elastic Agent with a CEL (Common Expression Language) input, the integration polls the Dataminr Pulse v4 API at configurable intervals—automatically managing OAuth token refresh and cursor-based pagination to ensure a continuous, resilient data flow.
+
+### Turn Signal Overload into Real-Time, AI-Powered Intelligence
+
+Stay ahead of the threat curve and be the first to see rapidly emerging and evolving threats, vulnerabilities, exploits, ransomware activity, third-party incidents, and more—often hours or days before traditional sources.
+
+**Unmatched Coverage, Precision, and Granularity**
+
+With Dataminr Pulse for Cyber Risk, security teams gain a critical time advantage. Dataminr processes more than 45 terabytes of daily public data, leveraging over 55 proprietary LLMs and 15 years of historic alerting information. With multimodal fusion AI, GenAI, and Agentic AI deeply embedded into the platform, security teams can:
+
+* **Dynamically detect and defend** digital assets beyond the perimeter.
+* **Unearth hidden threats** and close blind spots with advanced processing of text, images, video, and machine signals.
+* **Leverage Agentic AI-powered Intel Agents** to autonomously assemble adversary context, including TTPs, IOCs, CVEs, and MITRE ATT\&CK® mappings.
+* **Proactively prioritize and patch** fast-breaking vulnerabilities and exploits.
+
+
+**Accelerate Elastic Workflows with Actionable Context**
+
+* **Native ECS Mapping:** Ingested alerts are automatically mapped to the Elastic Common Schema (ECS) and stored in the logs-dataminr\_pulse.alerts-\* data stream for immediate correlation.
+* **Granular Entity Expansion:** The integration automatically expands alerts containing multiple discovered entities, such as vulnerabilities, threat actors, malware, IP addresses, and URLs into individual documents for deep, granular analysis.
+* **Integrated Detection & Response:** Populate Elastic Security signals and enrich indices to power advanced hunting, Kibana Dashboards, and Elastic Defend workflows.
+
+
+**Address Your Critical Use Cases with Dataminr**
+
+| Use Case | How Dataminr & Elastic Work Together |
+| :---- | :---- |
+| **Cyber Threat Intelligence** | Piece together attack context with crucial details about threat actors and malware directly within the Elastic Security Threat Intel view and custom dashboards specific to your Dataminr topics. |
+| **Vulnerability Prioritization** | See the earliest signals of PoC exploitation to prioritize patching within the Elastic Vulnerability Management framework. |
+| **Third-Party Risk** | Instantly identify and track supply chain attacks and vendor disruptions as they unfold in real-time. |
+| **Digital Risk Protection** | Spot credential dumps, phishing attempts, and brand impersonations involving your digital footprint. |
+| **Cyber-Physical Convergence** | Assess the complete blast radius of physical events and coordinate a unified response to converged threats. |
+
+# Dependencies
+
+Before installing, ensure the following requirements are met.
+
+## Dataminr Dependencies
+
+| Dependency | Requirement |
+| :---- | :---- |
+| Active Dataminr Pulse API account | Required |
+| Client ID | Required |
+| Client Secret | Required |
+| Dataminr Pulse API version | v4 |
+
+## Elastic Security Dependencies
+
+| Dependency | Requirement |
+| :---- | :---- |
+| Elastic Stack version | 8.13.0 or newer |
+| Kibana version | 8.13.0 or newer |
+| Elastic Agent version | 8.13.0 or newer |
+| Elastic subscription | Basic or higher |
+
+## Network Requirements
+
+The host running the Elastic Agent must have outbound HTTPS (port 443\) access to the following endpoints:
+
+| Destination | Purpose |
+| :---- | :---- |
+| api.dataminr.com | For authentication and to fetch alerts from Pulse API |
+| Fleet Server URL | Agent enrollment and policy management within the clients systems |
+
+**Note:** No inbound ports are required on the agent host.
+
+# Set up
+
+## Install the Integration
+
+1. Log in to elastic and navigate to the **Integrations** page under **Data Management**.
+2. Search for **Dataminr Pulse** in the integrations catalog.
+3. Click the **Dataminr Pulse** integration card, then click **Add Dataminr Pulse**.
+
+## Configure the Integration
+
+#### Dataminr Configuration
+
+To use this integration, you need a Dataminr Pulse API account with a valid Client ID and Client Secret. Contact your Dataminr account representative to obtain the API credentials.
+
+**Important:** Keep your Client Secret secure. It will be stored as a secret value in the Elastic Agent policy and will not be visible after saving.
+
+#### Elastic Configuration
+
+#### **Step 1: Configure and Deploy the Integration**
+
+After clicking **Add Dataminr Pulse** from the integrations page, fill in the configuration form.
+
+1. Browse to the data management and select integrations.
+2. Search ‘Dataminr Pulse’ to install the integration and follow the onscreen instructions.
+3. Optionally, configure the **Polling Interval**, and **Page Size**. The defaults work for most deployments.
+4. Select an existing **Agent policy** or create a new one.
+5. Click **Save and continue**, then **Save and deploy changes**.
+
+#### **Configuration Parameters**
+
+The table below describes all available configuration parameters.
+
+| Parameter | Description | Required | Default |
+| :---- | :---- | :---- | :---- |
+| Integration name | A descriptive name visible on the Elastic Agent policy. | No | dataminr\_pulse |
+| API URL | The full URL for the Dataminr Pulse alerts endpoint. | Yes | https://api.dataminr.com/pulse/v1/alerts |
+| Client ID | Your Dataminr API Client ID for OAuth token generation. | Yes | \- |
+| Client Secret | Your Dataminr API Client Secret for OAuth token generation. Stored securely. | Yes | \- |
+| Interval | How often the integration polls the Dataminr API for new alerts (e.g., 5m, 1m, 10m). | Yes | 5m |
+| Page Size | Maximum number of alerts returned per API request. The maximum allowed value is 100\. | Yes | 40 |
+| Tags | Custom tags applied to each ingested event. | No | forwarded, dataminr-pulse-alerts |
+| Preserve original event | When enabled, stores the raw API response in the event.original field. Useful for debugging. | No | false |
+| Enable request tracing | Logs full HTTP request/response details for debugging. Do not enable in production \- this logs credentials in plain text. | No | false |
+| Processors | Custom Elastic Agent processors in YAML format, applied before data is sent to Elasticsearch. | No | \- |
+
+#### **Configuration Performance Recommendations**
+
+* **Polling Interval**: Start with the default of 5m. Reduce to 1m only if you need near-real-time alert ingestion and your Dataminr API quota allows it.
+* **Page Size**: The default of 40 works well for most deployments. Increase up to 100 if you expect high alert volumes to reduce the number of API calls.
+* **Preserve original event**: Keep disabled in production. Enabling it roughly doubles the storage per document.
+
+#### **Step 2: Validate the Data Flow**
+
+After deploying the integration, verify that data is flowing correctly.
+
+##### **Fleet**
+
+1. Navigate to **Assets** \> **Fleet**.
+2. Under **Agents** locate your enrolled agent.
+3. Verify the agent status is **Healthy** (green).
+4. Click the agent name, then click the **Logs** tab.
+5. Look for log entries showing successful execution of CEL script within agent (Ex: “Unit state changed cel-default (STARTING-\>HEALTHY): Healthy”)
+
+##### **Discover**
+
+1. Navigate to **Discover** and create a session
+2. Set the **Index pattern** to logs-dataminr\_pulse.alerts\* under **Data view**
+3. Select the time to be the last hour, and confirm documents are appearing.
+
+##### **Index Management**
+
+1. Navigate to **Data Management** \> **Streams**
+2. Search for logs-dataminr\_pulse.alerts\*. Verify the index exists and the document count is increasing.
+
+##### **Dashboards**
+
+1. Navigate to **Dashboards**.
+2. Search for **Dataminr**. The integration includes pre-built dashboards for alert monitoring.
+3. Open a dashboard and verify it displays data.
+
+## Delete the Integration
+
+To remove the Dataminr Pulse integration from an agent policy:
+
+1. Navigate to **Assets** \> **Fleet \> Policies**
+2. Click the policy that contains the Dataminr Pulse integration.
+3. Locate the **Dataminr Pulse** integration entry and click the **Actions** menu (three dots), then select **Delete integration**.
+
+**Note:** Deleting the integration from the policy stops data collection but does not remove already-ingested data.
+
+## Reset Integration Assets
+
+If integration assets (dashboards, index templates, ingest pipelines) become corrupted or out of sync, you can reset them.
+
+1. Navigate to **Data Management** \> **Integrations**.
+2. Click **Dataminr Pulse**.
+3. Select the **Settings** tab.
+4. Click **Reinstall Dataminr Pulse**. This reinstalls dashboards, index templates, and ingest pipelines to their default state.
+
+# Data Mappings
+
+The integration maps Dataminr Pulse alert fields to Elastic Common Schema (ECS) fields. Custom fields are stored under the dataminr\_pulse namespace for reference.
+
+## ECS Field Mappings
+
+| Dataminr Pulse Field | ECS Field | Description |
+| :---- | :---- | :---- |
+| Alert timestamp | @timestamp | Event timestamp |
+| Alert headline | message | Single-sentence event summary |
+| Alert ID | event.id | Unique alert identifier |
+| Alert creation time | event.created | When the alert was created |
+| Alert priority (Alert, Urgent, Flash) | event.severity | Numeric severity (10, 20, 30\) |
+| Dataminr alert URL | event.url | Link to alert in Dataminr platform |
+| Dataminr alert location coordinates | geo.location | Coordinates of the Dataminr alert |
+| Dataminr alert location name | geo.name | address of the Dataminr alert |
+| Dataminr entity category | event.category | Categories \- Threat Actor, Vulnerability, Malware |
+| Threat actor name | threat.group.name | Threat actor name (MITRE ATT\&CK) |
+| Threat actor aliases | threat.group.alias | Threat actor alternative names |
+| Threat actor country of origin | threat.indicator.geo.country\_iso\_code | Country of Origin for threat actors |
+| CVE ID | vulnerability.id | CVE identifier |
+| CVSS score | vulnerability.score.base | CVSS base score |
+| Vulnerability description | vulnerability.description | Summary of the vulnerability |
+| Type to distinguish URL vs IP IOC | threat.enrichments\[\].indicator.type | Values \- ip4-ddr, ip6-addr, url |
+| URL | Threat.enrichments\[\].indicator.url.original | URL discovered to be related to the alert, as in the original form |
+| URL or IP addresses | threat.enrichments\[\].indicator.name | URL/IP addresses discovered to be related to the alert. |
+| IP Addresses | threat.enrichments\[\].indicator.ip | IP Addresses discovered to be related to the alert. |
+| Port | threat.enrichments\[\].indicator.port\[\] | Ports discovered to be associated with the IP addresses above |
+
+## Custom Dataminr Fields
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| dataminr\_pulse.categories.name | keyword | Alert topic categories |
+| dataminr\_pulse.companies.name | keyword | Affected company names |
+| dataminr\_pulse.sectors.name | keyword | Industry sectors |
+| dataminr\_pulse.source.href | keyword | URL to the public source post |
+| dataminr\_pulse.source.channels | keyword | Source channels (e.g., sensor) |
+| dataminr\_pulse.source.media.href | keyword | Media attachment URLs |
+| dataminr\_pulse.intel\_agents.summary | keyword | AI-generated critical context summary |
+| dataminr\_pulse.watchlists\_matched\_by\_type.name | keyword | Matched watchlist names |
+| dataminr\_pulse.alert\_type.name | keyword | Alert priority level (Alert, Urgent, Flash) |
+| dataminr\_pulse.live\_brief.summary | keyword | AI-generated event summary |
+| dataminr\_pulse.live\_brief.version | keyword | Live Brief version |
+| dataminr\_pulse.live\_brief.timestamp | date | Live Brief generation timestamp |
+| Dataminr\_pulse.threatactor | keyword | Threat actors discovered in the alert |
+| Dataminr\_pulse.threatactor.alias | keyword | Alternative names of threat actors discovered in the alert |
+| dataminr\_pulse.threatactor.country\_of\_origin | keyword | Country of origin for threat actors discovered in the alert |
+| dataminr\_pulse.vulnerability.name | keyword | Vulnerability identifiers (CVE IDs) |
+| Dataminr\_pulse.event.malware | keyword | Malwares discovered in the alert |
+| dataminr\_pulse.platforms | keyword | Operating systems that were discovered to be impacted because of the malwares |
+| Dataminr\_pulse.url | keyword | URLs discovered to be impacted |
+| Dataminr\_pulse.ip | keyword | IP address (for IP-type entities) |
+
+## Operational Fields
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| dataminr\_pulse.log.log\_type | keyword | Values: “alert-fetch” or “auth” |
+| dataminr\_pulse.log.api\_endpoint | keyword | Full API used to fetch alerts or for authentication |
+| dataminr\_pulse.log.http\_status\_code | keyword | HTTP response for the API call |
+| dataminr\_pulse.log.fetched\_alerts | long | Number of alerts fetched in the batch |
+| dataminr\_pulse.log.fetch\_timestamp | date | Timestamp when the Alert API call was made |
+| dataminr\_pulse.log.next\_cursor | keyword | Pagination cursor to be used in next batch |
+| dataminr\_pulse.log.status | keyword | If the iteration failed or succeded |
+
+# Troubleshooting
+
+## Enable Request Tracing
+
+Request tracing logs full HTTP request and response details, which is useful for diagnosing connectivity or authentication issues.
+
+**Important:** Request tracing logs credentials in plain text. Only enable it temporarily for debugging and disable it immediately after.
+
+1. Navigate to **Assets** \> **Fleet \> Policies**
+2. Click the policy that contains the Dataminr Pulse integration.
+3. Locate the **Dataminr Pulse** integration entry and click the **Actions** menu (three dots), then select **Edit integration**.
+4. Under advanced settings, set **Enable request tracing** to true.
+5. Click **Save and deploy changes**.
+6. To view traces, navigate to **Assets** \> **Fleet** \> **Agents**
+7. Click the agent, then click **Actions** \> **Request diagnostics**.
+8. Download the diagnostics bundle and examine the HTTP trace logs in the agent log files.
+
+### Common Errors
+
+| HTTP Status | Error | Explanation |
+| :---- | :---- | :---- |
+| 400 | Bad Request | The API URL is malformed or a request parameter is invalid. Verify the API URL and Base URL fields. |
+| 401 | Unauthorized | Authentication failed. Verify your Client ID and Client Secret are correct and the account is active. |
+| 403 | Forbidden | The API credentials do not have permission to access the requested resource. Contact your Dataminr account representative. |
+| 404 | Not Found | The API endpoint URL is incorrect. Ensure the API URL is set to https://api.dataminr.com/pulse/v1/alerts. |
+| 429 | Too Many Requests | API rate limit exceeded. Increase the Interval value or reduce the Page Size. |
+
+### No Data Appearing
+
+1. Check Agent Status: Navigate to Fleet \> Agents and verify the agent is Healthy.
+2. Check Agent Logs: Click the agent and review the Logs tab for error messages.
+3. Test Credentials: On the agent host, run:
+ 1. curl \-X POST [https://userauth.dataminr.com/auth/2/token](https://userauth.dataminr.com/auth/2/token) \-H "Content-Type: application/x-www-form-urlencoded" \-d "grant\_type=api\_key\&client\_id=YOUR\_ID\&client\_secret=YOUR\_SECRET"
+ 2. A successful response returns a JSON object with dmaToken and expire fields.
+4. Check Data Stream: In Dev Tools, run:
+ 1. GET logs-dataminr\_pulse.alerts-\*/\_count
+ 2. If the count is 0, the integration is not receiving data from the API. Review the agent logs for details.
+
+### Duplicate Alerts
+
+The integration uses document fingerprinting to prevent duplicates. If you observe duplicate documents:
+
+1. Verify the ingest pipeline is installed by running in Dev Tools:
+ 1. GET \_ingest/pipeline/logs-dataminr\_pulse.alerts-\*
+ 2. If missing, reset integration assets (see **Reset Integration Assets**).
\ No newline at end of file
diff --git a/packages/dataminr_pulse/img/Dataminr_Logo-Mark_Full-Color_RGB.svg b/packages/dataminr_pulse/img/Dataminr_Logo-Mark_Full-Color_RGB.svg
new file mode 100644
index 00000000000..32503d7354e
--- /dev/null
+++ b/packages/dataminr_pulse/img/Dataminr_Logo-Mark_Full-Color_RGB.svg
@@ -0,0 +1,7 @@
+
diff --git a/packages/dataminr_pulse/img/logo.svc b/packages/dataminr_pulse/img/logo.svc
new file mode 100644
index 00000000000..2d2ed8275ad
--- /dev/null
+++ b/packages/dataminr_pulse/img/logo.svc
@@ -0,0 +1,4 @@
+
\ No newline at end of file
diff --git a/packages/dataminr_pulse/img/screenshot.png b/packages/dataminr_pulse/img/screenshot.png
new file mode 100644
index 00000000000..276246f211b
Binary files /dev/null and b/packages/dataminr_pulse/img/screenshot.png differ
diff --git a/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1823.json b/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1823.json
new file mode 100644
index 00000000000..c695a2bd402
--- /dev/null
+++ b/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1823.json
@@ -0,0 +1,3111 @@
+{
+ "attributes": {
+ "description": "",
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [
+ {
+ "meta": {
+ "alias": null,
+ "disabled": false,
+ "key": "data_stream.dataset",
+ "negate": false,
+ "params": {
+ "query": "dataminr_pulse.alerts"
+ },
+ "type": "phrase"
+ },
+ "query": {
+ "match_phrase": {
+ "data_stream.dataset": "dataminr_pulse.alerts"
+ }
+ }
+ },
+ {
+ "meta": {
+ "alias": null,
+ "disabled": false,
+ "key": "dataminr_pulse.is_main",
+ "negate": false,
+ "params": {
+ "query": "yes"
+ },
+ "type": "phrase"
+ },
+ "query": {
+ "match_phrase": {
+ "dataminr_pulse.is_main": "yes"
+ }
+ }
+ }
+ ],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ }
+ }
+ },
+ "optionsJSON": {
+ "hidePanelTitles": false,
+ "syncColors": false,
+ "syncCursor": true,
+ "syncTooltips": false,
+ "useMargins": true
+ },
+ "panelsJSON": [
+ {
+ "embeddableConfig": {
+ "timeRange": {
+ "from": "now-48h",
+ "to": "now"
+ },
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "layers": {
+ "7a953b36-140a-463e-9ec2-b2e7a467b602": {
+ "columnOrder": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2",
+ "01f59e67-93e4-4ad6-8a96-c105840eb359"
+ ],
+ "columns": {
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') ",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3"
+ ],
+ "scale": "ratio"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1"
+ ],
+ "location": {
+ "max": 218,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "location": {
+ "max": 396,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') ",
+ "type": "function"
+ }
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "scale": "ratio"
+ },
+ "01f59e67-93e4-4ad6-8a96-c105840eb359": {
+ "dataType": "number",
+ "isBucketed": false,
+ "isStaticValue": true,
+ "label": "Static value: 20000",
+ "operationType": "static_value",
+ "params": {
+ "value": "20000"
+ },
+ "references": [],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2"
+ ],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "location": {
+ "max": 217,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "incompleteColumns": {},
+ "sampling": 1,
+ "ignoreGlobalFilters": true
+ },
+ "f28f4221-2d16-48ff-a506-a0213d3afdda": {
+ "columnOrder": [
+ "128f27b7-e830-476e-89c4-107e1aa13e0d",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "columns": {
+ "128f27b7-e830-476e-89c4-107e1aa13e0d": {
+ "dataType": "date",
+ "isBucketed": true,
+ "label": "@timestamp",
+ "operationType": "date_histogram",
+ "params": {
+ "dropPartials": false,
+ "includeEmptyRows": true,
+ "interval": "auto"
+ },
+ "scale": "interval",
+ "sourceField": "@timestamp"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "scale": "ratio"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "location": {
+ "max": 217,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aa": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') ",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1"
+ ],
+ "location": {
+ "max": 218,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "location": {
+ "max": 396,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\"') ",
+ "type": "function"
+ }
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "ignoreGlobalFilters": true,
+ "incompleteColumns": {},
+ "linkToLayers": [
+ "7a953b36-140a-463e-9ec2-b2e7a467b602"
+ ],
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\") and dataminr_pulse.is_main : \"yes\""
+ },
+ "visualization": {
+ "layerId": "7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "layerType": "data",
+ "maxAccessor": "01f59e67-93e4-4ad6-8a96-c105840eb359",
+ "metricAccessor": "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "palette": {
+ "name": "custom",
+ "params": {
+ "colorStops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ],
+ "continuity": "all",
+ "maxSteps": 5,
+ "name": "custom",
+ "progression": "fixed",
+ "rangeMax": null,
+ "rangeMin": null,
+ "rangeType": "number",
+ "reverse": false,
+ "steps": 3,
+ "stops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ]
+ },
+ "type": "palette"
+ },
+ "secondaryMetricAccessor": "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "showBar": false,
+ "subtitle": "",
+ "trendlineLayerId": "f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "trendlineLayerType": "metricTrendline",
+ "trendlineMetricAccessor": "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "trendlineSecondaryMetricAccessor": "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "trendlineTimeAccessor": "128f27b7-e830-476e-89c4-107e1aa13e0d"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsMetric"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 8,
+ "i": "e2350e54-e176-4ce0-a782-bf1f6b8b36f2",
+ "w": 11,
+ "x": 0,
+ "y": 0
+ },
+ "panelIndex": "e2350e54-e176-4ce0-a782-bf1f6b8b36f2",
+ "title": "Threat Actor Activities",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "timeRange": {
+ "from": "now-48h",
+ "to": "now"
+ },
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "7a953b36-140a-463e-9ec2-b2e7a467b602": {
+ "columnOrder": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3"
+ ],
+ "columns": {
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3"
+ ],
+ "scale": "ratio"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1"
+ ],
+ "location": {
+ "max": 288,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "location": {
+ "max": 526,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2"
+ ],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "location": {
+ "max": 287,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1,
+ "ignoreGlobalFilters": true
+ },
+ "f28f4221-2d16-48ff-a506-a0213d3afdda": {
+ "columnOrder": [
+ "128f27b7-e830-476e-89c4-107e1aa13e0d",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3"
+ ],
+ "columns": {
+ "128f27b7-e830-476e-89c4-107e1aa13e0d": {
+ "dataType": "date",
+ "isBucketed": true,
+ "label": "@timestamp",
+ "operationType": "date_histogram",
+ "params": {
+ "dropPartials": false,
+ "includeEmptyRows": true,
+ "interval": "auto"
+ },
+ "scale": "interval",
+ "sourceField": "@timestamp"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "scale": "ratio"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "location": {
+ "max": 287,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aa": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1"
+ ],
+ "location": {
+ "max": 288,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "location": {
+ "max": 526,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "ignoreGlobalFilters": true,
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "linkToLayers": [
+ "7a953b36-140a-463e-9ec2-b2e7a467b602"
+ ],
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\") and dataminr_pulse.is_main : \"yes\""
+ },
+ "visualization": {
+ "layerId": "7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "layerType": "data",
+ "metricAccessor": "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "palette": {
+ "name": "custom",
+ "params": {
+ "colorStops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ],
+ "continuity": "all",
+ "maxSteps": 5,
+ "name": "custom",
+ "progression": "fixed",
+ "rangeMax": null,
+ "rangeMin": null,
+ "rangeType": "number",
+ "reverse": false,
+ "steps": 3,
+ "stops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ]
+ },
+ "type": "palette"
+ },
+ "secondaryMetricAccessor": "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "subtitle": "",
+ "trendlineLayerId": "f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "trendlineLayerType": "metricTrendline",
+ "trendlineMetricAccessor": "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "trendlineSecondaryMetricAccessor": "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "trendlineTimeAccessor": "128f27b7-e830-476e-89c4-107e1aa13e0d"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsMetric"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 8,
+ "i": "c13ae850-190d-4162-bbc5-d09722eff0d7",
+ "w": 13,
+ "x": 11,
+ "y": 0
+ },
+ "panelIndex": "c13ae850-190d-4162-bbc5-d09722eff0d7",
+ "title": "Cyber Attacks",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "timeRange": {
+ "from": "now-48h",
+ "to": "now"
+ },
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "7a953b36-140a-463e-9ec2-b2e7a467b602": {
+ "columnOrder": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2",
+ "01f59e67-93e4-4ad6-8a96-c105840eb359"
+ ],
+ "columns": {
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3"
+ ],
+ "scale": "ratio"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1"
+ ],
+ "location": {
+ "max": 320,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "location": {
+ "max": 520,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "scale": "ratio"
+ },
+ "01f59e67-93e4-4ad6-8a96-c105840eb359": {
+ "dataType": "number",
+ "isBucketed": false,
+ "isStaticValue": true,
+ "label": "Static value: 20000",
+ "operationType": "static_value",
+ "params": {
+ "value": "20000"
+ },
+ "references": [],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2"
+ ],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "location": {
+ "max": 319,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1,
+ "ignoreGlobalFilters": true
+ },
+ "f28f4221-2d16-48ff-a506-a0213d3afdda": {
+ "columnOrder": [
+ "128f27b7-e830-476e-89c4-107e1aa13e0d",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "columns": {
+ "128f27b7-e830-476e-89c4-107e1aa13e0d": {
+ "dataType": "date",
+ "isBucketed": true,
+ "label": "@timestamp",
+ "operationType": "date_histogram",
+ "params": {
+ "dropPartials": false,
+ "includeEmptyRows": true,
+ "interval": "auto"
+ },
+ "scale": "interval",
+ "sourceField": "@timestamp"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "scale": "ratio"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "location": {
+ "max": 319,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aa": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1"
+ ],
+ "location": {
+ "max": 320,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "location": {
+ "max": 520,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "ignoreGlobalFilters": true,
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "linkToLayers": [
+ "7a953b36-140a-463e-9ec2-b2e7a467b602"
+ ],
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\") and dataminr_pulse.is_main : \"yes\""
+ },
+ "visualization": {
+ "layerId": "7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "layerType": "data",
+ "maxAccessor": "01f59e67-93e4-4ad6-8a96-c105840eb359",
+ "metricAccessor": "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "palette": {
+ "name": "custom",
+ "params": {
+ "colorStops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ],
+ "continuity": "all",
+ "maxSteps": 5,
+ "name": "custom",
+ "progression": "fixed",
+ "rangeMax": null,
+ "rangeMin": null,
+ "rangeType": "number",
+ "reverse": false,
+ "steps": 3,
+ "stops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ]
+ },
+ "type": "palette"
+ },
+ "secondaryMetricAccessor": "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "showBar": false,
+ "subtitle": "",
+ "trendlineLayerId": "f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "trendlineLayerType": "metricTrendline",
+ "trendlineMetricAccessor": "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "trendlineSecondaryMetricAccessor": "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "trendlineTimeAccessor": "128f27b7-e830-476e-89c4-107e1aa13e0d"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsMetric"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 8,
+ "i": "23c5e676-4dab-4965-92da-85ff6aa8ac33",
+ "w": 13,
+ "x": 24,
+ "y": 0
+ },
+ "panelIndex": "23c5e676-4dab-4965-92da-85ff6aa8ac33",
+ "title": "Data Breaches",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "timeRange": {
+ "from": "now-48h",
+ "to": "now"
+ },
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "7a953b36-140a-463e-9ec2-b2e7a467b602": {
+ "columnOrder": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2",
+ "01f59e67-93e4-4ad6-8a96-c105840eb359"
+ ],
+ "columns": {
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') ",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3"
+ ],
+ "scale": "ratio"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1"
+ ],
+ "location": {
+ "max": 400,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "location": {
+ "max": 700,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') ",
+ "type": "function"
+ }
+ },
+ "references": [
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X0",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X1",
+ "015bb955-8972-462f-98d2-85f8b2e4bbe5X2"
+ ],
+ "scale": "ratio"
+ },
+ "01f59e67-93e4-4ad6-8a96-c105840eb359": {
+ "dataType": "number",
+ "isBucketed": false,
+ "isStaticValue": true,
+ "label": "Static value: 20000",
+ "operationType": "static_value",
+ "params": {
+ "value": "20000"
+ },
+ "references": [],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2"
+ ],
+ "scale": "ratio"
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "location": {
+ "max": 399,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X0",
+ "cdd88c9c-502c-40bd-9e29-cb3c74834c90X1"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1,
+ "ignoreGlobalFilters": true
+ },
+ "f28f4221-2d16-48ff-a506-a0213d3afdda": {
+ "columnOrder": [
+ "128f27b7-e830-476e-89c4-107e1aa13e0d",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "columns": {
+ "128f27b7-e830-476e-89c4-107e1aa13e0d": {
+ "dataType": "date",
+ "isBucketed": true,
+ "label": "@timestamp",
+ "operationType": "date_histogram",
+ "params": {
+ "dropPartials": false,
+ "includeEmptyRows": true,
+ "interval": "auto"
+ },
+ "scale": "interval",
+ "sourceField": "@timestamp"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " ",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "number",
+ "params": {
+ "decimals": 0
+ }
+ },
+ "formula": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2"
+ ],
+ "scale": "ratio"
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of ",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "location": {
+ "max": 399,
+ "min": 0
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ }
+ },
+ "references": [
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X0",
+ "1cf93cf8-acf2-4023-955a-360fd8316ac8X1"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aa": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": " Today vs Yesterday",
+ "operationType": "formula",
+ "params": {
+ "format": {
+ "id": "percent",
+ "params": {
+ "decimals": 2
+ }
+ },
+ "formula": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') ",
+ "isFormulaBroken": false
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3"
+ ],
+ "scale": "ratio"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": false
+ },
+ "scale": "ratio",
+ "sourceField": "___records___",
+ "filter": {
+ "language": "kuery",
+ "query": "@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ }
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX3": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Part of Today vs Yesterday",
+ "operationType": "math",
+ "params": {
+ "tinymathAst": {
+ "args": [
+ {
+ "args": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1"
+ ],
+ "location": {
+ "max": 400,
+ "min": 1
+ },
+ "name": "subtract",
+ "text": "count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')",
+ "type": "function"
+ },
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "location": {
+ "max": 700,
+ "min": 0
+ },
+ "name": "divide",
+ "text": "(count(kql='@timestamp >= now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') - count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"')) / count(kql='@timestamp >= now-48h and @timestamp < now-24h and dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\"') ",
+ "type": "function"
+ }
+ },
+ "references": [
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX0",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX1",
+ "537558d3-6747-44f4-9b82-f30841d6d5aaX2"
+ ],
+ "scale": "ratio"
+ }
+ },
+ "ignoreGlobalFilters": true,
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "linkToLayers": [
+ "7a953b36-140a-463e-9ec2-b2e7a467b602"
+ ],
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\") and dataminr_pulse.is_main : \"yes\""
+ },
+ "visualization": {
+ "layerId": "7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "layerType": "data",
+ "maxAccessor": "01f59e67-93e4-4ad6-8a96-c105840eb359",
+ "metricAccessor": "015bb955-8972-462f-98d2-85f8b2e4bbe5",
+ "palette": {
+ "name": "custom",
+ "params": {
+ "colorStops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ],
+ "continuity": "all",
+ "maxSteps": 5,
+ "name": "custom",
+ "progression": "fixed",
+ "rangeMax": null,
+ "rangeMin": null,
+ "rangeType": "number",
+ "reverse": false,
+ "steps": 3,
+ "stops": [
+ {
+ "color": "#cc5642",
+ "stop": -1
+ },
+ {
+ "color": "#d6bf57",
+ "stop": 0
+ },
+ {
+ "color": "#209280",
+ "stop": 0.01
+ }
+ ]
+ },
+ "type": "palette"
+ },
+ "secondaryMetricAccessor": "cdd88c9c-502c-40bd-9e29-cb3c74834c90",
+ "showBar": false,
+ "subtitle": "",
+ "trendlineLayerId": "f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "trendlineLayerType": "metricTrendline",
+ "trendlineMetricAccessor": "537558d3-6747-44f4-9b82-f30841d6d5aa",
+ "trendlineSecondaryMetricAccessor": "1cf93cf8-acf2-4023-955a-360fd8316ac8",
+ "trendlineTimeAccessor": "128f27b7-e830-476e-89c4-107e1aa13e0d"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsMetric"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 8,
+ "i": "2f052de6-e0b9-4205-b01a-40e283bf64a9",
+ "w": 11,
+ "x": 37,
+ "y": 0
+ },
+ "panelIndex": "2f052de6-e0b9-4205-b01a-40e283bf64a9",
+ "title": "Vulnerabilities",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "1734381c-4809-4e1c-89c4-9b54a02f4141": {
+ "columnOrder": [
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "e1cbd21a-8161-436d-af86-786480bc8a1c"
+ ],
+ "columns": {
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0": {
+ "customLabel": true,
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Selected Topics",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [
+ "Threat Actors - Ransomware",
+ "Threat Actors - Hacktivists",
+ "Threat Actors - Advanced Persistent Threats"
+ ],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": false,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 3
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.categories.name"
+ },
+ "e1cbd21a-8161-436d-af86-786480bc8a1c": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : * "
+ },
+ "visualization": {
+ "columns": [
+ {
+ "columnId": "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "isTransposed": false,
+ "width": 252
+ },
+ {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "isTransposed": false
+ }
+ ],
+ "headerRowHeight": "single",
+ "headerRowHeightLines": 1,
+ "layerId": "1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "layerType": "data",
+ "paging": {
+ "enabled": false,
+ "size": 10
+ },
+ "rowHeight": "custom",
+ "rowHeightLines": 2
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsDatatable"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 11,
+ "i": "d06d12f2-efff-4825-bf64-0c37afa46409",
+ "w": 11,
+ "x": 0,
+ "y": 8
+ },
+ "panelIndex": "d06d12f2-efff-4825-bf64-0c37afa46409",
+ "title": "Trending Threat Actor Topics",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "1734381c-4809-4e1c-89c4-9b54a02f4141": {
+ "columnOrder": [
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "e1cbd21a-8161-436d-af86-786480bc8a1c"
+ ],
+ "columns": {
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0": {
+ "customLabel": true,
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Selected Topics",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [
+ "Malware",
+ "Network Disruptions",
+ "Phishing",
+ "DDoS",
+ "DDoS Attacks",
+ "Defacement",
+ "Domain Impersonation",
+ "Network Scans"
+ ],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": false,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 50
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.categories.name"
+ },
+ "e1cbd21a-8161-436d-af86-786480bc8a1c": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : *"
+ },
+ "visualization": {
+ "columns": [
+ {
+ "columnId": "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "isTransposed": false
+ },
+ {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "isTransposed": false
+ }
+ ],
+ "headerRowHeight": "single",
+ "headerRowHeightLines": 1,
+ "layerId": "1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "layerType": "data",
+ "paging": {
+ "enabled": false,
+ "size": 10
+ },
+ "rowHeight": "custom",
+ "rowHeightLines": 2
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsDatatable"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 11,
+ "i": "05f01c39-772c-4f08-b444-b882365cdeae",
+ "w": 13,
+ "x": 11,
+ "y": 8
+ },
+ "panelIndex": "05f01c39-772c-4f08-b444-b882365cdeae",
+ "title": "Trending Cyber Attacks Topics",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "1734381c-4809-4e1c-89c4-9b54a02f4141": {
+ "columnOrder": [
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "e1cbd21a-8161-436d-af86-786480bc8a1c"
+ ],
+ "columns": {
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0": {
+ "customLabel": true,
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Selected Topics",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [
+ "Data Exposure and Breaches",
+ "Doxxing and Leaked Credentials",
+ "Data Exposure & Breaches",
+ "Doxxing & Leaked Credentials",
+ "Leaked Credentials and Doxxing",
+ "Leaked Credentials & Doxxing"
+ ],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": false,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 50
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.categories.name"
+ },
+ "e1cbd21a-8161-436d-af86-786480bc8a1c": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : * "
+ },
+ "visualization": {
+ "columns": [
+ {
+ "columnId": "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "isTransposed": false
+ },
+ {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "isTransposed": false
+ }
+ ],
+ "headerRowHeight": "single",
+ "headerRowHeightLines": 1,
+ "layerId": "1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "layerType": "data",
+ "paging": {
+ "enabled": false,
+ "size": 10
+ },
+ "rowHeight": "custom",
+ "rowHeightLines": 2
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsDatatable"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 11,
+ "i": "034c9ea8-3086-42f0-9f4c-c59923b0c844",
+ "w": 13,
+ "x": 24,
+ "y": 8
+ },
+ "panelIndex": "034c9ea8-3086-42f0-9f4c-c59923b0c844",
+ "title": "Trending Data Breaches",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "1734381c-4809-4e1c-89c4-9b54a02f4141": {
+ "columnOrder": [
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "e1cbd21a-8161-436d-af86-786480bc8a1c"
+ ],
+ "columns": {
+ "13fd7d63-1d9f-42f4-8732-95140ccd1ba0": {
+ "customLabel": true,
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Selected Topics",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [
+ "Email and Web Servers",
+ "Email & Web Servers",
+ "Encryption and Certificates",
+ "Encryption & Certificates",
+ "Remote Access and Management Systems",
+ "Remote Access & Management Systems",
+ "Vulnerability Disclosures",
+ "Exploits",
+ "Bug Bounties",
+ "Emerging Vulnerabilities",
+ "Trending Vulnerabilities"
+ ],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": false,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 50
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.categories.name"
+ },
+ "e1cbd21a-8161-436d-af86-786480bc8a1c": {
+ "customLabel": true,
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ }
+ },
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : * "
+ },
+ "visualization": {
+ "columns": [
+ {
+ "columnId": "13fd7d63-1d9f-42f4-8732-95140ccd1ba0",
+ "isTransposed": false
+ },
+ {
+ "columnId": "e1cbd21a-8161-436d-af86-786480bc8a1c",
+ "isTransposed": false
+ }
+ ],
+ "headerRowHeight": "single",
+ "headerRowHeightLines": 1,
+ "layerId": "1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "layerType": "data",
+ "paging": {
+ "enabled": false,
+ "size": 10
+ },
+ "rowHeight": "custom",
+ "rowHeightLines": 2
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsDatatable"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 11,
+ "i": "94638cc1-ae1b-4235-9946-e2b45d8d7d36",
+ "w": 11,
+ "x": 37,
+ "y": 8
+ },
+ "panelIndex": "94638cc1-ae1b-4235-9946-e2b45d8d7d36",
+ "title": "Trending Vulnerabilities",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "columns": [
+ "message",
+ "dataminr_pulse.categories.name_joined",
+ "event.severity_label",
+ "dataminr_pulse.companies.name_joined",
+ "dataminr_pulse.sectors.name_joined"
+ ],
+ "description": "",
+ "grid": {
+ "columns": {
+ "event.severity": {
+ "width": 271
+ }
+ }
+ },
+ "hideChart": true,
+ "isTextBasedQuery": false,
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [],
+ "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Threat Actors - Advanced Persistent Threats\" or \"Threat Actors - Ransomware\" or \"Threat Actors - Hacktivists\")"
+ }
+ }
+ },
+ "sort": [
+ [
+ "dataminr_pulse.categories.name",
+ "desc"
+ ]
+ ],
+ "timeRestore": false,
+ "title": "Threat Actor Activity",
+ "viewMode": "documents",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 15,
+ "i": "bfff30d1-df2d-437f-9112-9bb503985e2d",
+ "w": 24,
+ "x": 0,
+ "y": 19
+ },
+ "panelIndex": "bfff30d1-df2d-437f-9112-9bb503985e2d",
+ "title": "Threat Actor Activity",
+ "type": "search"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "columns": [
+ "message",
+ "dataminr_pulse.categories.name_joined",
+ "event.severity_label",
+ "dataminr_pulse.companies.name_joined",
+ "dataminr_pulse.sectors.name_joined"
+ ],
+ "description": "",
+ "grid": {},
+ "hideChart": true,
+ "isTextBasedQuery": false,
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [],
+ "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Malware\" or \"Network Disruptions\" or \"Phishing\" or \"DDoS\" or \"DDoS Attacks\" or \"Defacement\" or \"Domain Impersonation\" or \"Network Scans\")"
+ }
+ }
+ },
+ "sort": [
+ [
+ "@timestamp",
+ "desc"
+ ]
+ ],
+ "timeRestore": false,
+ "title": "Cyber Attacks",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 15,
+ "i": "47742010-8dac-46c8-940b-1c5bfb674391",
+ "w": 24,
+ "x": 24,
+ "y": 19
+ },
+ "panelIndex": "47742010-8dac-46c8-940b-1c5bfb674391",
+ "title": "Cyber Attacks",
+ "type": "search"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "columns": [
+ "message",
+ "dataminr_pulse.categories.name_joined",
+ "event.severity_label",
+ "dataminr_pulse.companies.name_joined",
+ "dataminr_pulse.sectors.name_joined"
+ ],
+ "description": "",
+ "grid": {},
+ "hideChart": true,
+ "isTextBasedQuery": false,
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [],
+ "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Data Exposure and Breaches\" or \"Doxxing and Leaked Credentials\" or \"Data Exposure & Breaches\" or \"Doxxing & Leaked Credentials\" or \"Leaked Credentials and Doxxing\" or \"Leaked Credentials & Doxxing\")"
+ }
+ }
+ },
+ "sort": [
+ [
+ "@timestamp",
+ "desc"
+ ]
+ ],
+ "timeRestore": false,
+ "title": "Data Breaches",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 15,
+ "i": "1d85c773-2d65-4d6d-9aad-866af8c1fa4b",
+ "w": 24,
+ "x": 0,
+ "y": 34
+ },
+ "panelIndex": "1d85c773-2d65-4d6d-9aad-866af8c1fa4b",
+ "title": "Data Breaches",
+ "type": "search"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "columns": [
+ "message",
+ "dataminr_pulse.categories.name_joined",
+ "event.severity_label",
+ "dataminr_pulse.companies.name_joined",
+ "dataminr_pulse.sectors.name_joined"
+ ],
+ "description": "",
+ "grid": {},
+ "hideChart": true,
+ "isTextBasedQuery": false,
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [],
+ "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "query": {
+ "language": "kuery",
+ "query": "dataminr_pulse.categories.name : (\"Email and Web Servers\" or \"Email & Web Servers\" or \"Encryption and Certificates\" or \"Encryption & Certificates\" or \"Remote Access and Management Systems\" or \"Remote Access & Management Systems\" or \"Vulnerability Disclosures\" or \"Exploits\" or \"Bug Bounties\" or \"Emerging Vulnerabilities\" or \"Trending Vulnerabilities\")"
+ }
+ }
+ },
+ "sort": [
+ [
+ "dataminr_pulse.categories.name",
+ "desc"
+ ]
+ ],
+ "timeRestore": false,
+ "title": "Vulnerabilities",
+ "viewMode": "documents",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 15,
+ "i": "168673bd-051f-47d1-854d-bf43d419113b",
+ "w": 24,
+ "x": 24,
+ "y": 34
+ },
+ "panelIndex": "168673bd-051f-47d1-854d-bf43d419113b",
+ "title": "Vulnerabilities",
+ "type": "search"
+ }
+ ],
+ "timeRestore": true,
+ "timeFrom": "now-7d",
+ "timeTo": "now",
+ "title": "Dataminr Pulse - Cyber Threat Overview",
+ "version": 2
+ },
+ "coreMigrationVersion": "8.8.0",
+ "created_at": "2026-01-28T14:04:19.306Z",
+ "created_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
+ "id": "dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1823",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "e2350e54-e176-4ce0-a782-bf1f6b8b36f2:indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "e2350e54-e176-4ce0-a782-bf1f6b8b36f2:indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "c13ae850-190d-4162-bbc5-d09722eff0d7:indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "c13ae850-190d-4162-bbc5-d09722eff0d7:indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "23c5e676-4dab-4965-92da-85ff6aa8ac33:indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "23c5e676-4dab-4965-92da-85ff6aa8ac33:indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "2f052de6-e0b9-4205-b01a-40e283bf64a9:indexpattern-datasource-layer-7a953b36-140a-463e-9ec2-b2e7a467b602",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "2f052de6-e0b9-4205-b01a-40e283bf64a9:indexpattern-datasource-layer-f28f4221-2d16-48ff-a506-a0213d3afdda",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "d06d12f2-efff-4825-bf64-0c37afa46409:indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "05f01c39-772c-4f08-b444-b882365cdeae:indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "034c9ea8-3086-42f0-9f4c-c59923b0c844:indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "94638cc1-ae1b-4235-9946-e2b45d8d7d36:indexpattern-datasource-layer-1734381c-4809-4e1c-89c4-9b54a02f4141",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "bfff30d1-df2d-437f-9112-9bb503985e2d:kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "47742010-8dac-46c8-940b-1c5bfb674391:kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "1d85c773-2d65-4d6d-9aad-866af8c1fa4b:kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "168673bd-051f-47d1-854d-bf43d419113b:kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ],
+ "type": "dashboard",
+ "typeMigrationVersion": "10.2.0",
+ "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0"
+}
\ No newline at end of file
diff --git a/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1824.json b/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1824.json
new file mode 100644
index 00000000000..ff997cea0c5
--- /dev/null
+++ b/packages/dataminr_pulse/kibana/dashboard/dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1824.json
@@ -0,0 +1,822 @@
+{
+ "attributes": {
+ "description": "",
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [
+ {
+ "meta": {
+ "alias": null,
+ "disabled": false,
+ "key": "data_stream.dataset",
+ "negate": false,
+ "params": {
+ "query": "dataminr_pulse.alerts"
+ },
+ "type": "phrase"
+ },
+ "query": {
+ "match_phrase": {
+ "data_stream.dataset": "dataminr_pulse.alerts"
+ }
+ }
+ },
+ {
+ "meta": {
+ "alias": null,
+ "disabled": false,
+ "key": "dataminr_pulse.is_main",
+ "negate": false,
+ "params": {
+ "query": "yes"
+ },
+ "type": "phrase"
+ },
+ "query": {
+ "match_phrase": {
+ "dataminr_pulse.is_main": "yes"
+ }
+ }
+ }
+ ],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ }
+ }
+ },
+ "optionsJSON": {
+ "hidePanelTitles": false,
+ "syncColors": false,
+ "syncCursor": true,
+ "syncTooltips": false,
+ "useMargins": true
+ },
+ "panelsJSON": [
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-56bf765b-3d20-4830-b193-e2d9c56b1d5c",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "56bf765b-3d20-4830-b193-e2d9c56b1d5c": {
+ "columnOrder": [
+ "c2cae892-b47b-4610-bcad-3f4e657a5c43",
+ "8e7ab4a0-2dd8-401f-8e8d-7c50cd58f35e",
+ "970b2411-de94-491e-a913-4470ce19f50c",
+ "025ab4e2-22b2-4965-883d-882e7e9c9bd3"
+ ],
+ "columns": {
+ "025ab4e2-22b2-4965-883d-882e7e9c9bd3": {
+ "customLabel": true,
+ "dataType": "number",
+ "filter": {
+ "language": "kuery",
+ "query": "event.severity : 90"
+ },
+ "isBucketed": false,
+ "label": "Flash",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "event.severity"
+ },
+ "8e7ab4a0-2dd8-401f-8e8d-7c50cd58f35e": {
+ "customLabel": true,
+ "dataType": "number",
+ "filter": {
+ "language": "kuery",
+ "query": "event.severity : 30"
+ },
+ "isBucketed": false,
+ "label": "Alert",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "event.severity"
+ },
+ "970b2411-de94-491e-a913-4470ce19f50c": {
+ "customLabel": true,
+ "dataType": "number",
+ "filter": {
+ "language": "kuery",
+ "query": "event.severity : 50"
+ },
+ "isBucketed": false,
+ "label": "Urgent",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "event.severity"
+ },
+ "c2cae892-b47b-4610-bcad-3f4e657a5c43": {
+ "customLabel": true,
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Severity",
+ "operationType": "filters",
+ "params": {
+ "filters": [
+ {
+ "input": {
+ "language": "kuery",
+ "query": ""
+ },
+ "label": ""
+ }
+ ]
+ },
+ "scale": "ordinal"
+ }
+ },
+ "ignoreGlobalFilters": false,
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ },
+ "visualization": {
+ "axisTitlesVisibilitySettings": {
+ "x": true,
+ "yLeft": true,
+ "yRight": true
+ },
+ "fittingFunction": "None",
+ "gridlinesVisibilitySettings": {
+ "x": true,
+ "yLeft": true,
+ "yRight": true
+ },
+ "labelsOrientation": {
+ "x": 0,
+ "yLeft": 0,
+ "yRight": 0
+ },
+ "layers": [
+ {
+ "accessors": [
+ "8e7ab4a0-2dd8-401f-8e8d-7c50cd58f35e",
+ "970b2411-de94-491e-a913-4470ce19f50c",
+ "025ab4e2-22b2-4965-883d-882e7e9c9bd3"
+ ],
+ "colorMapping": {
+ "assignments": [],
+ "colorMode": {
+ "type": "categorical"
+ },
+ "paletteId": "eui_amsterdam_color_blind",
+ "specialAssignments": [
+ {
+ "color": {
+ "type": "loop"
+ },
+ "rule": {
+ "type": "other"
+ },
+ "touched": false
+ }
+ ]
+ },
+ "layerId": "56bf765b-3d20-4830-b193-e2d9c56b1d5c",
+ "layerType": "data",
+ "seriesType": "bar_horizontal",
+ "xAccessor": "c2cae892-b47b-4610-bcad-3f4e657a5c43",
+ "yConfig": [
+ {
+ "color": "#dc4d3d",
+ "forAccessor": "025ab4e2-22b2-4965-883d-882e7e9c9bd3"
+ },
+ {
+ "color": "#e88631",
+ "forAccessor": "970b2411-de94-491e-a913-4470ce19f50c"
+ },
+ {
+ "axisMode": "auto",
+ "color": "#f3ce55",
+ "forAccessor": "8e7ab4a0-2dd8-401f-8e8d-7c50cd58f35e"
+ }
+ ]
+ }
+ ],
+ "legend": {
+ "isVisible": true,
+ "position": "right"
+ },
+ "preferredSeriesType": "bar_horizontal",
+ "tickLabelsVisibilitySettings": {
+ "x": true,
+ "yLeft": true,
+ "yRight": true
+ },
+ "valueLabels": "hide",
+ "yLeftExtent": {
+ "mode": "full"
+ },
+ "yLeftScale": "log"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsXY"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 14,
+ "i": "13787329-329d-4c53-ac30-e4dd78fa8c25",
+ "w": 23,
+ "x": 0,
+ "y": 0
+ },
+ "panelIndex": "13787329-329d-4c53-ac30-e4dd78fa8c25",
+ "title": "Occurrence by Severity",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-3ca66f4c-d42b-437c-8e30-6dde3e6ef3c5",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "layers": {
+ "3ca66f4c-d42b-437c-8e30-6dde3e6ef3c5": {
+ "columnOrder": [
+ "98f752b2-4502-4e16-b6d7-8d276bc539e5",
+ "e1062427-71b6-4866-a836-8c6482598d72"
+ ],
+ "columns": {
+ "98f752b2-4502-4e16-b6d7-8d276bc539e5": {
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Top 10 values of dataminr_pulse.companies.name",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "e1062427-71b6-4866-a836-8c6482598d72",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": true,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 10
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.companies.name"
+ },
+ "e1062427-71b6-4866-a836-8c6482598d72": {
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count of records",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ }
+ },
+ "ignoreGlobalFilters": false,
+ "incompleteColumns": {},
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ },
+ "visualization": {
+ "layers": [
+ {
+ "categoryDisplay": "default",
+ "colorMapping": {
+ "assignments": [],
+ "colorMode": {
+ "type": "categorical"
+ },
+ "paletteId": "eui_amsterdam_color_blind",
+ "specialAssignments": [
+ {
+ "color": {
+ "type": "loop"
+ },
+ "rule": {
+ "type": "other"
+ },
+ "touched": false
+ }
+ ]
+ },
+ "layerId": "3ca66f4c-d42b-437c-8e30-6dde3e6ef3c5",
+ "layerType": "data",
+ "legendDisplay": "default",
+ "metrics": [
+ "e1062427-71b6-4866-a836-8c6482598d72"
+ ],
+ "nestedLegend": false,
+ "numberDisplay": "percent",
+ "primaryGroups": [
+ "98f752b2-4502-4e16-b6d7-8d276bc539e5"
+ ]
+ }
+ ],
+ "shape": "pie"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsPie"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 14,
+ "i": "64ca5760-22da-4090-9868-83c6abe30952",
+ "w": 25,
+ "x": 23,
+ "y": 0
+ },
+ "panelIndex": "64ca5760-22da-4090-9868-83c6abe30952",
+ "title": "Occurrence by Company",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "indexpattern-datasource-layer-45a7032e-e1be-4c23-ac24-96d5daf4d53d",
+ "type": "index-pattern"
+ }
+ ],
+ "state": {
+ "adHocDataViews": {},
+ "datasourceStates": {
+ "formBased": {
+ "currentIndexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "layers": {
+ "45a7032e-e1be-4c23-ac24-96d5daf4d53d": {
+ "columnOrder": [
+ "7b6c4138-e8ea-49cc-b34b-15fc95a782af",
+ "4d9f32af-becf-4133-9293-2193a289e9f2",
+ "0761e882-612a-41f4-a1ff-bbac828bf9b1"
+ ],
+ "columns": {
+ "0761e882-612a-41f4-a1ff-bbac828bf9b1": {
+ "dataType": "number",
+ "isBucketed": false,
+ "label": "Count of records",
+ "operationType": "count",
+ "params": {
+ "emptyAsNull": true
+ },
+ "scale": "ratio",
+ "sourceField": "___records___"
+ },
+ "4d9f32af-becf-4133-9293-2193a289e9f2": {
+ "customLabel": true,
+ "dataType": "date",
+ "isBucketed": true,
+ "label": "Severity",
+ "operationType": "date_histogram",
+ "params": {
+ "dropPartials": false,
+ "includeEmptyRows": false,
+ "interval": "auto"
+ },
+ "scale": "interval",
+ "sourceField": "@timestamp"
+ },
+ "7b6c4138-e8ea-49cc-b34b-15fc95a782af": {
+ "dataType": "string",
+ "isBucketed": true,
+ "label": "Top 10 values of dataminr_pulse.categories.name",
+ "operationType": "terms",
+ "params": {
+ "exclude": [],
+ "excludeIsRegex": false,
+ "include": [],
+ "includeIsRegex": false,
+ "missingBucket": false,
+ "orderBy": {
+ "columnId": "0761e882-612a-41f4-a1ff-bbac828bf9b1",
+ "type": "column"
+ },
+ "orderDirection": "desc",
+ "otherBucket": true,
+ "parentFormat": {
+ "id": "terms"
+ },
+ "size": 10
+ },
+ "scale": "ordinal",
+ "sourceField": "dataminr_pulse.categories.name"
+ }
+ },
+ "ignoreGlobalFilters": false,
+ "incompleteColumns": {},
+ "indexPatternId": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "sampling": 1
+ }
+ }
+ },
+ "indexpattern": {
+ "layers": {}
+ },
+ "textBased": {
+ "layers": {}
+ }
+ },
+ "filters": [],
+ "internalReferences": [],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ },
+ "visualization": {
+ "axisTitlesVisibilitySettings": {
+ "x": true,
+ "yLeft": true,
+ "yRight": true
+ },
+ "fittingFunction": "None",
+ "gridlinesVisibilitySettings": {
+ "x": false,
+ "yLeft": true,
+ "yRight": true
+ },
+ "hideEndzones": false,
+ "labelsOrientation": {
+ "x": 0,
+ "yLeft": 0,
+ "yRight": 0
+ },
+ "layers": [
+ {
+ "accessors": [
+ "0761e882-612a-41f4-a1ff-bbac828bf9b1"
+ ],
+ "colorMapping": {
+ "assignments": [],
+ "colorMode": {
+ "type": "categorical"
+ },
+ "paletteId": "eui_amsterdam_color_blind",
+ "specialAssignments": [
+ {
+ "color": {
+ "type": "loop"
+ },
+ "rule": {
+ "type": "other"
+ },
+ "touched": false
+ }
+ ]
+ },
+ "layerId": "45a7032e-e1be-4c23-ac24-96d5daf4d53d",
+ "layerType": "data",
+ "position": "top",
+ "seriesType": "line",
+ "showGridlines": false,
+ "splitAccessor": "7b6c4138-e8ea-49cc-b34b-15fc95a782af",
+ "xAccessor": "4d9f32af-becf-4133-9293-2193a289e9f2"
+ }
+ ],
+ "legend": {
+ "isVisible": true,
+ "position": "right"
+ },
+ "preferredSeriesType": "line",
+ "showCurrentTimeMarker": true,
+ "tickLabelsVisibilitySettings": {
+ "x": true,
+ "yLeft": true,
+ "yRight": true
+ },
+ "valueLabels": "hide",
+ "yLeftExtent": {
+ "mode": "full"
+ },
+ "yLeftScale": "linear"
+ }
+ },
+ "title": "",
+ "type": "lens",
+ "visualizationType": "lnsXY"
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 15,
+ "i": "b7420b4e-91d1-47e4-b949-948079e4b82e",
+ "w": 23,
+ "x": 0,
+ "y": 14
+ },
+ "panelIndex": "b7420b4e-91d1-47e4-b949-948079e4b82e",
+ "title": "Occurence Trend",
+ "type": "lens"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "description": "",
+ "layerListJSON": [
+ {
+ "alpha": 1,
+ "id": "a3b4557b-60df-41db-9460-69d10214b2a7",
+ "includeInFitToBounds": true,
+ "label": null,
+ "locale": "autoselect",
+ "maxZoom": 24,
+ "minZoom": 0,
+ "sourceDescriptor": {
+ "isAutoSelect": true,
+ "lightModeDefault": "road_map_desaturated",
+ "type": "EMS_TMS"
+ },
+ "style": {
+ "color": "",
+ "type": "EMS_VECTOR_TILE"
+ },
+ "type": "EMS_VECTOR_TILE",
+ "visible": true
+ },
+ {
+ "id": "a3f884f3-b144-4abb-a2ec-6653885f6d6c",
+ "sourceDescriptor": {
+ "geoField": "geo.location",
+ "id": "db3e5a54-c45b-4bf2-8bdb-786ba0bea518",
+ "indexPatternRefName": "layer_1_source_index_pattern",
+ "requestType": "heatmap",
+ "resolution": "SUPER_FINE",
+ "metrics": [
+ {
+ "type": "count"
+ }
+ ],
+ "type": "ES_GEO_GRID",
+ "applyGlobalQuery": true,
+ "applyGlobalTime": true,
+ "applyForceRefresh": true
+ },
+ "style": {
+ "type": "HEATMAP",
+ "colorRampName": "theclassic"
+ },
+ "type": "HEATMAP",
+ "visible": true,
+ "alpha": 0.75,
+ "minZoom": 0,
+ "maxZoom": 24,
+ "includeInFitToBounds": true
+ }
+ ],
+ "mapStateJSON": {
+ "adHocDataViews": [],
+ "center": {
+ "lat": 19.94277,
+ "lon": 0
+ },
+ "filters": [],
+ "query": {
+ "language": "kuery",
+ "query": ""
+ },
+ "refreshConfig": {
+ "interval": 60000,
+ "isPaused": true
+ },
+ "settings": {
+ "autoFitToDataBounds": true,
+ "backgroundColor": "transparent",
+ "browserLocation": {
+ "zoom": 2
+ },
+ "customIcons": [],
+ "disableInteractive": false,
+ "disableTooltipControl": false,
+ "fixedLocation": {
+ "lat": 0,
+ "lon": 0,
+ "zoom": 2
+ },
+ "hideLayerControl": false,
+ "hideToolbarOverlay": false,
+ "hideViewControl": false,
+ "initialLocation": "LAST_SAVED_LOCATION",
+ "keydownScrollZoom": false,
+ "maxZoom": 24,
+ "minZoom": 0,
+ "showScaleControl": false,
+ "showSpatialFilters": true,
+ "showTimesliderToggleButton": true,
+ "spatialFiltersAlpa": 0.3,
+ "spatialFiltersFillColor": "#DA8B45",
+ "spatialFiltersLineColor": "#DA8B45"
+ },
+ "timeFilters": {
+ "from": "now-15m",
+ "to": "now"
+ },
+ "zoom": 1.57
+ },
+ "title": "Distribution by location",
+ "uiStateJSON": {
+ "isLayerTOCOpen": true,
+ "openTOCDetails": []
+ },
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "layer_1_source_index_pattern",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {
+ "dynamicActions": {
+ "events": []
+ }
+ },
+ "hiddenLayers": [],
+ "isLayerTOCOpen": true,
+ "mapBuffer": {
+ "maxLat": 66.51326,
+ "maxLon": 180,
+ "minLat": -66.51326,
+ "minLon": -180
+ },
+ "mapCenter": {
+ "lat": 19.94277,
+ "lon": 0,
+ "zoom": 1.57
+ },
+ "openTOCDetails": []
+ },
+ "gridData": {
+ "h": 15,
+ "i": "6be7067a-067e-400c-b008-62820b3892ed",
+ "w": 25,
+ "x": 23,
+ "y": 14
+ },
+ "panelIndex": "6be7067a-067e-400c-b008-62820b3892ed",
+ "title": "Distribution by location",
+ "type": "map"
+ },
+ {
+ "embeddableConfig": {
+ "attributes": {
+ "columns": [
+ "event.severity_label",
+ "event.created",
+ "message",
+ "dataminr_pulse.companies.name_joined",
+ "dataminr_pulse.source.channels",
+ "dataminr_pulse.categories.name_joined",
+ "dataminr_pulse.watchlists_matched_by_type.name_joined",
+ "dataminr_pulse.sectors.name_joined"
+ ],
+ "description": "",
+ "grid": {},
+ "hideChart": true,
+ "isTextBasedQuery": false,
+ "kibanaSavedObjectMeta": {
+ "searchSourceJSON": {
+ "filter": [],
+ "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "query": {
+ "language": "kuery",
+ "query": "event.severity_label : * "
+ }
+ }
+ },
+ "sampleSize": 250,
+ "sort": [
+ [
+ "@timestamp",
+ "desc"
+ ]
+ ],
+ "timeRestore": false,
+ "title": "Latest 250 Alerts",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ]
+ },
+ "enhancements": {}
+ },
+ "gridData": {
+ "h": 13,
+ "i": "71b2761c-9199-46e4-8269-807e584e7ddf",
+ "w": 48,
+ "x": 0,
+ "y": 29
+ },
+ "panelIndex": "71b2761c-9199-46e4-8269-807e584e7ddf",
+ "title": "Latest 250 Alerts",
+ "type": "search"
+ }
+ ],
+ "timeRestore": true,
+ "timeFrom": "now-7d",
+ "timeTo": "now",
+ "title": "Dataminr Pulse - Alert Overview Dashboard",
+ "version": 2
+ },
+ "coreMigrationVersion": "8.8.0",
+ "created_at": "2026-01-28T14:04:19.306Z",
+ "created_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
+ "id": "dataminr_pulse-3364fad8-deb4-40cf-9dfe-50bf4d5a1824",
+ "references": [
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "13787329-329d-4c53-ac30-e4dd78fa8c25:indexpattern-datasource-layer-56bf765b-3d20-4830-b193-e2d9c56b1d5c",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "64ca5760-22da-4090-9868-83c6abe30952:indexpattern-datasource-layer-3ca66f4c-d42b-437c-8e30-6dde3e6ef3c5",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "b7420b4e-91d1-47e4-b949-948079e4b82e:indexpattern-datasource-layer-45a7032e-e1be-4c23-ac24-96d5daf4d53d",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "6be7067a-067e-400c-b008-62820b3892ed:layer_1_source_index_pattern",
+ "type": "index-pattern"
+ },
+ {
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "name": "71b2761c-9199-46e4-8269-807e584e7ddf:kibanaSavedObjectMeta.searchSourceJSON.index",
+ "type": "index-pattern"
+ }
+ ],
+ "type": "dashboard",
+ "typeMigrationVersion": "10.2.0",
+ "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0"
+}
\ No newline at end of file
diff --git a/packages/dataminr_pulse/kibana/index_pattern/eb1259c1-a4fb-43c5-a993-dc31b973c044.json b/packages/dataminr_pulse/kibana/index_pattern/eb1259c1-a4fb-43c5-a993-dc31b973c044.json
new file mode 100644
index 00000000000..a2f44916e60
--- /dev/null
+++ b/packages/dataminr_pulse/kibana/index_pattern/eb1259c1-a4fb-43c5-a993-dc31b973c044.json
@@ -0,0 +1,22 @@
+{
+ "attributes": {
+ "allowHidden": false,
+ "fieldAttrs": "{\"@timestamp\":{\"count\":1,\"customLabel\":\"Timestamp\"},\"event.created\":{\"count\":1,\"customLabel\":\"Event Time\"},\"message\":{\"count\":1,\"customLabel\":\"Caption\"},\"dataminr_pulse.source.channels\":{\"count\":1,\"customLabel\":\"Source Channels\"},\"dataminr_pulse.sectors.name_joined\":{\"count\":1,\"customLabel\":\"Sector\"},\"dataminr_pulse.companies.name_joined\":{\"count\":1,\"customLabel\":\"Company\"},\"dataminr_pulse.categories.name_joined\":{\"count\":1,\"customLabel\":\"Selected Topics\"},\"event.severity_label\":{\"count\":1,\"customLabel\":\"Alert Type\"},\"dataminr_pulse.watchlists_matched_by_type.name_joined\":{\"count\":1,\"customLabel\":\"Lists Matched\"}}",
+ "fields": "[]",
+ "name": "logs-dataminr_pulse.alerts-default",
+ "runtimeFieldMap": "{\"dataminr_pulse.companies.name_joined\":{\"type\":\"keyword\",\"script\":{\"source\":\"if (doc['dataminr_pulse.companies.name'].size() > 0) { emit(String.join(';\\n', doc['dataminr_pulse.companies.name'])); } else { emit('-'); }\"}},\"dataminr_pulse.categories.name_joined\":{\"type\":\"keyword\",\"script\":{\"source\":\"if (doc['dataminr_pulse.categories.name'].size() > 0) { emit(String.join(';\\n', doc['dataminr_pulse.categories.name'])); } else { emit('-'); }\"}},\"dataminr_pulse.sectors.name_joined\":{\"type\":\"keyword\",\"script\":{\"source\":\"if (doc['dataminr_pulse.sectors.name'].size() > 0) { emit(String.join(';\\n', doc['dataminr_pulse.sectors.name'])); } else { emit('-'); }\"}},\"dataminr_pulse.watchlists_matched_by_type.name_joined\":{\"type\":\"keyword\",\"script\":{\"source\":\"if (doc['dataminr_pulse.watchlists_matched_by_type.name'].size() > 0) { emit(String.join(';\\n', doc['dataminr_pulse.watchlists_matched_by_type.name'])); } else { emit('-'); }\"}},\"geo.location\":{\"type\":\"geo_point\",\"script\":{\"source\":\"if (doc['geo.location.lat'].size() > 0 && doc['geo.location.lon'].size() > 0) { def lat = doc['geo.location.lat'].value; def lon = doc['geo.location.lon'].value; emit(lat, lon); }\"}},\"event.severity_label\":{\"type\":\"keyword\",\"script\":{\"source\":\"if (doc['event.severity'].size() > 0) { def severity = doc['event.severity'].value; if (severity == 30) { emit('Alert'); } else if (severity == 50) { emit('Urgent'); } else if (severity == 90) { emit('Flash'); } else { emit('Unknown'); } }\"}}}",
+ "timeFieldName": "@timestamp",
+ "title": "logs-dataminr_pulse.alerts-*"
+ },
+ "coreMigrationVersion": "8.8.0",
+ "created_at": "2026-01-16T14:54:59.726Z",
+ "created_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
+ "id": "eb1259c1-a4fb-43c5-a993-dc31b973c044",
+ "managed": false,
+ "references": [],
+ "type": "index-pattern",
+ "typeMigrationVersion": "8.0.0",
+ "updated_at": "2026-01-20T10:50:53.461Z",
+ "updated_by": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0",
+ "version": "WzcxNSwxXQ=="
+}
diff --git a/packages/dataminr_pulse/manifest.yml b/packages/dataminr_pulse/manifest.yml
new file mode 100644
index 00000000000..43cae92e1b8
--- /dev/null
+++ b/packages/dataminr_pulse/manifest.yml
@@ -0,0 +1,30 @@
+format_version: 3.0.3
+name: dataminr_pulse
+title: Dataminr Pulse
+version: 1.0.0
+description: Collect real-time alerts from Dataminr Pulse API
+icons:
+ - src: /img/Dataminr_Logo-Mark_Full-Color_RGB.svg
+ title: Dataminr Pulse Logo
+ size: any
+ type: image/svg+xml
+type: integration
+categories:
+ - security
+ - threat_intel
+conditions:
+ kibana:
+ version: "^8.13.0"
+ elastic:
+ subscription: basic
+policy_templates:
+ - name: dataminr_pulse
+ title: Dataminr Pulse Alerts
+ description: Collect real-time alerts from Dataminr Pulse API
+ inputs:
+ - type: cel
+ title: Dataminr Pulse Alerts via CEL
+ description: Collect alerts using Common Expression Language (CEL) input
+owner:
+ github: elastic/security-external-integrations
+ type: elastic
\ No newline at end of file
diff --git a/packages/dataminr_pulse/scripts/setup-fleet.sh b/packages/dataminr_pulse/scripts/setup-fleet.sh
new file mode 100755
index 00000000000..d1362d8be89
--- /dev/null
+++ b/packages/dataminr_pulse/scripts/setup-fleet.sh
@@ -0,0 +1,96 @@
+#!/bin/bash
+set -euo pipefail
+
+# Configuration
+ES_URL="${ES_URL:-http://localhost:9200}"
+KIBANA_URL="${KIBANA_URL:-http://localhost:5601}"
+ELASTIC_PASSWORD="${ELASTIC_PASSWORD:-changeme}"
+KIBANA_PASSWORD="${KIBANA_PASSWORD:-changeme}"
+PACKAGE_DIR="$(cd "$(dirname "$0")/.." && pwd)"
+
+echo "=== Dataminr Pulse Integration - Fleet Setup ==="
+echo ""
+
+# Step 1: Wait for Elasticsearch
+echo "[1/6] Waiting for Elasticsearch..."
+until curl -s -u "elastic:${ELASTIC_PASSWORD}" "${ES_URL}/_cluster/health" | grep -qE '"status":"(green|yellow)"'; do
+ sleep 5
+done
+echo " Elasticsearch is ready."
+
+# Step 2: Set kibana_system password
+echo "[2/6] Setting kibana_system password..."
+curl -s -u "elastic:${ELASTIC_PASSWORD}" \
+ -X POST "${ES_URL}/_security/user/kibana_system/_password" \
+ -H "Content-Type: application/json" \
+ -d "{\"password\":\"${KIBANA_PASSWORD}\"}" > /dev/null
+echo " kibana_system password set."
+
+# Step 3: Wait for Kibana
+echo "[3/6] Waiting for Kibana..."
+until curl -s -u "elastic:${ELASTIC_PASSWORD}" "${KIBANA_URL}/api/status" | grep -q '"overall"'; do
+ sleep 5
+done
+echo " Kibana is ready."
+
+# Step 4: Wait for Fleet Server
+echo "[4/6] Waiting for Fleet Server..."
+until curl -s -u "elastic:${ELASTIC_PASSWORD}" \
+ "${KIBANA_URL}/api/fleet/agents" \
+ -H "kbn-xsrf: true" 2>/dev/null | grep -q '"total"'; do
+ sleep 5
+done
+echo " Fleet is ready."
+
+# Step 5: Install the custom integration package
+echo "[5/6] Installing Dataminr Pulse integration package..."
+
+if command -v elastic-package &> /dev/null; then
+ echo " Building package with elastic-package..."
+ cd "${PACKAGE_DIR}"
+ elastic-package build
+
+ PACKAGE_ZIP=$(find build -name "*.zip" -type f 2>/dev/null | head -1)
+
+ if [ -n "${PACKAGE_ZIP}" ]; then
+ echo " Uploading package: ${PACKAGE_ZIP}"
+ UPLOAD_RESULT=$(curl -s -u "elastic:${ELASTIC_PASSWORD}" \
+ -X POST "${KIBANA_URL}/api/fleet/epm/packages" \
+ -H "kbn-xsrf: true" \
+ -H "Content-Type: application/zip" \
+ --data-binary "@${PACKAGE_ZIP}")
+ echo " ${UPLOAD_RESULT}" | python3 -m json.tool 2>/dev/null || echo " ${UPLOAD_RESULT}"
+ else
+ echo " Warning: No package zip found in build/ directory."
+ fi
+else
+ echo " elastic-package CLI not found. Install the package manually:"
+ echo " Option A: Install elastic-package and re-run this script"
+ echo " Option B: Upload via Kibana UI: Fleet > Integrations > Upload integration"
+fi
+
+# Step 6: List enrollment tokens
+echo ""
+echo "[6/6] Enrollment tokens:"
+TOKENS=$(curl -s -u "elastic:${ELASTIC_PASSWORD}" \
+ "${KIBANA_URL}/api/fleet/enrollment_api_keys" \
+ -H "kbn-xsrf: true")
+echo "${TOKENS}" | python3 -m json.tool 2>/dev/null || echo "${TOKENS}"
+
+echo ""
+echo "=== Setup Complete ==="
+echo ""
+echo "Kibana: ${KIBANA_URL}"
+echo "Login: elastic / ${ELASTIC_PASSWORD}"
+echo ""
+echo "To configure the Dataminr Pulse integration:"
+echo " 1. Go to Fleet > Agent Policies"
+echo " 2. Select the default policy (or create a new one)"
+echo " 3. Add integration > Dataminr Pulse"
+echo ""
+echo "For testing with mock API (start with: docker compose --profile mock up -d):"
+echo " URL: http://mock-dataminr-api:8080"
+echo " Base URL: /pulse/v1"
+echo " Auth URL: http://mock-dataminr-api:8080/auth/2/token"
+echo " Client ID: test-client-id"
+echo " Client Secret: test-client-secret"