Skip to content

[Watchguard]: Pipeline failure on traffic logs #17908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
qcorporation wants to merge 3 commits into
base: main
Choose a base branch
from
+114 −3
Open

[Watchguard]: Pipeline failure on traffic logs #17908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b284a04
[Watchguard]: Pipeline failure on traffic logs
qcorporation Mar 19, 2026
97fdb5b
Update changelog.yml
qcorporation Mar 19, 2026
9b1d2d8
Merge branch 'main' into 17863-watchguard
qcorporation Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions packages/watchguard_firebox/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.6.2"
changes:
- description: Fix pipeline failure on traffic logs for ESP (non-TCP/UDP) sessions by adding null checks for source/destination port and IP before running the community_id processor.
type: bugfix
link: https://github.com/elastic/integrations/pull/17908
- version: "1.6.1"
changes:
- description: Fix Grok Pattern for Watchguard Member Info.
Expand Down
3 changes: 2 additions & 1 deletion packages/watchguard_firebox/data_stream/log/_dev/test/pipeline/test-traffic.log
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,4 +216,5 @@
<142>May 12 15:19:05 WatchGuard-Firebox FVE6035FD3AE3 Member2 (2024-05-10T09:49:05) firewall[10]: msg_id="2DFF-0006" Allow 1-Trusted 0-External tcp 10.0.1.2 67.43.156.0 60180 23 msg="ProxyAllow: IP DNSWatch blackholed domain" proxy_act="TCPUDP-Proxy.Standard.1" Protocol="telnet" geo_dst="USA" (TCP-UDPproxy-00)
<142>May 12 15:19:05 WatchGuard-Firebox FVE6035FD3AE3 Member2 (2024-05-10T09:49:05) firewall[10]: msg_id="2DFF-0007" Deny 1-Trusted 0-External tcp 10.0.1.2 67.43.156.0 60180 23 msg="ProxyAllow: IP DNSWatch content filtered domain" proxy_act="TCP-UDP-Proxy.Standard.1" Protocol="telnet" geo_dst="USA" (TCP-UDP-proxy-002)
<142>May 12 15:19:05 WatchGuard-Firebox FVE6035FD3AE3 Member2 (2024-05-10T09:49:05) firewall[10]: msg_id="3000-0148" Deny WAN-SAMPLE WAN-SAMPLE 40 tcp 20 247 1.128.0.0 67.43.156.1 0 5357 offset 5 S 2680103568 win 4 flags="SR" duration="0" sent_pkts="1" rcvd_pkts="0" sent_bytes="40" rcvd_bytes="0" (Unhandled External Packet-00)
<142>Feb 4 11:04:49 Member1 FVE6035FD3AE3 Watchguard_MemberOne (2026-02-04T10:04:49) firewall: msg_id="3000-0148" Allow DMZ LAN 60 tcp 20 63 10.0.1.2 10.0.1.3 55118 3306 offset 10 S 3052065981 win 61690  (TCP-UDP DMZ 2 LAN-00)
<142>Feb 4 11:04:49 Member1 FVE6035FD3AE3 Watchguard_MemberOne (2026-02-04T10:04:49) firewall: msg_id="3000-0148" Allow DMZ LAN 60 tcp 20 63 10.0.1.2 10.0.1.3 55118 3306 offset 10 S 3052065981 win 61690  (TCP-UDP DMZ 2 LAN-00)
<142>Mar 17 14:20:26 Firebox_M670 (2026-03-17T20:20:26) firewall: msg_id="3000-0151" Allow Firebox External_UPN esp 10.0.1.2 10.0.1.3 geo_src="USA" geo_dst="USA" duration="697" sent_pkts="2" rcvd_pkts="67635" sent_bytes="1600" rcvd_bytes="7220344" (Any From Firebox-00)
96 changes: 96 additions & 0 deletions ...ages/watchguard_firebox/data_stream/log/_dev/test/pipeline/test-traffic.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25731,6 +25731,102 @@
"window_size": 61690
}
}
},
{
"@timestamp": "2026-03-17T14:20:26.000+05:30",
"destination": {
"bytes": 1600,
"ip": "10.0.1.3",
"packets": 2
},
"ecs": {
"version": "8.17.0"
},
"event": {
"category": [
"network"
],
"duration": 697000000000,
"kind": "event",
"original": "<142>Mar 17 14:20:26 Firebox_M670 (2026-03-17T20:20:26) firewall: msg_id=\"3000-0151\" Allow Firebox External_UPN esp 10.0.1.2 10.0.1.3 geo_src=\"USA\" geo_dst=\"USA\" duration=\"697\" sent_pkts=\"2\" rcvd_pkts=\"67635\" sent_bytes=\"1600\" rcvd_bytes=\"7220344\" (Any From Firebox-00)",
"outcome": "success",
"timezone": "+05:30",
"type": [
"end"
]
},
"log": {
"syslog": {
"appname": "firewall",
"hostname": "Firebox_M670",
"priority": 142
}
},
"network": {
"bytes": 7221944,
"transport": "esp"
},
"observer": {
"egress": {
"interface": {
"alias": "External_UPN"
}
},
"hostname": "Firebox_M670",
"ingress": {
"interface": {
"alias": "Firebox"
}
},
"product": "Firebox",
"type": "firewall",
"vendor": "WatchGuard"
},
"related": {
"hosts": [
"Firebox_M670"
],
"ip": [
"10.0.1.2",
"10.0.1.3"
]
},
"rule": {
"name": [
"Any From Firebox-00"
]
},
"source": {
"bytes": 7220344,
"ip": "10.0.1.2",
"packets": 67635
},
"tags": [
"preserve_original_event",
"preserve_duplicate_custom_fields"
],
"watchguard_firebox": {
"log": {
"bytes_in": 7220344,
"bytes_out": 1600,
"destination_ip": "10.0.1.3",
"disposition": "Allow",
"duration": 697,
"geo_destination": "USA",
"geo_src": "USA",
"in_interface_name": "Firebox",
"log_type": "traffic",
"msg_id": "3000-0151",
"out_interface_name": "External_UPN",
"packets_in": 67635,
"packets_out": 2,
"policy_name": "Any From Firebox-00",
"source_ip": "10.0.1.2",
"syslog_timestamp": "2026-03-17T14:20:26.000+05:30",
"timestamp": "2026-03-17T20:20:26.000Z",
"transport": "esp"
}
}
}
]
}
8 changes: 7 additions & 1 deletion ...ges/watchguard_firebox/data_stream/log/elasticsearch/ingest_pipeline/pipeline_traffic.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1145,7 +1145,13 @@ processors:
value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'
- community_id:
tag: src_dst_community_id
if: 'ctx.watchguard_firebox.log.source_port != 0 && ctx.watchguard_firebox.log.destination_port != 0'
if: >-
ctx.watchguard_firebox?.log?.source_ip != null
&& ctx.watchguard_firebox?.log?.destination_ip != null
&& ctx.watchguard_firebox?.log?.source_port != null
&& ctx.watchguard_firebox.log.source_port != 0
&& ctx.watchguard_firebox?.log?.destination_port != null
&& ctx.watchguard_firebox.log.destination_port != 0
Comment on lines +1150 to +1154
Copy link
Contributor

@taylor-swanson taylor-swanson Mar 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remaining ? are probably redundant and can be removed.

Also, it's odd to me that we are running it against the vendor fields and not the ECS fields...

source_ip: watchguard_firebox.log.source_ip
source_port: watchguard_firebox.log.source_port
destination_ip: watchguard_firebox.log.destination_ip
Expand Down
2 changes: 2 additions & 0 deletions packages/watchguard_firebox/data_stream/log/fields/fields.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,8 @@
type: keyword
- name: geo_destination
type: keyword
- name: geo_src
type: keyword
- name: group_name
type: keyword
- name: header
Expand Down
1 change: 1 addition & 0 deletions packages/watchguard_firebox/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ The `log` data stream provides events from WatchGuard Firebox of the following t
| watchguard_firebox.log.gateway | | keyword |
| watchguard_firebox.log.gateway_endpoint | | keyword |
| watchguard_firebox.log.geo_destination | | keyword |
| watchguard_firebox.log.geo_src | | keyword |
| watchguard_firebox.log.group_name | | keyword |
| watchguard_firebox.log.header | | keyword |
| watchguard_firebox.log.headers_size | | long |
Expand Down
2 changes: 1 addition & 1 deletion packages/watchguard_firebox/manifest.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.1.4
name: watchguard_firebox
title: WatchGuard Firebox
version: "1.6.1"
version: "1.6.2"
description: Collect logs from WatchGuard Firebox with Elastic Agent.
type: integration
categories:
Expand Down
Loading