Skip to content

Add IPFIX support, including OpenBSD pflow(4) #162

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
drkhsh wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add IPFIX support, including OpenBSD pflow(4) #162

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 20 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# netflow2ng
NetFlow v9 collector for [ntopng](https://www.ntop.org/products/traffic-analysis/ntop/)
NetFlow v9 / IPFIX collector for [ntopng](https://www.ntop.org/products/traffic-analysis/ntop/)

[![Tests](https://github.com/synfinatic/netflow2ng/actions/workflows/tests.yml/badge.svg)](https://github.com/synfinatic/netflow2ng/actions/workflows/tests.yml)
[![codeql-analysis.yml](https://github.com/synfinatic/netflow2ng/actions/workflows/codeql-analysis.yml/badge.svg)](https://github.com/synfinatic/netflow2ng/actions/workflows/codeql-analysis.yml)
Expand All @@ -13,7 +13,8 @@ data and load that into ntopng you currently have no choice but to spend
199Euro on [nProbe](https://www.ntop.org/products/netflow/nprobe/) which
in my case is more expensive than the
[Ubiquiti USG](https://www.ui.com/unifi-routing/usg/) that I wanted to
collect NetFlow stats from.
collect NetFlow stats from. The same is true for IPFIX exporters, including
OpenBSD's [pflow(4)](https://man.openbsd.org/pflow.4) with `pflowproto 10`.

Hence, I created netflow2ng.

Expand Down Expand Up @@ -41,17 +42,18 @@ Hence, I created netflow2ng.
1. For a list of configuration arguments, run `netflow2ng -h`. As of v0.1.1, netflow2ng
defaults to the ntopng TLV format instead of JSON. If you want to use JSON, you must
use it with `ntopng` v6.3 or earlier.
1. Configure your network device(s) to send NetFlow stats to netflow2ng
1. Configure your network device(s) to send NetFlow or IPFIX stats to netflow2ng.
For OpenBSD: `ifconfig pflow0 flowsrc <bsd-ip> flowdst <netflow2ng-ip>:2055 pflowproto 10`.
1. Configure your [ntopng](https://www.ntop.org/products/traffic-analysis/ntop/)
service to read from netflow2ng: `ntopng -i tcp://192.168.1.1:5556` where
"192.168.1.1" is the IP address of your netflow2ng server.

### Features

* Collect NetFlow v9 stats from one or more probes
* Collect NetFlow v9 and IPFIX stats from one or more probes (incl. OpenBSD `pflow(4)`)
* Run a ZMQ Publisher for ntopng to collect metrics from
* Prometheus metrics
* NetFlow Templates available via /templates HTTP endpoint
* NetFlow / IPFIX templates available via /templates HTTP endpoint

### Ports

Expand All @@ -60,16 +62,23 @@ By default, netflow2ng listens on all addresses on the following ports. This can
* ZMQ connections (TCP): 5556
* Metrics: 8080

### NetFlow v9 Support
### NetFlow v9 / IPFIX Support

netflow2ng utilizes [goflow2](https://github.com/netsampler/goflow2) for NetFlow
decoding. For more information on what NetFlow fields are supported in
netflow2ng, please read the goflow docs.
and IPFIX decoding. The same UDP listener accepts NetFlow v9 (version 9) and IPFIX
(version 10) packets — goflow2 dispatches by the version field in the packet
header, so no per-protocol configuration is required. For more information on what
fields are supported, please read the goflow docs.

### sFlow/IPFIX/etc support?
OpenBSD's [pflow(4)](https://man.openbsd.org/pflow.4) `pflowproto 10` exporter is
known to work; it emits unidirectional flows so only the inbound byte/packet
counters are populated per record. Bidirectional IPFIX exporters additionally
populate the outbound counters (postOctetDeltaCount / postPacketDeltaCount).

In theory, adding sFlow/IPFIX/NetFlow v5 support should be pretty trivial, but
isn't something I plan on doing due to lack of hardware for testing/need.
### sFlow / NetFlow v5 support?

Not currently supported. Both would require additional decoder wiring and
producer-side changes; this codebase only implements the NetFlow v9 / IPFIX path.

### How is netflow2ng different from nProbe?

Expand Down
13 changes: 13 additions & 0 deletions formatter/mapping.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,16 @@ netflowv9:
destination: out_bytes
- field: 24
destination: out_packets
# IPFIX uses the same IANA Information Element IDs for these counters as NetFlow v9.
# OpenBSD pflow(4) pflowproto 10 is unidirectional and only sets fields 1 and 2;
# bidirectional exporters additionally set 23 and 24.
ipfix:
mapping:
- field: 1 # octetDeltaCount
destination: in_bytes
- field: 2 # packetDeltaCount
destination: in_packets
- field: 23 # postOctetDeltaCount
destination: out_bytes
- field: 24 # postPacketDeltaCount
destination: out_packets