Nfdum Y Nfsen

nfdump and NfSen
18th Annual FIRST Conference June 25-30, 2006 Baltimore Peter Haag
2006 © SWITCH

nfdump and NfSen
Some operational questions, popping up now and then: • Do you see this peek on port 445 as well ? • What caused this peek on your network graph ? • How did SoberR spread in your network ? • Do we have any traffic pattern of this incident ? • Which host/subnet consumes mostof your bandwidth ? • Which are the top talkers in your network ? • … Sober.R

2006 © SWITCH

2

nfdump and NfSen
How to find answers for all these questions?

.. in discussions with other teams:
– “Watch your flows for …” – “I’ve seen a lot of … in our flows …” – “Hosts are infected, when you see flows to …”

2006 © SWITCH

3

nfdump and NfSen
What is NetFlow? NetFlow is atraffic monitoring technology developed by Cisco Networks. Flows are unidirectional and contain connection related data such as:
– – – – – – Source and destination IP address. Source and destination port. Source and destination AS. Level3 protocol, ToS byte, TCP flags. Logical input and output interfaces. Bytes and packet counters.
TCP 172.16.71.66:13599 -> 192.168.10.34:80 .A..SF 215 9890Example:
2006-03-30 00:47:33.728 54.971

2006 © SWITCH

4

nfdump and NfSen
How to get netflow data and how to look at them? Routers do provide netflow data … but …
Router# show ip cache flow

… seems not to be the solution for every task. ⇒ Tools to collect and look at the netflow data

2006 © SWITCH

5

nfdump and NfSen
nfdump and NfSen:

NfSen: • Web based frontend •Display flows • Framework to automate tasks

nfdump: • Collect and store flows • Process flows on command line
2006 © SWITCH 6

nfdump and NfSen
nfdump overview :
Text

nfcapd netflow v5, v7 or v0 exporter storage sfcapd sflow exporter nfdump

Binary
nfcapd.2006xx

Collecting data

Processing data

2006 © SWITCH

7

nfdump and NfSen
nfdump features:
• • • • • • • • • • • •CMD line based tool comparable to tcpdump. Written in C ⇒ fast. Stores netflow data in time sliced files. Supports netflow format v5,v7 and v9. Supports sflow. All processing options support IPv4 and IPv6. Powerful pcap like filter syntax: ( proto tcp and dst net 172.16/16 and src port > 1024 and bytes < 600 ) or ( bps > 1k and … Flexible flow aggregation:srcip,dstip,srcport,dstport,srcas,dstas,proto Efficient filter engine: > 6 Mio flows/s on 3GHz Intel. Lots of fast Top N statistics. Anonymizing of IP addresses. ( Crypto-Pan ) User defined output formats.
8

2006 © SWITCH

nfdump and NfSen
Example:
List the first 20 tcp flows:
forth% nfdump -r /data/rz/nfcapd.200603300150 -K 123.. -c 20 ‘proto tcp’ Date flow start Duration Proto Src IP Addr:Port Dst IP Addr:Port 2006-03-3000:43:40.569 82.880 TCP 130.20.234.125:58035 -> 200.66.27.5:61486 2006-03-30 00:43:40.569 82.880 TCP 200.66.27.5:61486 -> 130.20.234.125:58035 2006-03-30 00:44:00.082 63.113 TCP 130.20.234.125:55697 -> 159.93.88.3:60454 2006-03-30 00:44:00.082 63.113 TCP 159.93.88.3:60454 -> 130.20.234.125:55697 2006-03-30 00:45:02.647 0.431 TCP 193.246.238.35:80 -> 192.254.4.182:56547 2006-03-30 00:45:02.6470.431 TCP 192.254.4.182:56547 -> 193.246.238.35:80 2006-03-30 00:45:02.813 0.000 TCP 130.20.234.124:59112 -> 194.50.123.176:45458 2006-03-30 00:45:02.913 0.000 TCP 192.254.4.167:58659 -> 49.20.115.83:80 2006-03-30 00:45:02.913 0.000 TCP 129.66.105.181:11248 -> 192.254.4.183:80 2006-03-30 00:45:02.913 0.000 TCP 192.254.4.183:80 -> 129.66.105.181:11248 2006-03-30 00:45:02.879 0.000 TCP129.66.105.181:11247 -> 192.254.4.183:80 2006-03-30 00:45:02.879 0.000 TCP 192.254.4.183:80 -> 129.66.105.181:11247 2006-03-30 00:45:02.913 0.355 TCP 214.203.35.177:19027 -> 130.20.234.125:80 2006-03-30 01:40:02.347 300.572 TCP dffe:e6..:199:fd.119 -> dc7e:18..:fe99:2.35541 2006-03-30 01:40:02.347 300.572 TCP dc7e:18..:fe99:2.35541 -> dffe:e6..:199:fd.119 2006-03-30 00:45:02.895 0.000 TCP 192.254.4.183:80 ->...