Today I’m rebuilding my ELK/ntopng setup within the Proxmox cluster. I used to run it entirely as Docker containers under Rancher 1.6, but with Rancher 2.0 I found the Elasticsearch requirements to be too heavy. As soon as I fired up the pods under Kubernetes, I/O to the NAS for the iSCSI volumes went through the roof, and other services suffered. To mitigate this I’m now running both ES nodes directly on the bulky Proxmox hosts (4 CPU / 32GB RAM) as LXC containers.

This brings us back to ntopng. The pro version would allow transparent bridging, and I could insert it directly between the internal network (physical port) and the pfSense VM. Alas, I’m not running the pro version, so I’m left with port mirroring.

No problem. I created a new bridge (vmbr1) in proxmox and added a new port on this bridge to pfSense. I configured a bridge for the LAN interface and set the new port as a SPAN port. A quick check with tcpdump showed that it is correctly spewing a copy of all traffic that comes in on the LAN port.

I fired up a new container in Proxmox that was bound to the main bridge and to the span bridge. This container should receive the mirrored traffic on span0, run it through ntopng, and then send it off to ES or Logstash or InfluxDB or wherever.

Except it doesn’t. It only shows broadcast and multicast traffic. What gives? Thinking it was an LXC problem I rebuilt it as a KVM VM. It still didn’t pass the traffic.

I looked all over the place before finding this post that implies there is a configuration option that I can pass to KVM to make it work. This would explain the KVM part but not the LXC part. It seems more like it’s a problem with the bridge itself.

Dig dig dig…dig dig dig…and I found this page for a Snort configuration that looked similar to what I was trying to do. I tried what he suggested, and it worked. Since it was a configuration option on the Proxmox host itself, I checked if the LXC containers also work. They do.

So, in summary, this will get you promiscuous traffic forwarded to other hosts on the bridge:

brctl setageing vmbr1 0
brctl setfd vmbr1 0

Ansible now manages /etc/network/if-up.d/vmbr1-mirror with these commands, and we’re once again moving forward with ntopng.