Monday, April 29, 2013

Extreme Packet Capture, Star Trek Style (Part 2)


Our approach to technology defines who we are, as individuals and groups. The groups could be companies, countries or a species, regardless the technology we employ demonstrates our origins, and our roots. The Klingons are a fictional race of warriors and hunters who pride themselves on war ships with camouflage cloaking, strong defensive shields and superior maneuverability. In contrast the fictional Federation is collection of races whose focus is on exploration. Their star charts, scientific scanners, and fast charging photon based phasers offer a unique contrast to the Klingon's much slower & less efficient particle beam disrupters.

The same holds true for packet capture solutions. Our company, Myricom, designed our product in collaboration with a government agency interested in network security. One of the key design criteria was the replacement of an already existing method with one that bypassed the operating system so lossless packet capture at wire-rate could be achieved. The process of capturing network packets is transparent to the end user application, and they are stored in memory via a user space or kernel space driver. This technique enables our product to support over a dozen existing applications right out of the box. Another vendor designed their capture product for the financial market where saving the market data to disk for later analysis is critically important. Both of these approaches fit perfectly for the problems they solve, but one is more versatile. 

In part one I promised to wrap up this series by talking about injection and sample code, so let's begin. Injection is simply taking packets in memory and putting them onto the ethernet. Where FastStack Sniffer10G differentiates itself from other approaches is that it has total control over the network interface, with nothing between it and the wire. Therefore when you capture packets, you can modify the contents if you like, and then inject them back onto the wire without anyone being the wiser. Most security appliances do just this, they act as a man in the middle, a guard who looks at everything and only lets in, our out if you're really careful, things that are acceptable. Since Sniffer10G is an in-memory solution this can be sustained at wire-rate provided your man-in-the-middle code is pretty tight, and you leverage multiple queues for processing traffic in parallel. There is no transparent way to offer injection so you need to use the  Application Programming Interface (API), but several useful sample programs are included, with source.

The sample programs provided that do injection are: snf_pktgen, snf_replay, and snf_bridge. The snf_pktgen is just what it's name implies a simple packet generator. You can tell snf_pktgen what packet size to use, how many packets to send (or infinite) and the number of concurrent parallel threads to use to send them. It will make a best effort to pack the wire full of packets with the size you provided. Similar to that we have snf_replay which will play back a sequence of packets already constructed to the ethernet. Here you pass snf_replay the file name that contains the packets. An optional packet rate (undocumented for various reasons). Another option to read the whole packet file into memory prior to writing it to the ethernet. Also optional insertion of a vlan tag for those packets without tags. The number of times to replay the file to the ethernet, and the number of threads you'd like transmitting concurrently. Finally, we have snf_bridge, I've not personally used this one. With snf_bridge you define the port to use for capture and injection (they can be different), the number of in memory rings to use, and the cpu binding mask. The you can specify the number of packets to forward before exiting. The number of times to retry forwarding a packet before dropping it. The amount of time to wait between capture & injection, in milliseconds. Finally, the option to reflect non UDP and TCP packets to the network device. All of these sample programs, and several more, are available in the /opt/snf/bin/tests directory in binary form and /opt/snf/share/examples directory as source code.

So Federation, Romulan, or Myricom, a clear problem statement, and the tool box we bring to the table defines the products we build and the solutions we offer. Unless of course you're Captain Kirk, who prefers when possible to redefine the problem into something that he can solve with the resources at hand. If you ever want to chat packet capture please don't hesitate to flip open your communicator and ring me up, 919-389-5064.

No comments:

Post a Comment