PoC Assembly - Network Anomaly Detection Assembly

Project Overview

The goal of this challenge is to implement a lightweight network anomaly detection tool. Tools like Wireshark and tshark have many useful features and options but can be too CPU intensive. We would like to improve the CPU resource utilization as much as possible while performing basic anomaly detection.

Competition Task Overview

This challenge should implement a network utility tool to capture and analyze the network data. Our client can use the following commands to complete this task, using tshark (command line Wireshark) in a UNIX shell script:

This command functionally works but has significant CPU impact while executing and requires additional monitoring because it is an external process. The goal of this challenge is to implement a lightweight tool to support the above functions with less CPU usage.  You can choose to start from scratch or remove some unused features in the above command to make it use less CPU.

Requirements:

Implementation of a plugin for the network stack that will extract the following metrics on an ongoing basis from any network interface on the system and write the result to a plain text file: 

1. Source IP (for inbound)Destination IP (for outbound)
2. Date
3. # Bytes
4. # Packets

2. Plugin should have minimal overhead on network flows: the http request response time should not be greatly affected by this utility. We will compare the response time with/without the utility. 

Testing

The following virtual machine will be used for testing: Amazon EC2 t2.small, running AWS Linux Image

Virtual Machine

We will provide a Amazon VM for anyone working on this problem. Please request in the forums if you are interested.

Submission Feedback:

Upon receiving your submissions, we will execute your solution on the test environment within 12 hours. The copilot will run the tests and provide basic feedback and testing results.

You should provide a script for testing. e.g. start the tool, send some requests and watch the CPU usage. For example, we will access the homepage of the top 10 websites listed here: http://www.alexa.com/topsites

TShark Quick Start Guide

See this URL for a brief tutorial/guide to get up and running quickly with TShark: https://www.wireshark.org/docs/man-pages/tshark.html

Note, you do not have to use TShark or it’s libraries. It will be useful for testing your solution against the client’s current solution to see comparison in CPU usage.

Final Submission Guidelines

• Source Code
• Deployment Guide
• Test Result