Tag: network monitoring

Intro

In this series, I’m going to show you how you can utilise open source technology to build your own network monitoring solution good enough to be deployed in any enterprise environment! The two core technologies that we’re going to use are Zeek (formerly Bro) and ELK.

For those unaware, Zeek is an open-source network monitoring framework which creates alerts and events based from data collected by a network tap. One way in which I used to describe Zeek to people is that it’s essentially an IDS but on steroids. It’s used throughout the industry, especially in the network anomaly space, in fact, the UK cybersecurity company Darktrace uses Zeek as a key component of their product.

The plan for this solution is to tap our home network with Zeek and feed the logs into Elk, with Elk we can run queries across our data, build out some beautiful dashboards with Kibana, and even create some analytics to automate some detections. We will also look into deploying an endpoint agent on some devices to and feed those logs into ELK too.

Prerequisites

In order to follow along, you’ll need

• A server/old PC capable of running Zeek and ELK
  • In my case, I have an HP Proliant ML350e running ESXI
  • The server needs to have a minimum of 2 free network ports
  • And for it to run smoothly at least 8GB memory and a decent processor.
• A managed network switch which is capable of port mirroring.

Network Architecture

The diagram below shows a rough guide to my home network. The key points to take from this diagram are the mirror tap on the network and the mirror connection. To get the most out of Zeek, you need to tap the connection on your switch that goes to your router, essentially the connection from your LAN to the internet. By doing this we will capture all traffic from our home networking going out to the internet, from devices like iPhones connected to the Wi-Fi and the PiHole acting as our DNS server.

VMware Setup

You should first check that your physical NICs on your server are visible in ESXI. Go to networking>Physical NICs in ESXi and you should see them there.

Both of these physical connections should connect into your switch, with one going into your mirror port, and the other a regular switch port. You can set your mirror port in your switch config. I have a Netgear GS110TP where port 2 is the one that goes into my router, and I have port 6 free, so that is the one I’ll configure to be the mirror port.

You’ll need to know which ports on your server correspond to which ESXi physical ports. For example, port 6 on my switch plugs into Eth1 on my server, and Eth1 is called vmnic1 in ESXi. So vmnic1 will be the mirror NIC.

Next, we need to create a virtual switch will be used for the mirror data. Go to Networking > Virtual Switches > Add Standard Virtual Switch, and enter the details shown below, selecting your mirror NIC as “Uplink 1”. Make sure that you set promiscuous mode as “accept”.

Now add a port group by going to Networking> Port Groups > Add port group, and assign the virual switch you just created to it. Again make sure Promiscuous mode is enabled.

Now you’re ready to create your virtual machine, I’m using Ubuntu Server 18.04 for mine. When creating you VM, make sure you add both network adapters, including the one you’re using for the Mirrored traffic.

Preparing for Zeek

Once you’ve got Ubuntu installed, do the usual updates.

sudo apt-get update
sudo apt-get upgrade

Now check that both of your network interfaces are detected by Ubuntu. It is highly likely that your Mirror port will be down. As you can see from the image below, both of my interfaces are detected but only the management interface (ens160) is up with an IP address assigned.

To fix this, let’s first put the interface into promiscuous mode, and then bring it “up”.

ip link set "your mirror int" promisc on
ip link set "your mirror int" up

Lets now check that we’re receiving traffic on the mirror port by running tcpdump.

tcpdump -i "your mirror int" 

If everything has worked, you should see the mirrored traffic flowing through the interface similar to the image below.

Installing Zeek

We’re now ready to crack on and install Zeek. To start, we need to install all the perquisites. Do so by running the command below.

sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev

Next we need to create the working directory for Zeek, for some reason Zeek does not do this by default on install.

sudo mkdir /opt/zeek
sudo chown -R zeek:zeek /opt/zeek
sudo chmod 740 /opt/zeek

Next download Zeek with GIT

git clone -–recursive https://github.com/zeek/zeek

Unpack the compressed files, and enter the Zeek download directory. Then set the /opt/Zeek directory we created earlier as the install directory.

cd /home/paul/zeek
./configure --prefix=/opt/zeek

Now we’re ready to install Zeek, run the following make commands, and leave it to install (this can take some time)

make
make install 

Next we need to add the PATH environment variable

export PATH=/usr/local/zeek/bin:$PATH

And now we need to do add some basic config to the node.cfg file which is located in the /opt/etc/ directory. Uncomment the manager, proxy and worker-1 settings, and define your mirror interface in the worker-1 settings.

Now we’re ready to deploy Zeek by running the command below

zeekctl deploy

If all goes well we should not get any errors, and we can check to make sure everything started up properly by checking the status of zeek.

zeekctl status

Everything looks good! So now let’s go and check our logs where our captured data is being written to. Logs are located in directory /opt/zeek/logs/
The directory “current” holds the logs for the current day, while logs from previous days are archived off into their own directories. There’s also a different log file for different data types, like DNS connections, HTTP connections, etc…

Lets check the DNS logs

tail -f dns.log

Awesome! So everything is working correctly!

That concludes this post, keep an eye out for part 2 of this series, we’re going to deploy ELK and feed our logs into Elastic Search where we can build some beautiful dashboards to display our captured data.