Tag: elk stack

How to install Elastic SIEM and Elastic EDR

December 3, 2020 / Paul1

Installing Elastic EDR & SIEM

A few months ago I released a couple of blog posts on how to create enterprise monitoring at home with ELK and Zeek. This post is a continuation of that series….sort of. A lot has changed since those posts, mainly updates to the ELK stack and the release of a number of free EDR tools. OpenEDR released by Comodo and Elastic EDR. So I thought now would be a good time to see what’s changed with Elastic, and try out their new EDR. So for this post, I’m going to show how to install Elastic SIEM and Elastic EDR from scratch.

Network Design

Below is a very simple network diagram for this post. ELK is running on a Ubuntu 20.04 Server hosted on ESXi. Zeek is also running on a Ubuntu 20.04 server, and a port on my switch is being mirrored to a port on my ESXi server. You can read more about Zeek and port mirroring in my previous blog here.

Also running on ESXi is a Windows 10 machine, where we will install the Elastic EDR agent. I also have a Windows AD machine. This host does not feature in this post but will be used in future posts where I perform additional testing with the Elastic EDR.

Installing ELK Basics

I’m going to breeze through this section, as I’ve covered it before, and there are tons of guides out there already on how to get a basic ELK setup working. In my case, I’m using the newest release of ELK which is 7.10.

For my ELK setup, I’m using a single Ubuntu Server 20.04 virtual machine running on ESXi. This server will run Elasticsearch and Kibana.

First install transport-https

apt-get install curl apt-transport-https

Next add the Elastic repositories to your source list.

curl -s https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | tee /etc/apt/sources.list.d/elastic-7.x.list

Then update.

apt-get update

Now install Elasticsearch

apt-get install elasticsearch

Once Elasticsearch is installed, we need to make a couple of changes to its configuration file. This file is located in /etc/elasticsearch/elasticsearch.yml In order to access this file, you need root privileges.

First, we need to change the network.host value. Its default is set to the localhost. Change it to the IP address of the host you installed Elasticsearch onto.

network.host: <elasticsearch_ip>

Next within the same file, we need to change two node name values. In my case, I’ve just set both values to node-1

node.name: <node_name>
cluster.initial_master_nodes: ["<node_name>"]

Now you should be ready to start Elasticsearch and check that it’s started correctly.

service elasticsearch start
service elasticsearch status

You can also check that Elasticsearch is accessible from other hosts by running:

curl http://<elasticsearch_ip>:9200

The output should look similar to the one below, (Disclosure: this pic is stolen from my previous ELK post, hence why the details don’t match my new deployment)

Installing Kibana

Run the command below to install Kibana.

apt-get install kibana

Once the installation is complete, edit your /etc/kibana/kibana.yml and specify the IP address hosting Kibana.

server.host: "Your_IP"

In the same file, also specify the IP address of your Elastic Instance.

elasticsearch.hosts: ["https://Your_ElasiticSearch_IP:9200"]

Start Kibana and check it’s status.

service kibana start
serivce kibana status

Installing Beats

Filebeat is used to ship data from devices to Elasticsearch. There are a number of different Filebeat modules for different products that send logs and data to Elasticsearch in the required format. In this example, we’re using the module for Zeek, but Elastic has greatly expanded it’s support for additional products in recent months, including AWS, CrowdStrike, ZScaler and more.

For now, we’re just going to install Filebeat on our host running Zeek, we’ll worry about configuring it later.

apt-get install filebeat

Configuring X-Pack

As it stands, the only functionality we have within our ELK deployment is log ingestion and visualisation. We can ingest logs into ElastiSearch, and manipulate the data with Kibana visualisations, but the core functionality of a SIEM is missing. We’re unable to build detections or use cases. This feature is not available “out of the box” and in order to use it, we must first configure security between all of our different nodes. X-Pack is the Elastic package which is basically responsible for all Elastic Security functionality.

One key component that is required is to configure SSL connections between each node, there are a number of ways to do this. We are going to use X-Pack to do this as well.

Firstly, on the host with Elasticsearch installed, we need to create a YAML file, /usr/share/elasticsearch/instances.yml which is going to contain the different nodes/instances that we want to secure with SSL. In my case, I only have Elasticsearch, Kibana and Zeek.

instances:
    - name: "elasticsearch"
      ip:
        - "192.168.1.232"
    - name: "kibana"
      ip:
        - "192.168.1.232"
    - name: "zeek"
      ip:
        - "192.168.1.234"

Next, we’re going to use Elastic’s certutil tool to generate certificates for our instances. This will also generate a Certificate Authority as well.

/usr/share/elasticsearch/bin/elasticsearch-certutil cert ca --pem --in instances.yml --out certs.zip

This will create a .crt and .key file for each of our instances, and also a ca.crt file as well.

You can unzip the different certificates with unzip.

unzip /usr/share/elasticsearch/certs.zip -d /usr/share/elasticsearch/

Now we have our certificates, we can configure each instance.

Configuring Elasticsearch SSL

Firstly, we need to create a folder to store our certificates in on our Elasticsearch host.

mkdir /etc/elasticsearch/certs/ca -p

Next, we need to copy our unzipped certificates to their relevant folders and set the correct permissions.

cp ca/ca.crt /etc/elasticsearch/certs/ca
cp elasticsearch/elasticsearch.crt /etc/elasticsearch/certs
cp elasticsearch/elasticsearch.key /etc/elasticsearch/certs
chown -R elasticsearch: /etc/elasticsearch/certs
chmod -R 770 /etc/elasticsearch/certs

Next, we need to add our SSL configuration to our /etc/elasticsearch/elasticsearch.yml file.

# Transport layer
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca/ca.crt" ]

# HTTP layer
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.verification_mode: certificate
xpack.security.http.ssl.key: /etc/elasticsearch/certs/elasticsearch.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/certs/elasticsearch.crt
xpack.security.http.ssl.certificate_authorities: [ "/etc/elasticsearch/certs/ca/ca.crt" ]

Now restart Elasticsearch.

service elasticsearch restart

Configuring Kibana SSL

Now we’re going to repeat the process but this time for Kibana. The configuration is slightly different for Kibana.

Again, move your certificates to the correct folder and set the correct permissions.

Note: The following steps are assuming you are running a standalone ELK configuration. If you are running a distributed deployment, you’ll need to move certificates to the appropriate hosts.

mkdir /etc/kibana/certs/ca -p
cp ca/ca.crt /etc/kibana/certs/ca
cp kibana/kibana.crt /etc/kibana/certs
cp kibana/kibana.key /etc/kibana/certs
chown -R kibana: /etc/kibana/certs
chmod -R 770 /etc/kibana/certs

Next, in your file /etc/kibana/kibana.yml add the settings for SSL between Elasticsearch and Kibana.

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://192.168.1.232:9200"]
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/ca/ca.crt"]
elasticsearch.ssl.certificate: "/etc/kibana/certs/kibana.crt"
elasticsearch.ssl.key: "/etc/kibana/certs/kibana.key"

And then in the same file, add the configuration between Kibana and your browser.

# These settings enable SSL for outgoing requests from the Kibana server to the browser.
server.ssl.enabled: true
server.ssl.certificate: "/etc/kibana/certs/kibana.crt"
server.ssl.key: "/etc/kibana/certs/kibana.key"

Then, restart Kibana.

service kibana restart

Configuring Beats (Zeek) SSL

For the next step, we need to configure SSL for our host running Zeek and Beats. If you’re not running Zeek or any other product that uses a Filebeats module like Suricata, Windows Event Log, etc.. then you can skip this step.

First copy your certificates over to your host running Zeek, then create the cert directories with the correct permissions. You’ll need to copy both the Zeek certificates and the CA certificate.

mkdir /etc/filebeat/certs/ca -p
cp ca/ca.crt /etc/filebeat/certs/ca
cp zeek/zeek.crt /etc/filebeat/certs
cp zeek/zeek.key /etc/filebeat/certs
chmod 770 -R /etc/filebeat/certs

Next, we need to add our changes to the /etc/filebeat/filebeat.yml

First our Elasticsearch config settings.

# Elastic Output
output.elasticsearch.hosts: ['192.168.1.232:9200']
output.elasticsearch.protocol: https
output.elasticsearch.ssl.certificate: "/etc/filebeat/certs/zeek.crt"
output.elasticsearch.ssl.key: "/etc/filebeat/certs/zeek.key"
output.elasticsearch.ssl.certificate_authorities: ["/etc/filebeat/certs/ca/ca.crt"]

And then our Kibana config settings.

# Kibana Host
host: "https://192.168.1.232:5601"
  ssl.enabled: true
  ssl.certificate_authorities: ["/etc/filebeat/certs/ca/ca.crt"]
  ssl.certificate: "/etc/filebeat/certs/zeek.crt"
  ssl.key: "/etc/filebeat/certs/zeek.key"

Then restart Filebeat.

service filebeat restart

Now you can check that FileBeats is able to contact Elastic by running the command below. Everything should return back “ok”.

filebeat test output

Adding Authentication

We also need to add authentication to Elastic. This is pretty easy to do. Firstly enabled X-Pack by editing your /etc/elasticsearch/elasticsearch.yml

# X-Pack Setting
xpack.security.enabled: true

Next, we need to generate passwords for all of our built-in Elastic roles and users. There’s a tool included with Elasticsearch to do this. Run the command below to generate these passwords and save them somewhere safe (password manager). We’ll be using the Elastic password for the next part.

/usr/share/elasticsearch/bin/elasticsearch-setup-passwords auto

Next, add the newly created Elastic credential to your Filebeat config file, /etc/filebeat/filebeat.yml

# Elastic Credentials
output.elasticsearch.username: "elastic"
output.elasticsearch.password: "Your_Elastic_Pass_Here"

Restart Filebeat.

service filebeat restart

Next, do the same for your Kibana config file, /etc/kibana/kibana.yml. Also, enable X-Pack here as well.

# Elastic Credentials
xpack.security.enabled: true
elasticsearch.username: "elastic"
elasticsearch.password: "Your_Elastic_Pass_Here"

Restart Kibana.

service kibana restart

You should now see a login box when you navigate to your Kibana IP address.

Adding data to Elasticsearch

So we now have everything set up, the next thing we need to do is start ingesting data into Elasticsearch. In my case, I’m going to be using Zeek to tap my network traffic. I’ve already covered how to do this in my previous two blogs so I’m not going to cover it again here.

To see how to install Zeek, read part 1 here.

And to see how to configure the Filebeat Zeek module, read from “Configuring Zeek” in my part 2 blog here.

Enabling Detections

Once you have completed all the above steps, and have data ingesting into Elastic, you will probably notice that you are still unable to create detections. There is one final step to complete. Edit your Kibana config file, /etc/kibana/kibana.yml and add a xpack.encryptedSavedObjects.encryptionKey. I believe this can be any 32 character string.

# X-Pack Key 
xpack.security.encryptionKey: "something_at_least_32_characters"

You should now be able to view the built-in detection rules and create your own. We’ll look at how to create rules in another future post.

Installing Elastic EDR Agent

Now we’re ready to install Elastic EDR First, navigate to the “Fleet” dashboard by clicking on the link under the management tab located on the side menu.

From the fleet management menu, click “add agent”. Now it’s likely that you’ll be requested to add an integration policy before you can install agents, just follow the wizard and keep the defaults.

We’re going to use the “Enroll in Fleet” option to install the EDR.

First, download the Elastic Agent onto your Windows/Linux Host.

Once you have the agent downloaded, keep the default policy selected under the Agent policy.

Before moving onto Step 3 we have another step to complete first.

If we were to install the agent onto our Windows host now, we would receive an error that our self-signed certificate is from an untrusted source, and our agent install would fail.

So what we need to do is to add our CA certificate as a trusted certificate on our Windows host.

Install Certificates

First search for “Local Security Policy” in your Windows search bar. Then go to Security Settings > Public Key Policies > Certificate Path Validation Settings

Then, check the Define these policy settings and also check Allow user trusted root CAs to be used to validate certificates and Allow users to trust peer trust certificates options if they’re not already selected.

Finally check Third-Party Root CAs and Enterprise Root CAs.

Click apply and ok.

Next search for certmgr.msc in the Windows Search Bar.

Then go to Trusted Root Certification Authorities > Certificates and right-click anywhere in the whitespace and select All Tasks > Import

Then simple follow the wizard and select the CA.crt we created earlier.

Install the Agent

Now we’re ready to install the Elastic Agent. Copy the install command from the Fleet Dashboard and run it with PowerShell on your Windows host where the agent is located.

If all has gone right, you should see the agent has been successfully enrolled via the fleet dashboard.

We’re not done yet however, we need to check that data is being ingested correctly into ElasticSearch from our agent. You can do this by navigating to the Data Streams tab. You should see this populated with endpoint data. If there is no data here, check your fleet settings by clicking the settings cog in the top right corner. Ensure that your ElasticSearch settings are properly set to the correct IP and not set to LocalHost.

Finally, we need to enable detection rules for our EndPoint agent. Go to the detections dashboard in the Kibana Security app, and “enable” any endpoint rules you want.

Now you can test that it works, perform some badness on your Windows host and see what happens! In my case, Elastic EDR successfully detected and prevented both the Mimikatz files and also the process execution of Mimikatz.

We can also see this in our Kibana Security Detections Dashboard.

And we can also dive a little deeper with a nice little process try graphic, and overview of the event. Very cool!

Conclusion

And so that brings this post to an end, I hope you’ve found it useful. So far with the limited testing I have done, I have found Elastic’s EDR to be impressive. It has the look and feel of an Enterprise-grade tool, which considering its open-source, is amazing. I plan to do some more thorough testing in the future to see how it performs, but as it stands it certainly seems to be leading EDR in the open-source world. It would be my go-to recommendation for small to medium businesses who may not have the budget for other expensive tools. The question is, can it give the big paid tools a run for their money.

Resources

https://www.elastic.co/guide/index.html

https://www.elastic.co/endpoint-security/

https://docs.zeek.org/en/current/

Home Monitoring: Sending Zeek logs to ELK

June 25, 2020 / Paul1

Prerequisites

This post marks the second instalment of the “Create enterprise monitoring at home” series, here is part one in case you missed it. In this post, we’ll be looking at how to send Zeek logs to ELK Stack using Filebeat. A few things to note before we get started,

I’m running ELK in its own VM, separate from my Zeek VM, but you can run it on the same VM if you want.
- ELK is running on a Ubuntu 18.04 VM.
It’s pretty easy to break your ELK stack as it’s quite sensitive to even small changes, I’d recommend taking regular snapshots of your VMs as you progress along.

Installing Elastic

Installing Elastic is fairly straightforward, firstly add the PGP key used to sign the Elastic packages.

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

If you need to, add the apt-transport-https package.

sudo apt-get install apt-transport-https

Then add the elastic repository to your source list.

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list

Finally install the ElasticSearch package.

sudo apt-get update && sudo apt-get install elasticsearch

Once installed, we need to make one small change to the ElasticSearch config file, /etc/elasticsearch/elasticsearch.yml. We’re going to set the bind address as 0.0.0.0, this will allow us to connect to ElasticSearch from any host on our network. It’s worth noting, that putting the address 0.0.0.0 here isn’t best practice, and you wouldn’t do this in a production environment, but as we are just running this on our home network it’s fine.

Once that’s done, let’s start the ElasticSearch service, and check that it’s started up properly.

sudo service elasticsearch start
sudo service elasticsearch status

You should get a green light and an active running status if all has gone well. Next, we want to make sure that we can access Elastic from another host on our network. I’m going to use my other Linux host running Zeek to test this. Run the curl command below from another host, and make sure to include the IP of your Elastic host.

curl -X GET "IP OF YOUR ELASTIC HOST:9200/?pretty"

If all has gone right, you should get a reponse simialr to the one below.

Installing Kibana

Now it’s time to install and configure Kibana, the process is very similar to installing elastic search. We’ve already added the Elastic APT repository so it should just be a case of installing the Kibana package.

sudo apt-get update && sudo apt-get install kibana

One it’s installed we want to make a change to the config file, similar to what we did with ElasticSearch. Change the server host to 0.0.0.0 in the /etc/kibana/kibana.yml file.

Once it’s installed, start the service and check the status to make sure everything is working properly.

sudo service kibana start
sudo service kibana status

You should get a green light and an active running status if all has gone well. Now let’s check that everything is working and we can access Kibana on our network. Browse to the IP address hosting kibana and make sure to specify port 5601, or whichever port you defined in the config file. You should see a page similar to the one below.

Configuring Zeek

Now that we’ve got ElasticSearch and Kibana set up, the next step is to get our Zeek data ingested into ElasticSearch. There are a couple of ways to do this. Kibana has a Filebeat module specifically for Zeek, so we’re going to utilise this module.

First, go to the SIEM app in Kibana, do this by clicking on the SIEM symbol on the Kibana toolbar, then click the “add data” button. As shown in the image below, the Kibana SIEM supports a range of log sources, click on the “Zeek logs” button.

You have to install Filebeats on the host where you are shipping the logs from. So in our case, we’re going to install Filebeat onto our Zeek server. Follow the instructions specified on the page to install Filebeats, once installed edit the filebeat.yml configuration file and change the appropriate fields. The username and password for Elastic should be kept as the default unless you’ve changed it. Make sure to change the Kibana output fields as well.

Once that is done, we need to configure Zeek to convert the Zeek logs into JSON format. First, stop Zeek from running.

zeekctl stop

Then edit the line @load policy/tuning/json-logs.zeek to the file /opt/zeek/share/zeek/site/local.zeek

Restart zeek

zeekctl deploy

And now check that the logs are in JSON format. Even if you are not familiar with JSON, the format of the logs should look noticeably different than before.

tail -f /opt/zeek/logs/current/dns.log

Now we need to configure the Zeek Filebeat module. First, enable the module.

sudo filebeat modules enable zeek

Then edit the config file, /etc/filebeat/modules.d/zeek.yml. We need to specify each individual log file created by Zeek, or at least the ones that we wish for Elastic to ingest. For each log file in the /opt/zeek/logs/ folder, the path of the “current” log, and any previous log have to be defined, as shown below.

dns:
   enabled: true
   var.paths: [ "/opt/zeek/logs/current/dns.log", "/opt/zeek/logs/*.dns.json" ]

If there are some default log files in the opt folder, like capture_loss.log that you do not wish to be ingested by Elastic then simply set the “enabled” field as false. It’s important to set any logs sources which do not have a log file in /opt/zeek/logs as enabled: false, otherwise, you’ll receive an error. Also be sure to be careful with spacing, as YML files are space sensitive.

Once that’s done, you should be pretty much good to go, launch Filebeat, and start the service.

sudo filebeat setup
sudo service filebeat start

If everything has gone right, you should get a successful message after checking the

If you go the network dashboard within the SIEM app you should see the different dashboards populated with data from Zeek! The dashboards here give a nice overview of some of the data collected from our network. We’ll learn how to build some more protocol-specific dashboards in the next post in this series.

Enriching with Suricata

This next step is an additional extra, it’s not required as we have Zeek up and working already. However adding an IDS like Suricata can give some additional information to network connections we see on our network, and can identify malicious activity. Some people may think adding Suricata to our SIEM is a little redundant as we already have an IDS in place with Zeek, but this isn’t really true. While Zeek is often described as an IDS, it’s not really in the traditional sense. Zeek collects metadata for connections we see on our network, while there are scripts and additional packages that can be used with Zeek to detect malicious activity, it does not necessarily do this on its own. Suricata is more of a traditional IDS and relies on signatures to detect malicious activity. Now I often question the reliability of signature-based detections, as they are often very false positive heavy, but they can still add some value, particularly if well-tuned.

Installing Suricata

I’m not going to detail every step of installing and configuring Suricata, as there are already many guides online which you can use. I used this guide as it shows you how to get Suricata set up quickly. I’m going to install Suricata on the same host that is running Zeek, but you can set up and new dedicated VM for Suricata if you wish. Just make sure you assign your mirrored network interface to the VM, as this is the interface in which Suricata will run against.

Once you have Suricata set up its time configure Filebeat to send logs into ElasticSearch, this is pretty simple to do. Navigate to the SIEM app in Kibana, click on the “add data” button, and select Suricata Logs

Follow the instructions, they’re all fairly straightforward and similar to when we imported the Zeek logs earlier. Step 3 is the only step that’s not entirely clear, for this step, edit the /etc/filebeat/modules.d/suricata.yml by specifying the path of your suricata.json file.

var.paths: ["/my/path/suricata.json"]

Once that’s done, complete the setup with the following commands.

./filebeat setup
./filebeat -e

If all has gone right, you should recieve a success message when checking if data has been ingested.

We can also confirm this by checking the networks dashboard in the SIEM app, here we can see a break down of events from Filebeat.

Conclusion

And that brings this post to an end! It’s fairly simple to add other log source to Kibana via the SIEM app now that you know how. I’d recommend adding some endpoint focused logs, Winlogbeat is a good choice.

I’d say the most difficult part of this post was working out how to get the Zeek logs into ElasticSearch in the correct format with Filebeat. It’s not very well documented. In the next post in this series, we’ll look at how to create some Kibana dashboards with the data we’ve ingested.

Create enterprise monitoring at home with Zeek and Elk (Part 1)

April 16, 2020 / Paul1

Intro

In this series, I’m going to show you how you can utilise open source technology to build your own network monitoring solution good enough to be deployed in any enterprise environment! The two core technologies that we’re going to use are Zeek (formerly Bro) and ELK.

For those unaware, Zeek is an open-source network monitoring framework which creates alerts and events based from data collected by a network tap. One way in which I used to describe Zeek to people is that it’s essentially an IDS but on steroids. It’s used throughout the industry, especially in the network anomaly space, in fact, the UK cybersecurity company Darktrace uses Zeek as a key component of their product.

The plan for this solution is to tap our home network with Zeek and feed the logs into Elk, with Elk we can run queries across our data, build out some beautiful dashboards with Kibana, and even create some analytics to automate some detections. We will also look into deploying an endpoint agent on some devices to and feed those logs into ELK too.

Prerequisites

In order to follow along, you’ll need

A server/old PC capable of running Zeek and ELK
- In my case, I have an HP Proliant ML350e running ESXI
- The server needs to have a minimum of 2 free network ports
- And for it to run smoothly at least 8GB memory and a decent processor.
A managed network switch which is capable of port mirroring.

Network Architecture

The diagram below shows a rough guide to my home network. The key points to take from this diagram are the mirror tap on the network and the mirror connection. To get the most out of Zeek, you need to tap the connection on your switch that goes to your router, essentially the connection from your LAN to the internet. By doing this we will capture all traffic from our home networking going out to the internet, from devices like iPhones connected to the Wi-Fi and the PiHole acting as our DNS server.

VMware Setup

You should first check that your physical NICs on your server are visible in ESXI. Go to networking>Physical NICs in ESXi and you should see them there.

Both of these physical connections should connect into your switch, with one going into your mirror port, and the other a regular switch port. You can set your mirror port in your switch config. I have a Netgear GS110TP where port 2 is the one that goes into my router, and I have port 6 free, so that is the one I’ll configure to be the mirror port.

You’ll need to know which ports on your server correspond to which ESXi physical ports. For example, port 6 on my switch plugs into Eth1 on my server, and Eth1 is called vmnic1 in ESXi. So vmnic1 will be the mirror NIC.

Next, we need to create a virtual switch will be used for the mirror data. Go to Networking > Virtual Switches > Add Standard Virtual Switch, and enter the details shown below, selecting your mirror NIC as “Uplink 1”. Make sure that you set promiscuous mode as “accept”.

Now add a port group by going to Networking> Port Groups > Add port group, and assign the virual switch you just created to it. Again make sure Promiscuous mode is enabled.

Now you’re ready to create your virtual machine, I’m using Ubuntu Server 18.04 for mine. When creating you VM, make sure you add both network adapters, including the one you’re using for the Mirrored traffic.

Preparing for Zeek

Once you’ve got Ubuntu installed, do the usual updates.

sudo apt-get update
sudo apt-get upgrade

Now check that both of your network interfaces are detected by Ubuntu. It is highly likely that your Mirror port will be down. As you can see from the image below, both of my interfaces are detected but only the management interface (ens160) is up with an IP address assigned.

To fix this, let’s first put the interface into promiscuous mode, and then bring it “up”.

ip link set "your mirror int" promisc on
ip link set "your mirror int" up

Lets now check that we’re receiving traffic on the mirror port by running tcpdump.

tcpdump -i "your mirror int"

If everything has worked, you should see the mirrored traffic flowing through the interface similar to the image below.

Installing Zeek

We’re now ready to crack on and install Zeek. To start, we need to install all the perquisites. Do so by running the command below.

sudo apt-get install cmake make gcc g++ flex bison libpcap-dev libssl-dev python-dev swig zlib1g-dev

Next we need to create the working directory for Zeek, for some reason Zeek does not do this by default on install.

sudo mkdir /opt/zeek
sudo chown -R zeek:zeek /opt/zeek
sudo chmod 740 /opt/zeek

Next download Zeek with GIT

git clone -–recursive https://github.com/zeek/zeek

Unpack the compressed files, and enter the Zeek download directory. Then set the /opt/Zeek directory we created earlier as the install directory.

cd /home/paul/zeek
./configure --prefix=/opt/zeek

Now we’re ready to install Zeek, run the following make commands, and leave it to install (this can take some time)

make
make install

Next we need to add the PATH environment variable

export PATH=/usr/local/zeek/bin:$PATH

And now we need to do add some basic config to the node.cfg file which is located in the /opt/etc/ directory. Uncomment the manager, proxy and worker-1 settings, and define your mirror interface in the worker-1 settings.

Now we’re ready to deploy Zeek by running the command below

zeekctl deploy

If all goes well we should not get any errors, and we can check to make sure everything started up properly by checking the status of zeek.

zeekctl status

Everything looks good! So now let’s go and check our logs where our captured data is being written to. Logs are located in directory /opt/zeek/logs/
The directory “current” holds the logs for the current day, while logs from previous days are archived off into their own directories. There’s also a different log file for different data types, like DNS connections, HTTP connections, etc…

Lets check the DNS logs

tail -f dns.log

Awesome! So everything is working correctly!

That concludes this post, keep an eye out for part 2 of this series, we’re going to deploy ELK and feed our logs into Elastic Search where we can build some beautiful dashboards to display our captured data.