January 19, 2021
In our previous post, we discussed Network Traffic Analysis on AWS and described several setups we’ve used to enable traffic analysis in our lab environments. One scenario makes use of VPC Traffic Mirroring and Zeek to provide low level networking events to defenders. The other leverages Snort and Elastic Stack to display higher level alerts for common malicious activity. Both of these solutions can be effective depending on the use case, and we feel it’s valuable to have the ability to easily implement either in a lab environment.
There are a number of existing guides to implement VPC Traffic Mirroring, Zeek, Snort, and Elastic, and some are even specific to AWS. However, during our implementations we ran into several areas of the setup process that were either non-obvious or needed tweaking for our specific scenarios. Our goal with this post is to provide a simple but comprehensive walkthrough for implementing network monitoring in an AWS lab environment using VPC Traffic Mirroring and Zeek. In a follow up post, we’ll walk through our Snort and Elastic Stack implementation.
The lab we’ll be creating in this post has several AWS resources which cost money to run. The following list covers the costs of of the individual resources and the total estimated cost per hour to run this lab environment. The estimates are for the US-East-1 region.
Total Estimated Lab Cost - $0.0643/hr
If following along and deploying resources, be sure to terminate the above resources when finished with the lab to avoid unexpected costs.
Our network monitoring setup that utilizes traffic mirroring and Zeek can be seen below. Multiple lab instances are configured to mirror traffic to a network load balancer, which then forwards this traffic on to a single Zeek instance.
For simplicity, our implementation walkthrough will be limited to a single instance. We’ll also assume that the lab infrastructure is running in a VPC with a single public subnet (with internet access) and that all traffic is allowed between the systems in that subnet (no restrictive ACLs or security groups exist).
AWS currently supports traffic mirroring for systems which run on top of their proprietary Nitro system. There are a number of instance families you can choose from. We prefer the T3 instance type for its affordable, burstable generic compute capabilities. Our suggested minimum instance sizes are t3.small for Windows and t3.micro for Linux. Since we’re only concerned with network monitoring for the purposes of this post, we’ll launch a t3.micro instance running Ubuntu Server 20.04.
This can be accomplished easily both through the AWS Console or with the CLI using the command below:
aws ec2 run-instances \
--image-id ami-0885b1f6bd170450c \
--count 1 \
--instance-type t3.micro \
--key-name <your-keypair-name> \
--security-group-ids <your-security-group> \
--subnet-id <your-subnet-id>
This system will be used to capture mirrored network traffic and inspect it with the Zeek network security monitoring tool. Follow the exact same process as step one to launch another Ubuntu Server 20.04 system into your lab environment. After this our lab will have:
The next step is to create a Network Load Balancer (NLB) which will capture the mirrored traffic from our lab system before forwarding it along to the Zeek system for analysis. During the process of creating the NLB, we also create a Target Group which defines where the traffic from the load balancer will be set. To deploy these resources via the CLI, fun the following commands:
aws elbv2 create-load-balancer \
--name MyLoadBalancer \
--type network \
--subnets <my-subnet>
aws elbv2 create-target-group \
--name MyTarget \
--protocol UDP \
--port 4789 \
--vpc-id <my-vpcId>
aws elbv2 register-targets \
--target-group-arn <targetgroup-arn> \
--targets Id=<instance-id> \
aws elbv2 create-listener \
--load-balancer-arn <loadbalancer-arn> \
--protocol UDP --port 4789 \
--default-actions Type=forward,TargetGroupArn=<targetgroup-arn>
In the EC2 console, we can select “Load Balancers” from the left hand panel, then “Create Load Balancer”. There are several types of load balancers available, and we’ll use the “Network Load Balancer” in this lab scenario.
Next we configure our load balancer to accept mirrored network traffic. There are several key settings that should match the following:
We’ll also need to select the VPC, availability zone, and subnet that our traffic will be routed to. In this case the lab VPC should only have a single availability zone and subnet to choose from. We can also allow the IP address to be assigned automatically from the available addresses within the subnet CIDR range.
The next step is to create a new target group, then register our target which will be receiving traffic from the NLB. In steps 3 and 4 in the EC2 console, ensure the settings match the following:
Network Load Balancers perform health checks on their targets, and only forward traffic if the target system reports as healthy. We’ve found that TCP health checks at 30 second intervals on port 22 are the most reliable. By changing this setting to a service that should always be listening (SSH), we remove any dependency of Zeek running for the instance to pass the health check. These settings can be defined in step 3 in the EC2 console and should match the following:
After reviewing that all our configurations are correct, we can create the NLB and target group and register the Zeek system’s InstanceID as a target. This registration will take several minutes.
We can also accomplish this setup with the following AWS CLI commands (documented here), making sure to use the appropriate values in lieu of our variable placeholders.
aws elbv2 create-load-balancer --name <my-load-balancer> --type network --subnets <my-subnet>
aws elbv2 create-target-group --name <my-target> --protocol UDP --port 4789 --vpc-id <my-vpcId>
aws elbv2 register-targets --target-group-arn <targetgroup-arn> --targets Id=<instance-id>
aws elbv2 create-listener --load-balancer-arn <loadbalancer-arn> --protocol UDP --port 4789 \
--default-actions Type=forward,TargetGroupArn=<targetgroup-arn>
Now that our Network Load Balancer is configured to start forwarding traffic to our Zeek instance, we can start to mirror network traffic from our Ubuntu Server instance to the NLB. Traffic mirroring sessions and related objects are managed in the VPC Dashboard. We’ll need to create three objects to start mirroring our instance traffic.
The first step is to create the Mirror Target. This is the NLB that our instance network traffic will be sent to. In the mirror target configuration page, there are two mandatory settings which should match the following:
Next, we’ll need a Mirror Filter. The mirror filter defines what traffic is copied in our mirror sessions. In this case, we want all inbound and outbound network activity mirrored to the NLB so that Zeek can eventually receive and analyze it. When creating our mirror filter, we’ll need to define Inbound and Outbound rules. To allow all traffic to be mirrored, we can create a single rule for each with the following parameters:
Now that we have our prerequisite pieces created, we can finally start a Traffic Mirror Session.
We can create all these objects via the CLI with the following commands. For our traffic filter, we’re creating both an ingress and egress rule for all TCP and UDP traffic.
aws ec2 create-traffic-mirror-filter
aws ec2 create-traffic-mirror-filter-rule \
--destination-cidr-block 0.0.0.0/0 \
--protocol 6 \
--rule-action accept \
--rule-number 1 \
--source-cidr-block 0.0.0.0/0 \
--traffic-direction ingress \
--traffic-mirror-filter-id <mirror-filter-id>
aws ec2 create-traffic-mirror-filter-rule \
--destination-cidr-block 0.0.0.0/0 \
--protocol 6 \
--rule-action accept \
--rule-number 2 \
--source-cidr-block 0.0.0.0/0 \
--traffic-direction egress \
--traffic-mirror-filter-id <mirror-filter-id>
aws ec2 create-traffic-mirror-filter-rule \
--destination-cidr-block 0.0.0.0/0 \
--protocol 17 \
--rule-action accept \
--rule-number 3 \
--source-cidr-block 0.0.0.0/0 \
--traffic-direction ingress \
--traffic-mirror-filter-id <mirror-filter-id>
aws ec2 create-traffic-mirror-filter-rule \
--destination-cidr-block 0.0.0.0/0 \
--protocol 17 \
--rule-action accept \
--rule-number 4 \
--source-cidr-block 0.0.0.0/0 \
--traffic-direction egress \
--traffic-mirror-filter-id <mirror-filter-id>
aws ec2 create-traffic-mirror-target \
--network-load-balancer-arn <nlb-arn>
aws ec2 create-traffic-mirror-session \
--traffic-mirror-target-id <target-id>\
--network-interface-id <eni-ubuntu-instance> \
--session-number 1 \
--packet-length 25 \
--traffic-mirror-filter-id <mirror-filter-id>
Now that we have our traffic mirror session and NLB sending mirrored traffic to the Zeek instance, it’s time to install and configure Zeek. To install Zeek and all its dependencies, we can simply add the appropriate package repository and run an apt install command.
echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_20.04/ /' | sudo tee /etc/apt/sources.list.d/security:zeek.list
curl -fsSL https://download.opensuse.org/repositories/security:zeek/xUbuntu_20.04/Release.key | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null
sudo apt update
sudo apt install zeek-lts
During the installation, we’re prompted to select the mail setup. For this scenario, we can simply select “No Configuration” as we won’t be sending any emails from this server. This will later result in an error during the Zeek service start, but won’t affect our monitoring.
The next steps are to use the ZeekCTL utility to perform a final installation, edit one configuration file, and start Zeek. Navigate to the /opt/zeek/bin directory and run the following commands.
cd /opt/zeek/bin
./zeekctl
install
exit
Next we’ll update the node.cfg configuration file. This file defines what interface Zeek is monitoring for network traffic. We’ll need to make sure this matches the interface that has an IP in our VPC’s subnet. Most likely, this will be the “ens5” interface.
cd /opt/zeek/etc
nano node.cfg
# This is a complete standalone configuration. Most likely you will
# only need to change the interface.
[zeek]
type=standalone
host=localhost
interface=ens5
After saving the changes to the node.cfg file, we can use ZeekCTL to start Zeek!
cd /opt/zeek/bin
./zeekctl
start
At this point, we have Zeek monitoring our network interface, ens5, which is receiving mirrored network traffic from our source instance. To view the traffic from this host, we can check out the various logs at /opt/zeek/logs. We can also run Zeek scripts to perform analysis on these logs, or ship them off to a central location such as a SIEM.
In this post, we’ve outlined a step by step guide for implementing network traffic monitoring with VPC Traffic Mirroring & Zeek in an AWS lab environment. Along the way, we created several types of AWS resources and configured them to work in tandem. Our AWS lab now has the following:
This setup is a great start for analyzing network traffic in a safe environment. Combined with a more realistic lab network with connected systems and simulated traffic, this can become a fully fledged cyber range for simulated offensive and defensive exercises, hiring assessments, IR and forensic simulations, and more! If you’re looking to get up and running quickly, be sure to check out our Enterprise Labs for red and blue teams alike.
In our next post, we’ll walk through an alternative setup suited for smaller lab environments. We’ll use Snort and Elastic Stack to analyze network traffic from local systems, alert on common malicious activity, and visualize it in a centralized dashboard. Stay tuned!
.svg)
