This article Kevin and you will learn about AWS VPC Flow Logs. As you all know, to design a separate network architecture on AWS Cloud, you need to create VPCs, in each VPC you can create resources divided by many different AZs and subnets. VPC Flow Logs is a feature that allows you to collect information about IP traffic from network interfaces in your VPC.
Some of the things that VPC Flow can help you with:
- See how security group rules work
- Monitor traffic going in and out of your instance
- Specify the IN and Out traffic direction of the Network interfaces.
This Flow Data collects incoming and outgoing traffic but is in a different flow than your traffic, so it doesn’t affect your network thruput or latency. You can freely add or delete flow logs without having to worry about latency or throughput of your instances.
Who can generate flow logs?
If you configure Flow Logs for an entire subnet, each interface of the instance in that subnet has flow logs enabled.
To configure a VPC flow logs you need to define 3 parameters:
- Data source
- Type of traffic (all, accepted, rejected)
- Destination to store the logs (Amazon S3 or CloudWatch Logs)
When you launch an EC2 instance after the flow logs are configured, AWS automatically adds these new EC2 interfaces to the flow logs.
AWS Services supported VPC Flow Logs:
- NAT Gateway
- Transit GW
You can delete the flow logs at any time and the logs will no longer be sent to the Destination (S3 or CloudWatch) but the pre-collected logs stored in CloudWatch or S3 will still be there. storage in S3 will be charged.
Flow logs capture window (aggregation interval)
By default, logs will be collected and summarized into log patterns called flow log records. By default, the aggregation interval has a maximum duration of 10 minutes. As for the format of the flow log record, you can customize it.
With a Nitro based instance the default aggregation interval is 1 minute.
After the logs are collected and condensed into a flow log record. AWS needs 1 more time to publish to S3 or CloudWatch log.
- Publish to S3: 10 minutes
- Publish to CloudWatch: 5 minutes
Some limitations of AWS VPC Flow logs
- VPC Flow logs for EC2 Classic are not available (Currently deprecated).
- Do not use VPC Flow Logs for Peered VPCs that are not in your AWS account. Assuming VPC1 belongs to my account peering with VPC2 in another AWS Account, then I can only enable vpc flow logs for VPC1, and VPC2 cannot because it is in another account.
- After creating flow logs you cannot change the configuration and format of the flow logs. You can delete and recreate the flow logs according to the desired configuration.
- If the Network interface is attached to a Nitro based system, the aggregation interval is always 1 minute or less.
Flow logs capture accepted and rejected requests but do not include the following traffic:
- Traffic that contact to Amazon DNS server. If you use your DNS it will still track.
- Traffic of windows server related to license activation
- Traffic related instance metadata 169,254.169,254
- Traffic related Amazon Time Sync Service 169,254,169,254
- DHCP traffic
- Reserved IP Address for default vpc
- Traffic between endpoint network interface and network load balancer.
The cost of using vpc flow logs is due to the input and storage of data in CloudWatch or S3. To track this cost, you can combine it with cost allocation tags to see where the costs come from…