Post 67 - The Princess and the Pcap

13 Dec 2024

Task 13, Day 7 `AWS log analysis` Oh, no. I’M SPEAKING IN CLOUDTRAIL!

Monitoring in an AWS Environment

CloudWatch

Monitoring and observability platform. Monitors system and app metrics, can configure alarms on those metrics. Running an application in the cloud can mean using lots of different services, and logs from each source. CloudWatch agent install on correct instance allows compiling, monitoring, saving logs. Terms to know:

Log Events: single log entry recording an event.
Log Streams: collection of events from single source.
Log Groups: collection of log streams based on logical sense, i.e. same service on multiple hosts.

CloudTrail

Monitors actions in AWS environment. Actions include things done by a user, a role, or an AWS service. Things to know:

Always on: enabled by default for all users.
JSON-formatted: nuff said.
Event History: record of the action of the last 90 days, queryable and filterable.
Trails: event history is default trail, but other custom trails can be set and can extend beyond 90 days.
Deliverable: CloudTrail can combine logs from various sources and deliver to CloudWatch.

JQ

Lightweight and flexible cli processor to transform and filter JSON data for ease of use. Similar to sed, awk, and grep. JQ has two inputs: the filter (made up of . for current input and [] for the specified array) and the input file. Sample JSON file:

[

{ "book_title": "Wares Wally", "genre": "children", "page_count": 20 },

{ "book_title": "Charolottes Web Crawler", "genre": "young_ware", "page_count": 120 },

{ "book_title": "Charlie and the 8 Bit Factory", "genre": "young_ware", "page_count": 108 },

{ "book_title": "The Princess and the Pcap", "genre": "children", "page_count": 48 },

{ "book_title": "The Lion, the Glitch, and the Wardrobe", "genre": "young_ware", "page_count": 218 },

]

Sample command: jq '.[]' book_list.json

Sample output:

{
    "book_title": "Wares Wally",
    "genre": "children",
    "page_count": 20
}
{ 
    "book_title": "Charolottes Web Crawler",
    "genre": "young_ware",
    "page_count": 120
}
{
    "book_title": "Charlie and the 8 Bit Factory",
    "genre": "young_ware",
    "page_count": 108
}
{
    "book_title": "The Princess and the Pcap",
    "genre": "children",
    "page_count": 48
}
{
    "book_title": "The Lion, the Glitch, and the Wardrobe",
    "genre": "young_ware",
    "page_count": 218 
}

Can nest filters: jq '.[] | .book_title' book_list.json

Sample output:

"Wares Wally"
"Charolottes Web Crawler"
"Charlie and the 8 Bit Factory"
"The Princess and the Pcap"
"The Lion, the Glitch, and the Wardrobe"

The Task

Comb through the CloudTrail and CloudWatch logs looking for details on donations to find out who did what.

Recommended Stuff

Log Universe room.