Find slowest queries in cloudwatch in mysql query log

·

1 min read

RDS with engine mysql sends to cloudwatch slow query log in format like

# Time: 140905  6:33:11
# User@Host: user[dbname] @ hostname [1.2.3.4]
# Query_time: 10.116250  Lock_time: 0.000035 Rows_sent: 10  Rows_examined: 20878
use dbname;
SET timestamp=1409898791;
...SLOW QUERY HERE LIKE 'select * from users order by created_at desc' ...

Sorting, for example, by Query_time or Lock_time in CloudWatch Logs Insights (a service that allow to query your log files with SQL-like language) is impossible. In this case, the ability to parse raw logs in cloudwatch comes to the rescue https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-Parse.html

Here is an optimized query to show last 100 queries from query log


fields @timestamp
| parse @message /Query_time:\s*(?<Query_time>[0-9]+(?:\.[0-9]+)?)\s*Lock_time:\s*(?<Lock_time>[0-9]+(?:\.[0-9]+)?)\s*Rows_sent:\s*(?<Rows_sent>[0-9]+)\s*Rows_examined:\s*(?<Rows_examined>[0-9]+)[\s\S]*?;/
| parse @message /SET timestamp=\d+;\n(?<Query>[^;]*)/
| sort Query_time desc
| limit 100