AWS observability: CloudWatch and beyond
Metrics, logs and traces. This post digs into the topic, drawing on recent discussion from Hacker News, Reddit, and the wider community.
The conversation that prompted this post is linked in the sources. I have tried to capture the substance of the discussion rather than just summarize it, because the comments are usually where the real insight lives.
What happened
Managed databases trade operational burden for cost and flexibility. Know what you are giving up.
Backups are not disaster recovery. Test your restores or you do not have backups.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Autoscaling is not a substitute for capacity planning. It reacts to load; it does not predict it.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Spot instances are free money if your workload can tolerate interruption. Most batch and async workloads can.
Spot instances are free money if your workload can tolerate interruption. Most batch and async workloads can.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Why it matters
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Backups are not disaster recovery. Test your restores or you do not have backups.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Backups are not disaster recovery. Test your restores or you do not have backups.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Backups are not disaster recovery. Test your restores or you do not have backups.
Practical takeaways
- Infrastructure as code is not optional at scale.
- The shared responsibility model means the cloud secures the infrastructure and you secure everything else.
- Tagging is boring and essential.
- Managed databases trade operational burden for cost and flexibility.
- Infrastructure as code is not optional at scale.
- Multi-region is not multi-cloud.
- Autoscaling is not a substitute for capacity planning.
- Reserved instances and savings plans reward commitment.
- Serverless is great until it isn't.
- Spot instances are free money if your workload can tolerate interruption.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Spot instances are free money if your workload can tolerate interruption. Most batch and async workloads can.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
Sources
This post is informed by discussion on aws.amazon.com and related threads on Hacker News and Reddit. Primary source: https://aws.amazon.com/cloudwatch/.
Autoscaling is not a substitute for capacity planning. It reacts to load; it does not predict it.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Managed databases trade operational burden for cost and flexibility. Know what you are giving up.
Serverless is great until it isn't. Cold starts, concurrency limits, and vendor lock-in are real costs that show up at scale.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Managed databases trade operational burden for cost and flexibility. Know what you are giving up.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Networking is where cloud bills hide. Data transfer charges between regions and out to the internet add up fast.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Backups are not disaster recovery. Test your restores or you do not have backups.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
Networking is where cloud bills hide. Data transfer charges between regions and out to the internet add up fast.
Networking is where cloud bills hide. Data transfer charges between regions and out to the internet add up fast.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
Tagging is boring and essential. Without consistent tags, cost attribution is guesswork.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Autoscaling is not a substitute for capacity planning. It reacts to load; it does not predict it.
Infrastructure as code is not optional at scale. Clicking in the console is fine for learning and unacceptable for production.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Multi-region is not multi-cloud. Running in two AWS regions is redundancy. Running on AWS and GCP is strategy, and it is expensive.
Networking is where cloud bills hide. Data transfer charges between regions and out to the internet add up fast.
Networking is where cloud bills hide. Data transfer charges between regions and out to the internet add up fast.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
Cloud cost is the new technical debt. Teams that do not track spend by service and team end up with bills that nobody can explain and nobody can reduce.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Backups are not disaster recovery. Test your restores or you do not have backups.
The shared responsibility model means the cloud secures the infrastructure and you secure everything else. Most breaches are in the everything else.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Backups are not disaster recovery. Test your restores or you do not have backups.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Backups are not disaster recovery. Test your restores or you do not have backups.
Autoscaling is not a substitute for capacity planning. It reacts to load; it does not predict it.
Managed databases trade operational burden for cost and flexibility. Know what you are giving up.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Managed databases trade operational burden for cost and flexibility. Know what you are giving up.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
The cloud is just someone else's computer, and that someone else is very good at charging you for convenience. The question is always whether the convenience is worth the markup.
IAM is the foundation of cloud security. Get it wrong and nothing else matters. Get it right and most attacks become much harder.
Backups are not disaster recovery. Test your restores or you do not have backups.
Reserved instances and savings plans reward commitment. If your workload is stable, not using them is leaving money on the table.
Autoscaling is not a substitute for capacity planning. It reacts to load; it does not predict it.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.
Observability is the difference between knowing something is broken and knowing why. Invest in it before you need it.