With the recent announcement of the AWS generative AI service Bedrock going GA, an old discussion was kicked-off internally at $dayjob about how developers can unintentionally rack up a big AWS bill, especially for new services.

After doing a bit of digging and only finding a few partial solutions, I decided to create an SCP to protect an organization against some of the longer and more expensive API calls.

Expensive and long-running API calls

In AWS, almost everything is an API call away. This is fantastic most of time, except when you accidentally call some of the less pay-as-you-go services, like AWS Shield Advanced which automatically costs your $3,000USD per month, as Nick did:

Luckily Nick had a happy resolution, but this is an unfortunately good example of an AWS API call that can cost you a lot or lock you in to long-term commitments, similar to buying AWS EC2 Reserved Instances or Savings Plans (interestingly one of the few things in AWS that doesn't begin with "AWS" or "Amazon").

With this idea in mind we can start to create a list similar actions for other services:

If you've got some actions you'd like to see included, comment on the gist and I'll update it.

As with so many AWS-related things, the starting point for this was provided by AWS Community Hero Ian Mckay, who shared a Gist with the initial list of actions to block (he also wrote the foreword to my book about AWS IAM #shamelessplug).

Service Control Policy

Now that we have the list of actions, we can explicitly deny them in an SCP:

I like this approach because it explicitly denies the actions that your developers probably don't want to call anyway, but it's important to note this list won't save you from everything expensive on AWS. You can still spin up some very big (and expensive) instance/database types, or create infinite loops between services, and run NAT Gateways, but because these kind of activities use pay-as-you-go pricing it will take some time to drive your bill up; The benefit of this approach is that you probably won't stop your  developers doing their day job.

Some of the things that this SCP will protect you from are:

  • Committing to reserved instances and savings plans
  • Turning on Shield Advanced
  • Provisioning dedicated capacity in Bedrock
  • Subscribing to Marketplace services
  • Purchasing reserved capacity for DynamoDB, ElasticSearch (aka. OpenSearch), RDS
  • Registering domains with Route 53
  • Putting legal holds and retentions on objects in S3
  • Locking vaults in Glacier
  • Creating an Outpost (which will result in AWS delivering a rack to you!)

Creating SCPs isn't much harder than any other resources in AWS, but changing SCPs can have far-reaching implications that are beyond the scope of this article, so take care. Ideally you're using AWS CloudFormation (or some other cromulent other IaC solution) to manage this and other resources in your environment.