Getting started with AWS SAM CLI and Golang

Since golang support in Lambda was announced earlier this year, I've been meaning to give it a go (see what I did there?) This post is how I set up my development environment for testing golang-based Lambda functions locally using the AWS Serverless Application Model (aka. SAM) CLI tool. The code is on GitHub.

Getting Started

New starters to serverless development quickly realise that the development feedback loop can get painfully long. Without any assistance, the serverless feedback loop looks like this:

Code > Release > Test

Since your code is running on someone else's environment - which is one of the main benefits of serverless - it gets involved in the feedback loop. This release step requires you to push you code across a network. While network speeds are genearlly getting faster and faster (sometimes I'm not sure about Australia) it’s still a heck of a lot slower than not pushing it. The time that release step adds to your development process adds up over time.

Having to deploy a piece of code to confirm it works is eventually necessary in a cloud-based environment, but not something you want to have to do all the time. This is basically the modern version of "Compiling!":

Not only that, but you’re guaranteed that the remote environment is different from your development environment. While this isn’t a problem unique to serverless development - as any developer who’s uttered the words “it worked on my machine” would have to admit - not knowing if your code is going to run or not doesn’t help your productivity.

This is where the AWS SAM CLI comes in to the picture: It lets you run a local instance of your Lambda function. This means that you can do some (if not most) of your testing locally, and only deploy your code when you need to test integrations with other services. The new feedback loop looks more like:

Code > Test (Local) > Release > Test (Remote)

Having this local test stage allows you to catch Lambda runtime errors, which is particularly useful when you’re getting started with serverless applications. The SAM CLI will give you confidence your function will “just work” when you push it to the cloud.

SAM CLI

The AWS SAM CLI is a refreshed implementation of the previously released SAM Local project. The interface should be exactly the same, but just be aware that you might see some examples out there that still reference sam-local or aws-sam-local.

Under the hood, the AWS SAM CLI is using the open-sourced Docker container created by the LambCI project to run your code locally. You can trigger your functions directly (with an event payload of your choosing), or use the provided Amazon API Gateway shim to allow you to trigger your functions via HTTP. This is what we'll be using, to demonstrate a back-end API endpoitn.

AWS SAM

Before jumping in to the setup, it’s worthwhile to have an understanding of what the AWS Serverless Application Model is - If you already know SAM, then skip past this section.

Just like with CloudFormation you declare your infrastructure in a template file. You launch that template file with parameters that customise it to your specific use-case.

It's worth noting that SAM is effectively syntactic sugar on top of CloudFormation - you can't do thing in SAM that you can't do in CloudFormation, it just makes doing certain serverless things easier and simpler.

A SAM template declares the transformation that applies to it. This allows us to use the resources that start with AWS::Serverless::… in it. This is also useful to know because it means you can use normal CloudFormation resources alongside your SAM resources - the transformation leaves them alone, and they are created as you would be expected.

You can see all the serverless resources SAM implements on GitHub. You can even inspect (and improve) the code that actually performs the transformation by contributing to the SAM specification repo.

Dependencies

To follow along with the steps below, you’ll need to have a few dependencies:

• Install the SAM CLI
• Download and setup your golang environment

SAM Template

Here we simple declare a function that is triggered by a web-based event - that is, someone navigating to a given HTTP path:

---
AWSTemplateFormatVersion: 2010-09-09

Transform: AWS::Serverless-2016-10-31

Resources:

  HelloFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: main
      Runtime: go1.x
      Events:
        GetEvent:
          Type: Api
          Properties:
            Path: /
            Method: get

Outputs:

  Endpoint:
    Value:  !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"

The Outputs section included just makes it easy to find the HTTP endpoint when it gets deployed. You can find many examples (simple and complex) in the official repo’s examples directory.

With the required infrastructure defined, we can work on the code that is triggered when the event is received.

Golang Project

For this starter project, I chose to put the function’s code in an obviously-named function/ directory. I’m just a beginner to golang, so I used the simplest golang handler I could - one that returns a static JSON payload:

package main

import (
        "encoding/json"
        "log"

        "github.com/aws/aws-lambda-go/events"
        "github.com/aws/aws-lambda-go/lambda"
)

type body struct {
        Message string `json:"message"`
}

// Handler is the Lambda function handler
func Handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
        log.Println("Lambda request", request.RequestContext.RequestID)

        b, _ := json.Marshal(body{Message: "hello world"})

        return events.APIGatewayProxyResponse{
                Body:       string(b),
                StatusCode: 200,
        }, nil
}

func main() {
        lambda.Start(Handler)
}

With some code to run, I can now “glue” it all together wiht some make commands.

Makefile

Here are a list of the useful commands in the Makefile for the starter. You can see how to use them in the following sections "Running Locally" and "Deployment".

There are a few other commands defined that are used internally (e.g. clean) that don’t need to be called directly.

install
Installs the projects dependencies - effectively just the aws-lambda-go library.

build
This command specifically compiles your golang binary for Linux, as that's the Lambda runtime you get (specifically AWS Linux). If you forget to do this on a Mac (like I did originally) you'll get a really strange error.

api
Starts the function and web interface locally.

test
Runs all the unit tests.

package
Calls the sam package command (which is an alias for aws cloudformation package) to push your code to S3 for deployment.

NOTE: For this command to work, you will need to have defined an environment variable $S3_BUCKET with the name of an existing S3 bucket that SAM can stage the code for deployment. Once your code is deployed, the bucket/code is no longer required. See below for an example.

deploy
Deploys the changes that have been prepared by the package command.

NOTE: For this command to work, you will need to have defined an environment variable $STACK_NAME with the name you want the application’s CloudFormation stack to have. As per normal CloudFormation behaviour a unique name will created a new stack, using an existing name would update it. See below for an example.

Running Locally

Once your application code is ready to go (you’ve run your unit tests with make test, haven’t you?) can you start the local API with:

make api

After fetching the required docker image (don’t worry, it will get faster after the first time) your API’s connection details will appear in the output:

* Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)

You can open the URL up in your favourite browser, or by by curling it:

curl http://127.0.0.1:3000/
{"message":"hello world"}

While this example doesn’t do much, once you start playing around with the Lambda environet variables like request and context objects you will be able to iterate much quicker than without it.

Deployment

Once you’ve tested your application locally as much as you can, you’re ready for deployment. You could deploy the application via the AWS CLI, but the SAM CLI also includes the commands you need. Both the package and deploy commands can be run with one make command:

make deploy

Behind the scenes this is actually invoking multiple commands (because dependencies), and looks like this if you were to do it manually:

make clean lambda build package deploy

As noted above in the make command descriptions, the deploy and package commands require some environment variables to be set; You can set them in your environment, change the definitions at the top of the Makefile, or set them inline like this:

S3_BUCKET=myExistingDeploymentBucket STACK_NAME=myFunctionStack make deploy

Next Steps

At this point you have a working API that you can deploy consistently and quickly. You can (and should!) commit the files to source control so that changes can be tracked and managed. All the files mentioned in this post are up on my GitHub.

Making AWS API Calls
If your function is making calls to the AWS APIs, you will need to manage your AWS credentials yourself. Obviously the local-running process has no AWS IAM context (like a deployed function would have an IAM Role), and will inherit anything you give it (either by default, or setting a specific AWS_PROFILE).

Auto-reload
The next thing I plan to add to this project is a watcher to rebuild my binary on change automatically. A quick poll of my colleagues using golang has brought up Unknwon/bra, but I haven’t had a chance to try it myself yet.

Debugging
There’s an approved PR on the SAM CLI repo still waiting for release that will enable debugging golang functions.

More Complex Applications
This is obviously a very simple starting point for a golang application (i.e. microservice).

I’m interested in how others have laid-out their complex serverless golang applications - If you’ve got experiences and tips, please share in the comments below or via email.