running and debugging go lambda functions locally

A solution without CLIs, Docker or big frameworks; just a binary, a debugger and simple configs.

Debugging locally a lambda function written in Go is not a trivial task, this article is my trial at this subject, it represents the result of a week of intense research and frustration pursuing a thing that should be trivial for any developer: running your code on your machine and attaching a debugger to it. This setup it great for development and essential for building high quality software, but even after days of effort I wasn't able to properly step-through my code, so I gave up and decided to do thing the old way. After trying various approaches with recommended tools like aws-sam-cli and the serverless-framework with no success, I ended up with a very simple setup that let me step-through my code, check values and dig into the function execution, with the real debugging, and it was adopted in the back-end team at my company.

So here it goes. The setup is basically the following:

Build your lambda function with debugging symbols
Run it and attach the debugger
Make a RPC call to it using a client (included here)
Follow the code execution on visual studio

Needed software

go package delve , install it running: go get github.com/go-delve/delve/cmd/dlv
go package awslambdarpc, install running go get github.com/blmayer/awslambdarpc
I used visual studio code but it should work on other IDEs
And for convenience and speed, the file-picker VSCode extension.

Make sure your $GOPATH/bin folder is in your path so that VSCode can find them.

Before we start

Just a little brief, a lambda function is basically a RPC (remote procedure call) server, and RPC servers work in a different way: they advertise methods that they have available, and clients call these methods passing its name and the arguments needed, if any. In a lambda function the function exposed is called Invoke() and it's defined on the Function type, so the method called is: Function.Invoke, this function takes only one argument: an InvokeRequest. This type is defined in the aws-lambda-go/lambda/messages package, and it's defined as:

type InvokeRequest struct {
    Payload               []byte
    RequestId             string
    XAmznTraceId          string
    Deadline              InvokeRequest_Timestamp
    InvokedFunctionArn    string
    CognitoIdentityId     string
    CognitoIdentityPoolId string
    ClientContext         []byte
}

Luckily the only field that matters to us is Payload, and it is simply the JSON that will be passed to the lambda as input. The last piece of information is that a lambda function, as a RPC server, listens on a port, this port is chosen at runtime by an environment variable named _LAMBDA_SERVER_PORT. This is the code responsible: GitHub . So we must define it.

Configuration

First we must build our lambda function with debugging symbols, the build command goes like this:

go build -v -gcflags='all=-N -l' your/file.go

The important part is this -gcflags='all=-N -l' which is the flag for turning on the debugging symbols. You may like to add it to your Makefile or whatever you use to build, we will setup a task in VSCode briefly.

Now create the input JSON files your function is supposed to receive, it's convenient to create a folder for them as they tend to multiply, I chose events. Pay attention to the type your function is expecting to receive, some functions take input from different AWS services, so you must adjust the JSON according to that. This received type is defined on your Handler function you pass to the lambda.Start function in the main file, here is an example:

func Handler(req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    ...
}

In this case the input type for this function is an APIGatewayProxyRequest and the output type is a APIGatewayProxyResponse, that means you input and output JSONs will be of that form. Take a look at the events package to understand the format, as it can be confusing sometimes and can lead you to loose hours trying to get it right.

The launch file

VSCode uses the launch file, in .vscode/launch.json, to configure debugging sessions, here we will declare the needed port for the lambda function and how the debug session is to be setup, this is mine:

{
    "version": "0.2.0",
    "configurations": [
	{
	    "name": "Launch",
	    "type": "go",
	    "request": "launch",
	    "mode": "exec",
	    "program": "${workspaceFolder}/backend",
	    "env": {
		"_LAMBDA_SERVER_PORT": "8080"
	    },
	    "args": []
	}
    ],
    "compounds": [
	{
	    "name": "build and debug",
	    "configurations": ["Launch"],
	    "preLaunchTask": "build-debug"
	}
    ]
}

I chose port 8080 for the lambda, but you can change for whatever you prefer. This compounds field is very convenient: it lets you run a task before starting the debug session, so we point the build-debug task, to build our function for us.

The tasks file

This file, .vscode/tasks.json, is where common build tasks are declared, but you can declare many other things, for example, getting input from the user. Here we will define two things:

The build-debug target
And the event file for the RPC event, this is optional and needs the VSCode extension, if you don't want it you can run it manually in the terminal.

This is the tasks.json file I'm currently using:

{
    "version": "2.0.0",
    "inputs": [
	{
	    "id": "json",
	    "type": "command",
	    "command": "filePicker.pick",
	    "args": {
		"masks": "events/*.json",
		"display": {
		    "type": "fileName",
		    "json": "name"
		},
		"output": "fileRelativePath"
	    }
	}
    ],
    "tasks": [
	{
	    "label": "build-debug",
	    "type": "shell",
	    "command": "go build -v -gcflags='all=-N -l' ${file}"
	},
	{
	    "label": "event",
	    "type": "shell",
	    "command": "awslambdarpc -e ${input:json}",
	    "problemMatcher": []
	}
    ]
}

Some explanation here: the masks field is where you point the folder with your JSON events, you can change it at your discretion, this file is then replaced on the ${input:json} part. This is responsible for issuing the RPC request to the running lambda.

And that's all.

Running

Now it's clean and simple: with the .go file open on VSCode:

Click on Run on your sidebar, or type Command+Shift+d, Ctrl+Shift+d on Windows, then select build and run and click run. Now your lambda function will be built and run.
Then issue an event to your lambda using the run task command from the terminal bar with Command+Shift+p or Ctrl+Shift+p on Windows.
Select event, a file picker will open to show available options from the events folder.
Select the json you want and press enter, the json will be sent to the lambda function on the session and the debugger will trigger.

After these commands if everything ran well, you should then see something like:

This setup does not need docker or complicated CLIs with many configuration files, here we just explored already used configuration files with minimal changes, I hope you enjoy using this setup.

Going beyond

This workflow is great for local testing/debugging, but at first sight it can seen complicated, however after using it 2 or 3 times you notice that you'll be much quicker. This small RPC client awslambdarpc can be imported as a library into your code and used to run your test files, using it to run tests can help you validate input/output from your application.