Programming at the edge with Fastly Compute

So you’ve heard about computing at the edge, and you’ve heard that Fastly let’s you run JavaScript, Go, Rust and any other language that compiles to Wasm at the edge… well, let’s take a look and while we’re at it let’s try and understand how caching works there too.

Getting started

OK, first thing you’re going to need is a fastly account so follow the link and sign up for FREE (for more details see here).

NOTE: Fastly has recently updated its UI to make creating a new Compute service even easier by providing a step through wizard-style experience to set up and configure a new service (go check it out).

Next you’re going to need the Fastly CLI to make interacting with Fastly’s services and products much more efficient, so again, follow the link and get it installed, or you can clone the public repo and run make install.

When running the CLI you’ll need an API token so you can either generate a token via the Fastly UI and then copy/paste it into the CLI using the following command (where it will prompt you to enter the token):

fastly profile create

Or you can use SSO (Single Sign-On) to automatically generate a short-lived token that is automatically assigned to the CLI profile (and the token will be refreshed automatically when it expires :chef-kiss:):

fastly profile create --sso

To make sure the CLI is set up correctly, run the following command:

fastly whoami

Create a new Compute project

A Compute project is a directory that contains your service/application code + some configuration files required by the CLI to build a Compute ‘package’.

A Compute package is a .tar.gz file containing a main.wasm binary and a fastly.toml manifest file. The CLI handles the creation of the package, and even the initial service/application code generation.

To create a new package you can run the following command, which will prompt you for all the information, such as what language you want to use and which Fastly starter kit should be used as the basis for your application code:

fastly compute init

Or if you know what language you want to use, then you can avoid all the prompts and let the CLI choose the ‘default’ starter kit using the following command:

fastly compute init --language go --non-interactive

The above command selects Go as the language I want to use for building my compute service.

The init subcommand is going to generate the following files:

.
├── README.md
├── fastly.toml
├── go.mod
├── go.sum
└── main.go

The only file here that will likely need explaining is the fastly.toml manifest file:

authors = [""]
cloned_from = "https://github.com/fastly/compute-starter-kit-go-default"
description = ""
language = "go"
manifest_version = 3
name = "example"
service_id = ""

[scripts]
  build = "go build -o bin/main.wasm ."
  env_vars = ["GOARCH=wasm", "GOOS=wasip1"]
  post_init = "go get github.com/fastly/compute-sdk-go@latest"

Compute packages are configured using this fastly.toml file in the root of the project directory tree. This file specifies configuration metadata related to a variety of tasks:

  1. Attribution of the package (e.g., name, author)
  2. Information required by the CLI to compile and upload it to a compatible Fastly service
  3. Configuration of local server environments
  4. Bootstrapping of service configuration

I’ll explain more, but for now let’s run our project locally so we can see it working…

Run your code locally

At this point we can actually run our project locally without even having to deploy it!

Running the following command, will compile your project, create a package, and pass the package to a local server environment called Viceroy which runs your package and exposes a local web server for you to interact with:

fastly compute serve

NOTE: If you want to iteratively develop your application then you can pass the --watch flag to have the CLI monitor your files for changes and to hot reload your application.

TIP: If you want to configure files to be ignored then create a .fastlyignore file. It works like .gitignore so should be familiar to you.

You should see CLI output that looks a bit like the following:

$ fastly compute serve

✓ Verifying fastly.toml
✓ Identifying package name
✓ Identifying toolchain
✓ Running [scripts.build]
✓ Creating package archive

SUCCESS: Built package (pkg/example.tar.gz)

✓ Running local server

INFO: Command output:
--------------------------------------------------------------------------------
2024-06-12T09:51:25.191538Z  WARN no backend definitions found in /private/tmp/example/fastly.toml
2024-06-12T09:51:25.191658Z  INFO Listening on http://127.0.0.1:7676

Opening http://127.0.0.1:7676 in your web browser should show the result of running the Go default starter kit code, which generates an <iframe> loading a Fastly hosted welcome page.

Sending requests to an origin server

OK, so let’s open up the main.go and delete all the code and replace it with the following:

package main

import (
	"context"
	"fmt"
	"io"
	"time"

	"github.com/fastly/compute-sdk-go/fsthttp"
)

func main() {
	fsthttp.ServeFunc(func(ctx context.Context, w fsthttp.ResponseWriter, r *fsthttp.Request) {
		r.Header.Add("TheTime", time.Now().String())
		r.CacheOptions.TTL = 30 

		resp, err := r.Send(ctx, "httpme")
		if err != nil {
			w.WriteHeader(fsthttp.StatusBadGateway)
			fmt.Fprintln(w, err.Error())
			return
		}

		resp.Header.Set("Cache-Control", "public, s-maxage=86400")
		w.Header().Reset(resp.Header)
		w.WriteHeader(resp.StatusCode)
		io.Copy(w, resp.Body)
	})
}

The fsthttp.ServeFunc essentially sets up a listener for incoming requests and the given function is the request handler. Inside the request handler we do the following things:

  • Add a HTTP header to the incoming request object: 
    
r.Header.Add("TheTime", time.Now().String())
  • Override the HTTP caching behaviour of the response (I’ll come back to this): 
    
r.CacheOptions.TTL = 30
  • Forward the request to a ‘backend’ called httpme (again, we’ll come back to this): 
    
r.Send(ctx, "httpme")
  • Set (and possibly override) the Cache-Control HTTP header on the response: 
    
resp.Header.Set("Cache-Control", "public, max-age=60")
  • Copy all of the response headers from the backend to the user’s response: 
    
w.Header().Reset(resp.Header)
  • Send an appropriate response status code: 
    
w.WriteHeader(resp.StatusCode)
  • Start streaming the backend response body to the user: 
    
io.Copy(w, resp.Body)

OK, so there were two things we need to get a better grip on:

  1. Fastly backends
  2. Fastly caching behaviour

Let’s get into it…

Fastly backends

A backend is something that needs to be defined explicitly, it’s an origin server owned and managed by you. For example, you might have an API service that you want your Compute service/application to communicate with to get data.

To define a backend we need to first create one within our Compute service. We can do this in many ways (e.g. the Fastly UI, the Fastly API or the Fastly CLI).

For simplicity we’re going to use the fastly.toml manifest file to configure a backend, that will be created when we come to deploy our Compute service/application for the first time. We’ll also use the manifest file to configure a backend that will be used when running our Compute service/application locally, because we might want to use a mock backend locally (although for our case we’ll use the same production backend).

Add the following to your fastly.toml manifest file:

[setup.backends.httpme]
address = "http-me.glitch.me"
port = 443

[local_server.backends.httpme]
override_host = "http-me.glitch.me"
url = "https://http-me.glitch.me"

The first block [setup.backends.httpme] is used only once by the CLI when you come to deploy your service (I’ll show you that later). It tells the CLI, when it’s creating your Compute service, to also create a backend called httpme and to make sure it has the specified address and port.

The second block [local_server.backends.httpme] is used by Viceroy when running your Compute service/application locally. Whenever your code uses the Send method on a fsthttp.Request, it will ensure the request is forwarded to the specified backend, which in this case is the real service https://http-me.glitch.me.

So in our application code where we have r.Send("httpme") you can now see that this is sending the request to a ‘backend’ object called httpme which we’ve now defined/created indirectly via the fastly.toml manifest file.

Now, defining backends is a bit tedious, and the primary reason for this design is security. That said, if you’re willing to forego security, then you can ask use something called a ‘dynamic backend’ which allows you to define backends at runtime in your code (see here for an example).

Fastly caching behaviour

Now let’s look at Fastly’s caching behaviour as it’s a bit confusing. I’m going to reference the official Fastly doc Caching content with Fastly:

The Fastly edge cache is an enormous pool of storage across our network which allows you to satisfy end user requests with exceptional performance and reduce the need for requests to your origin servers. Most use cases make use of the readthrough cache interface, which works automatically with the HTTP requests that transit your Fastly service to save responses in cache so they can be reused. The first time a cacheable resource is requested at a particular POP, the resource will be requested from your backend server and stored in cache automatically. Subsequent requests for that resource can then be satisfied from cache without having to be forwarded to your servers.

So what this means is that Fastly uses something called a ‘readthrough’ cache by default. The readthrough cache respects HTTP caching semantics. So if your backend returns a response with a HTTP header such as:

Cache-Control: s-maxage=300, max-age=0

…then the readthrough cache will cache the response for 300s (i.e. it’ll use the value assigned to s-maxage). So when the next request comes into your Compute service/application, and it reaches the r.Send() line, then that will respond immediately with the cached content and not actually make a call to the backend.

But what about the max-age? Well, that’s for the browser. The user’s web browser will only see Cache-Control: max-age=0 as s-maxage is for proxies and CDNs and so Fastly will strip it from the response header.

NOTE: If you want more information on HTTP caching then take a look at my blog post HTTP Caching Guide which explains all the details.

Now, this is where the following lines in our Compute service/application code are going to affect things. Let’s take a look…

So before we forwarded the incoming request to the backend (using Send) we actually configured the readthrough to ignore what is being set in the backend’s response (i.e. ignoring the Cache-Control header) and to blanket cache the response for 30 seconds. We do this using:

// https://pkg.go.dev/github.com/fastly/compute-sdk-go@v1.3.1/fsthttp#CacheOptions
r.CacheOptions.TTL = 30

The reason we have to do this before sending the request is because of how the readthrough cache works (i.e. it automatically handles caching the responses).

NOTE: Fastly also provides a ‘simple’ cache interface and a more advanced ‘core’ cache interface exposed via the Compute SDKs. You can see examples here.

Before we send our response to the user we actually modify the cache behaviour again, but this time for the user’s web browser by overriding the backend’s max-age=0 with max-age=60:

resp.Header.Set("Cache-Control", "public, max-age=60")

⚠️ WARNING: There’s a caveat to the readthrough cache that you’ll want to be careful with. If your backend sends Cache-Control: private, then understandably the readthrough cache will not cache the response because your backend has defined that behaviour. In this case, setting r.CacheOptions.TTL will have NO EFFECT. It’s only usable for responses that Fastly considers cacheable and private is not cacheable. You would need your backend to change the Cache-Control value to be something cacheable if you wanted r.CacheOptions.TTL to have any kind of effect.

Deploying your Compute service

To wrap up this post, let’s get our Compute service/application deployed:

fastly compute publish

The above command will prompt you whenever information is required, or you can simplify the process and add the --non-interactive flag to let Fastly choose default values for everything:

fastly compute publish --non-interactive

For the first time deploying you may find it takes a bit of time because Fastly is uploading your package across its global fleet of servers. For me I’ve noticed the first deploy takes around ~30s but after that, any further changes I make to my application is almost immediately uploaded/replicated 🎉

You should see output similar to the following:

$ fastly compute publish --non-interactive

✓ Verifying fastly.toml
✓ Identifying package name
✓ Identifying toolchain
✓ Running [scripts.build]
✓ Creating package archive

SUCCESS: Built package (pkg/example.tar.gz)

✓ Verifying fastly.toml
✓ Creating service
✓ Creating domain 'regularly-living-cheetah.edgecompute.app'
✓ Uploading package
✓ Activating service (version 1)
✓ Checking service availability (status: 200)

Manage this service at:
	https://manage.fastly.com/configure/services/IuZqijThLa4VawBcGy7ba6

View this service at:
	https://regularly-living-cheetah.edgecompute.app

SUCCESS: Deployed package (service IuZqijThLa4VawBcGy7ba6, version 1)

Good luck, and I hope you enjoy programming at the edge with Fastly 🙂