For quite sometime now (since version 0.9.0), Spin has had built-in key-value support. In this article, we will quickly investigate how we can use that built-in key-value support when building serverless applications with Fermyon Spin.

What are key-value stores

A key-value store is a type of database that stores data as a collection of key-value pairs. Each key in the database maps to a corresponding value, which can be retrieved by querying the key. Key-value stores are often used for storing and retrieving data quickly, as they can perform reads and writes with very low latency. They are commonly used in web applications, caching layers, and distributed systems and can be implemented using various technologies, such as Redis, Cassandra, etcd, SQLite, and others.

Why should you use key-value stores in serverless apps

Developers should consider using a key-value store when building serverless applications because serverless environments have limited resources and require fast and efficient data access. Key-value stores are designed to provide quick access to data, making them an excellent fit for serverless applications where performance is critical. Additionally, key-value stores are highly scalable and can easily handle large amounts of data. This makes them ideal for serverless applications that need to handle unpredictable or variable workloads. Using a key-value store can also simplify application development, as it provides a straightforward data model that is easy to work with and requires minimal setup and maintenance. Overall, a key-value store can help developers build faster, more efficient, and more scalable serverless applications.

Key-value stores in Spin

Starting with version 0.9.0, Spin includes SQLite, and we, as developers, can use it from within our applications as a key-value store. This means we don’t have to run, manage, or maintain any external key-value store to utilize the functionality in our serverless workloads. Basically, Spin provides everything we need out of the box.

To enable support for key-value storage in Spin applications, we have to update our configuration (Spin.toml) and add the key_value_stores setting on the scope of a component ([[component]]) as shown in the following snippet:

[[component]]
id="hello-kv-store"
key_value_stores = ["default"]

If your Spin application consists of multiple components, you can add the key_value_stores = ["default"] setting to every component that requires access to the same key-value store.

Key-value store in multi-component Spin application

Let’s build a simple URL shortener using the Spin key-value store

For demonstration purposes, let’s build a simple URL shortener that provides the following capabilities:

  • create new short links (POST /create)
  • open a short link (GET /<short_code>)
  • receive a list of all generated short links (GET /all)

Before we dive into the implementation, let’s ensure we’re using the latest version of Spin and its templates:

spin --version
# spin 1.0.0 (df99be2 2023-03-21)

spin templates upgrade

Now, we will create the Spin application leveraging the http-rust template:

cd ~/sources
# Create the new Spin application using the http-rust template
spin new --accept-defaults http-rust shortener
cd shortener

# Fire-up your editor (VSCode here)
code .

Add necessary dependencies to Cargo.toml

Now that we have bootstrapped our Spin app, we can specify the necessary dependencies that we want to use to build the app. Besides serde, we’ll use the rand crate to create random char-sequences for our short links. That said, add the following dependencies to the Cargo.toml file:

# ...
[dependencies]
# ...
serde = { version = "1", features = ["derive"] }
serde_json = "1"
rand = "0.8"

Configure Spin the key-value store in Spin.toml

As mentioned before, we must declare our intent to use the key-value store in our Spin application. That said, update the component configuration in your Spin manifest (Spin.toml) and add the key_value_stores = ["default"] setting:

[[component]]
id = "shortener"
key_value_stores = ["default"]
# ...

Users of our URL-shortener, can create new short-links by sending an HTTP POST request to /create and providing the actual (long) URL as JSON request payload in the following form:

{
    "url": "https://www.thorsten-hans.com"
}

Let’s quickly create the request and response models in Rust; for the CreateLinkModel struct, I added a small function to simplify deserialization when using it in the handler.

{
    "url": "https://www.thorsten-hans.com",
    "short": "thans"
}

The LinkCreateModel is just a simple struct that allows us to send a proper response body in the form of:

use anyhow::Result;
use bytes::Bytes;
use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize)]
pub struct LinkCreatedModel {
    pub url: String,
    pub short: String,
}

#[derive(Debug, Deserialize)]
pub struct CreateLinkModel {
    pub url: String,
}

impl CreateLinkModel {
    pub(crate) fn from_bytes(b: &Bytes) -> Result<Self> {
        let r: CreateLinkModel = serde_json::from_slice(b)?;
        Ok(r)
    }
}

We can implement the actual “handler” with both models in place. Besides the actual request model, we pass a reference to spin_sdk::key_value::Store to the function for accessing the key-value store. Creating a short representation for our URL is encapsulated in the get_short method, which you can also find below:

use anyhow::Result;
use models::{CreateLinkModel, LinkCreatedModel};
use rand::{distributions::Alphanumeric, thread_rng, Rng};
use spin_sdk::key_value::Store;

fn handle_create_short_link(store: &Store, model: CreateLinkModel) -> Result<LinkCreatedModel> {
    let mut short = get_short();
    let mut exists = store.exists(&short)?;
    while exists {
        short = get_short();
        exists = store.exists(&short)?;
    }

    store.set(&short, &model.url).unwrap();
    Ok(LinkCreatedModel {
        short,
        url: model.url,
    })
}

fn get_short() -> String {
    thread_rng()
        .sample_iter(&Alphanumeric)
        .take(5)
        .map(char::from)
        .collect()    
}

Next on the list is opening a short link. Again, let’s focus on the handler implementation for now. We must check if the provided shortcode exists in the key-value store and return the original URL as Some(String). If the shortcode is unknown, we simply return None:

fn handle_open_short_link(store: &Store, short: String) -> Option<String> {
    match store.exists(&short) {
        Ok(exists) => {
            if !exists {
                return None;
            }
            let Ok(url) = store.get(&short) else {
                return None;
            };
            Some(String::from_utf8(url).unwrap())
        }
        Err(_) => None,
    }
}

Last, let’s look at the handler responsible for loading all known short links. Again we use spin_sdk::key_value::Store. This time, we will iterate over all existing keys of the key-value store and return all found keys with their corresponding values as HashMap<String,String>:

fn handle_get_all_short_links(store: &Store) -> Result<HashMap<String, String>> {
    let mut map: HashMap<String, String> = HashMap::new();
    let keys = store.get_keys()?;
    for key in keys {
        let value = store.get(&key)?;
        let value = String::from_utf8(value)?;
        map.insert(key, value);
    }
    Ok(map)
}

Now that we have all the handlers in place, it is time to connect the dots and instruct our Spin component to invoke the correct handler based on information we can find by inspecting the HTTP request.

Wiring up the handlers

Enumerations in Rust are pure dope! They’re so powerful and combined with pattern matching. They make even complex code easy to read and understand. That said, let’s take a look at the code responsible for identifying which handler should be invoked at runtime:

use crate::models::CreateLinkModel;

pub enum Api {
    CreateLink(CreateLinkModel),
    GetAllLinks,
    OpenLink(String),
    NotFound,
    BadRequest,
}

impl From<spin_sdk::http::Request> for Api {
    fn from(value: spin_sdk::http::Request) -> Self {
        match value.method() {
            &http::Method::POST => {
                match value.headers().get("spin-path-info"){
                    Some(path) => {
                        println!("Path info: {}", path.to_str().unwrap());  
                        let path = path.to_str().unwrap();
                        if path.starts_with("/create") {
                            let Ok(model) = CreateLinkModel::from_bytes(&value.body().clone().unwrap()) else {
                                return Api::BadRequest;
                            };
                            return Api::CreateLink(model);
                        }
                        Api::NotFound
                    },
                    None => {
                        println!("No path info");
                        Api::NotFound
                    }
                }
            }
            &http::Method::GET => {
                match value.headers().get("spin-path-info") {
                    Some(path) => {
                        let path = path.to_str().unwrap();
                        if path == "/all" {
                            return Api::GetAllLinks;
                        }
                        let path = path.replace("/", "");
                        Api::OpenLink(path)
                    },
                    None => Api::NotFound
                }
            },
            _ => Api::NotFound,
        }
    }
}

We precisely identified the API-callers’ intention within just a few lines of Rust. We returned the correct variant as a result of the From trait implementation we associated with our Api enum. Finally, we can implement the actual Spin component:

#[http_component]
fn handle_shortener(req: Request) -> Result<Response> {
    let r = Api::from(req);
    // create the "default" key-value store
    let store = Store::open_default()?;
    match r {
        Api::CreateLink(model) => {
            let res = handle_create_short_link(&store, model)?;
            let json = serde_json::to_string(&res)?;
            Ok(http::Response::builder()
                .status(http::StatusCode::CREATED)
                .body(Some(json.into()))?)
        }
        Api::GetAllLinks => {
            let Ok(map) = handle_get_all_short_links(&store) else {
                return Ok(http::Response::builder()
                    .status(http::StatusCode::INTERNAL_SERVER_ERROR)
                    .body(None)?);
            };
            let json = serde_json::to_string(&map)?;
            Ok(http::Response::builder()
                .status(http::StatusCode::OK)
                .body(Some(json.into()))?)
        }
        Api::OpenLink(short) => match handle_open_short_link(&store, short) {
            Some(url) => Ok(http::Response::builder()
                .status(http::StatusCode::PERMANENT_REDIRECT)
                .header(http::header::LOCATION, url)
                .body(None)?),
            None => Ok(http::Response::builder()
                .status(http::StatusCode::NOT_FOUND)
                .body(None)?),
        },
        Api::NotFound => Ok(http::Response::builder()
            .status(http::StatusCode::NOT_FOUND)
            .body(None)?),
        Api::BadRequest => Ok(http::Response::builder()
            .status(http::StatusCode::BAD_REQUEST)
            .body(None)?),
    }
}

Having everything in place, we can move on and test our URL-shortener.

Testing the URL shortener

First, let’s fire-up the URL shortener. to do so, move to the project directory in your terminal and run the spin build --up command. For actual testing, we will use cURL as shown here:

### Create a short-link
### Grab short from response using jq -r '.short'
short=$(curl -X POST -d '{"url":"https://www.thorsten-hans.com"}' \
    -H 'Content-Type: application/json' \
    http://localhost:3000/create | jq -r '.short')

### Open a short-link
curl -iX GET http://localhost:3000/$short

# HTTP/1.1 308 Permanent Redirect
# location: https://www.thorsten-hans.com
# content-length: 0
# date: Sat, 08 Apr 2023 15:04:32 GMT


### Retrieve all short-links
curl -s http://localhost:3000/all | jq

# {
#   "NHtLu": "https://www.thorsten-hans.com"
# }

Conclusion

In conclusion, key-value stores are an excellent fit for serverless applications with critical performance and limited resources. With the built-in key-value store provided by Spin, we - as developers - can easily utilize key-value stores without having to run, manage or maintain external and complex key-value services. Interacting with the key-value store is super easy because the Spin SDK is - once again - super focused and use-case oriented.

Sharing the same key-value store between multiple components makes it very handy for more complex scenarios where you may want to split your overall application into different Spin components.

You can find the entire sourcec code URL shortener in my spin-kv-url-shortener repository on GitHub.