Custom Atomic Operations

One of the advantage of using Lua scipts is its atomicity which guranteed scripts executions are all-or-nothing. This makes it a natural fit for implementing custom atomic operations.

What You’ll Learn

This tutorial will cover:

How Lua scripts are used to create custom operations.
How to pass arguments using KEYS and ARGV input arguments.
Understanding Script Caching and how it reduces network bandwidth.

Prerequisites

Before starting, ensure you have the following:

Python: Version 3.9 to 3.13.
Docker: To run a standalone Valkey instance.
Valkey GLIDE: The Python client library.
Basic Knowledge of Lua.

Start a Valkey instance using Docker:

docker run -d --name my-valkey -p 6379:6379 valkey/valkey:8

Install the GLIDE client:

pip install valkey-glide

Scenario: A Rare Sneaker Drop

Imagine you are launching a limited-edition sneaker with only 100 pairs available. The moment the product page goes live, you expect hundreds if not thousands of customers clicking “BUY” simultaneously.

How do you ensure no more than 100 pairs are purchased correctly? This is a classic race condition where multiple clients tries to read the same object to then modify it.

Using Valkey Commands

To avoid a race condition, a client will need to ensure that the item_key has not changed while it is updating the item_key. To do this, we can use the WATCH commands in combination with MULTI/EXEC. The pseudo code would look something like:

FUNCTION purchase_item(item_key, quantity_requested):

    # Start a retry loop. Necessary because the transaction might fail.
    WHILE True:

        try:
            # Watch the item_key for changes
            # If another client modifies 'item_key' after this line,
            # our subsequent transaction will fail.
            WATCH(item_key)

            current_stock = GET(item_key)

            IF current_stock < quantity_requested:
                UNWATCH() # Clean up
                RETURN "Fail: Insufficient Stock"

            # Start the Transaction block
            MULTI()

            # Queue the commands
            DECRBY(item_key, quantity_requested)

            # Execute
            # This returns NULL/Empty if the item_key was modified by someone else
            # since the WATCH command.
            result = EXEC()

            IF result is distinct from NULL:
                RETURN "Success"

            # If result was NULL, the loop continues and we try again immediately.

        except Error:
            BREAK loop

The pattern above is called optimistic locking. While it ensures consistency between your clients, it is not efficient in high contention situations; Many clients are trying to buy the same item. Any client that fails to modify item_key successfully must retry until they they do. With a lot of clients this can generate a “retry storm”, putting additional load on the Valkey server and increasing latency.

Using Lua Scripts

A better alternative is to use Lua scripting. Lua scripts are executed as a single atomic operation, allowing you to implement custom atomic action safely and efficiently.

The following Lua script implement our “purchase” logic. It checks for current_stock >= requested_quantity:

If yes, it deducts the stock.
If no, it fails.

local key = KEYS[1]
local quantity_requested = tonumber(ARGV[1])

-- Get current stock (if key doesn't exist, treat as 0)
local current_stock = tonumber(server.call('GET', key) or 0)

if current_stock >= quantity_requested then
    -- Success: Deduct stock
    server.call('DECRBY', key, quantity_requested)
    return 1
else
    -- Fail: Not enough stock
    return 0
end

When executed, this script is guranteed to be all-or-nothing.

The script interacts with Valkey using server.call() which executes Valkey commands with arguments. On error, server.call() will raise it to the user, stoping script immediately. To handle errors, use server.pcall().

Keys and inputs are passed into KEYS and ARGV parameters in our scripts which we will cover in the next section.

Handling Inputs with KEYS and ARGV

Valkey provides KEYS and ARGV to handles input parameters, each with their own specifc purpose:

KEY is used to pass in keys values in Valkey (ex: product:123).
ARGV is used for general input parameters.

product_key = "product:shoe:stock"
buy_quantity = "3"

result = await client.invoke_script(
    LUA_PURCHASE_SCRIPT,
    keys=[product_key],   # Maps to KEYS[1] in Lua
    args=[buy_quantity]   # Maps to ARGV[1] in Lua
)

Note that both KEYS and ARGV parameters must be string.

Why Use `KEYS`?

Valkey uses declared KEYS to locate the correct node (in both standalone and cluster mode). To ensure execution safety:

Pass all accessed keys via the keys= argument.
Only interact with keys listed in KEYS.
Never hardcode or programmatically generate key names.

Using `ARGV` and Script Caching

GLIDE caches Sciprt() server-side using content hashes to reduce overhead. Always use ARGV for variable inputs. Hardcoding values generates unique hashes for otherwise identical scripts, which defeats caching and can exhaust host memory.

Putting It All Together

The following is an an example implementation for a client that handles inventory deduction atomically using Lua scripting. It will:

Seed the database with 100 items.
Execute the Lua script to attempt a purchase of 3 items.
Handle the result and print the remaining stock.

import asyncio
from glide import GlideClient, GlideClientConfiguration, NodeAddress, Script

# This script checks stock and deducts it in a single atomic operation.
LUA_INVENTORY_SCRIPT = Script(r"""
    local key = KEYS[1]
    local quantity_requested = tonumber(ARGV[1])

    -- Get current stock (handle nil as 0)
    local current_stock = tonumber(server.call('GET', key) or 0)

    if current_stock >= quantity_requested then
        -- Success: Deduct stock
        server.call('DECRBY', key, quantity_requested)
        return 1
    else
        -- Fail: Not enough stock
        return 0
    end
""")

async def main():
    config = GlideClientConfiguration(
        addresses=[NodeAddress("localhost", 6379)]
    )

    client = await GlideClient.create(config)

    # Initialize stock
    product_key = "product:shoe:stock"
    await client.set(product_key, "100")
    print(f"Initial Stock: {await client.get(product_key)}")

    buy_quantity = "3" # Arguments must be strings

    result = await client.invoke_script(
        LUA_INVENTORY_SCRIPT,
        keys=[product_key],   # Maps to KEYS[1]
        args=[buy_quantity]   # Maps to ARGV[1]
    )

    if result == 1:
        print("✅ Purchase Successful!")
    else:
        print("❌ Purchase Failed: Insufficient Stock.")

    final_stock = await client.get(product_key)
    print(f"Remaining Stock: {final_stock}")

    client.close()

if __name__ == "__main__":
    asyncio.run(main())