Why a TTL cache is useful

A TTL cache stores values for a limited amount of time. After the TTL expires, the entry is treated as stale and recomputed or reloaded on demand.

Common use cases include:

  • caching API responses for a short period
  • memoizing expensive calculations
  • storing feature flags or metadata that changes infrequently
  • reducing repeated database lookups in a hot path

A TTL cache is especially useful when freshness matters more than perfect consistency. If a value can be a few seconds old, caching can dramatically reduce latency and load.

Design goals

For this example, the cache should:

  • store typed values, not strings or JSON blobs
  • support per-entry expiration
  • remove stale entries lazily during access
  • be usable from multiple threads
  • keep the API small and ergonomic

We will build a generic cache with the following shape:

  • T is the cached value type
  • K is the key type, such as String or u64
  • each entry stores the value plus an expiration timestamp
  • the cache uses Arc<Mutex<...>> for shared access

This is a good baseline for many applications. If you later need very high concurrency or eviction policies like LRU, you can swap the internals without changing the public API much.

Core data model

A TTL cache needs two pieces of information for each entry: the value and when it expires. Using Instant is the right choice for expiration timing because it is monotonic and not affected by system clock changes.

use std::collections::HashMap;
use std::hash::Hash;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};

#[derive(Debug, Clone)]
struct Entry<V> {
    value: V,
    expires_at: Instant,
}

impl<V> Entry<V> {
    fn is_expired(&self) -> bool {
        Instant::now() >= self.expires_at
    }
}

The Entry type is intentionally small. It keeps expiration logic close to the data it protects, which makes the cache implementation easier to read.

Implementing the cache

Now we can define the cache itself. The cache wraps a HashMap inside a Mutex, and the whole structure is wrapped in Arc so it can be cloned and shared across threads.

#[derive(Clone)]
struct TtlCache<K, V> {
    inner: Arc<Mutex<HashMap<K, Entry<V>>>>,
    ttl: Duration,
}

impl<K, V> TtlCache<K, V>
where
    K: Eq + Hash,
{
    fn new(ttl: Duration) -> Self {
        Self {
            inner: Arc::new(Mutex::new(HashMap::new())),
            ttl,
        }
    }

    fn insert(&self, key: K, value: V) {
        let entry = Entry {
            value,
            expires_at: Instant::now() + self.ttl,
        };

        let mut map = self.inner.lock().expect("cache mutex poisoned");
        map.insert(key, entry);
    }

    fn get(&self, key: &K) -> Option<V>
    where
        V: Clone,
    {
        let mut map = self.inner.lock().expect("cache mutex poisoned");

        match map.get(key) {
            Some(entry) if !entry.is_expired() => Some(entry.value.clone()),
            Some(_) => {
                map.remove(key);
                None
            }
            None => None,
        }
    }

    fn remove(&self, key: &K) -> Option<V> {
        let mut map = self.inner.lock().expect("cache mutex poisoned");
        map.remove(key).map(|entry| entry.value)
    }

    fn purge_expired(&self) {
        let mut map = self.inner.lock().expect("cache mutex poisoned");
        map.retain(|_, entry| !entry.is_expired());
    }
}

Why get returns a clone

This implementation returns an owned V, which requires V: Clone. That keeps the API simple and avoids exposing lock guards or references tied to the mutex lifetime.

For many values, cloning is cheap enough. If you need to cache large objects, you can store Arc<V> instead and clone the Arc, which is usually much cheaper than cloning the underlying data.

Using the cache in practice

Here is a realistic example: caching a computed user profile summary for 30 seconds.

use std::time::Duration;

#[derive(Debug, Clone)]
struct UserSummary {
    display_name: String,
    score: u32,
}

fn expensive_lookup(user_id: u64) -> UserSummary {
    // Simulate a slow operation.
    UserSummary {
        display_name: format!("user-{user_id}"),
        score: (user_id % 100) as u32,
    }
}

fn main() {
    let cache = TtlCache::<u64, UserSummary>::new(Duration::from_secs(30));

    let user_id = 42;

    let summary = cache.get(&user_id).unwrap_or_else(|| {
        let computed = expensive_lookup(user_id);
        cache.insert(user_id, computed.clone());
        computed
    });

    println!("{summary:?}");
}

This pattern is common: check the cache first, compute on miss, then store the result. In a real application, this can reduce repeated database or service calls.

Adding a get_or_insert_with helper

The previous example is fine, but a cache is more ergonomic when it can compute missing values directly. We can add a helper that accepts a closure.

impl<K, V> TtlCache<K, V>
where
    K: Eq + Hash + Clone,
{
    fn get_or_insert_with<F>(&self, key: K, f: F) -> V
    where
        V: Clone,
        F: FnOnce() -> V,
    {
        if let Some(value) = self.get(&key) {
            return value;
        }

        let value = f();
        self.insert(key, value.clone());
        value
    }
}

This version clones the key only if needed. It is a small tradeoff for a cleaner API. In many real-world caches, this method becomes the primary entry point.

Example usage

fn main() {
    let cache = TtlCache::<String, String>::new(Duration::from_secs(10));

    let key = "config:region".to_string();

    let value = cache.get_or_insert_with(key.clone(), || {
        "us-east-1".to_string()
    });

    println!("{key} = {value}");
}

Handling expiration strategy

There are two common TTL strategies:

StrategyBehaviorProsCons
Lazy expirationRemove expired entries only when accessedSimple, low overheadExpired entries may remain in memory until touched
Eager expirationPeriodically scan and remove stale entriesKeeps memory cleanerRequires background work and scheduling

This tutorial uses lazy expiration plus an optional purge_expired method. That is often enough for small and medium workloads. If your cache grows large or has many one-time keys, consider adding a cleanup task.

A simple cleanup loop can run in a background thread:

use std::thread;

fn spawn_cleanup<K, V>(cache: TtlCache<K, V>, interval: Duration)
where
    K: Eq + Hash + Send + 'static,
    V: Send + 'static,
{
    thread::spawn(move || loop {
        thread::sleep(interval);
        cache.purge_expired();
    });
}

This is useful when stale entries could accumulate faster than they are read.

Best practices for a typed TTL cache

A cache is easy to get wrong if its behavior is unclear. Keep these practices in mind:

  1. Use Instant for expiration
  • Avoid SystemTime for TTL checks unless you need wall-clock semantics.
  1. Prefer typed values
  • Store structured data instead of serialized strings when possible.
  1. Keep lock scopes short
  • Hold the mutex only while reading or mutating the map.
  1. Avoid expensive cloning
  • If values are large, consider storing Arc<V>.
  1. Define clear cache semantics
  • Decide whether stale values are returned, removed, or recomputed.
  1. Test expiration behavior
  • TTL bugs are often timing bugs, so tests should cover both hit and miss paths.

Testing expiration logic

TTL code is especially worth testing because timing behavior can be subtle. A good test should verify that values are available before expiration and unavailable afterward.

#[cfg(test)]
mod tests {
    use super::*;
    use std::thread::sleep;

    #[test]
    fn entry_expires_after_ttl() {
        let cache = TtlCache::<u32, String>::new(Duration::from_millis(50));

        cache.insert(1, "hello".to_string());
        assert_eq!(cache.get(&1), Some("hello".to_string()));

        sleep(Duration::from_millis(60));
        assert_eq!(cache.get(&1), None);
    }

    #[test]
    fn get_or_insert_with_computes_once_for_hit() {
        let cache = TtlCache::<u32, String>::new(Duration::from_secs(1));

        cache.insert(7, "cached".to_string());

        let value = cache.get_or_insert_with(7, || "computed".to_string());
        assert_eq!(value, "cached");
    }
}

For more robust tests, keep TTL values short and use a small buffer around sleep durations. Avoid asserting exact millisecond boundaries, since scheduling can vary across systems.

When to use a library instead

A hand-rolled TTL cache is excellent for learning and for small internal tools. However, a production system may need features such as:

  • LRU eviction
  • per-entry TTLs
  • size limits
  • async integration
  • metrics and hit-rate tracking
  • lock-free or sharded concurrency

If you need those features, a dedicated caching crate may be a better fit. Still, understanding the implementation helps you choose the right abstraction and debug cache behavior more effectively.

Summary

A typed in-memory TTL cache is a compact and practical Rust example. By combining HashMap, Instant, and a small amount of synchronization, you can build a cache that is safe, ergonomic, and suitable for many everyday workloads.

The key ideas are straightforward:

  • store values with expiration timestamps
  • remove stale entries lazily or in a cleanup pass
  • keep the API typed and predictable
  • use Arc<Mutex<...>> when shared access is needed

Once you understand this pattern, it becomes easy to adapt it for Arc<V> values, per-key TTLs, or more advanced eviction policies.

Learn more with useful resources