Skip to content
CodeMDD.io | Coding Tutorials

Effortlessly Improve Performance with Python Cache

[

Caching in Python Using the LRU Cache Strategy

In this tutorial, we will explore caching in Python and learn how to implement the LRU (Least Recently Used) Cache strategy using the functools module. Caching is an optimization technique that allows us to store frequently accessed data in a faster and more accessible location, reducing the load on computing resources and improving performance.

Table of Contents

  • Caching and Its Uses
    • Implementing a Cache Using a Python Dictionary
    • Caching Strategies
    • Diving Into the Least Recently Used (LRU) Cache Strategy
    • Peeking Behind the Scenes of the LRU Cache
  • Using @lru_cache to Implement an LRU Cache in Python
    • Playing With Stairs
    • Timing Your Code
    • Using Memoization to Improve the Solution
    • Unpacking the Functionality of @lru_cache
  • Adding Cache Expiration
    • Evicting Cache Entries Based on Both Time and Space
    • Caching Articles With the New Decorator
  • Conclusion

Caching and Its Uses

Caching is an optimization technique that can significantly improve the performance of an application by storing frequently accessed or recently used data in a faster and more accessible location. This reduces the need to repeatedly fetch the same data from its original source, resulting in faster response times and decreased load on computing resources.

Let’s consider an example of a newsreader application that fetches the latest news articles from different sources. If the user repeatedly navigates back and forth between a couple of news articles, the application would need to fetch the same content each time, which can be time-consuming and resource-intensive.

To mitigate this issue, we can implement caching in our application. Caching allows us to store the fetched articles locally, so that the next time the user requests the same article, it can be served directly from the cache instead of fetching it again from the source. This improves the user experience and reduces the server load.

Implementing a Cache Using a Python Dictionary

In Python, we can implement a simple caching solution using a dictionary. In the newsreader example, instead of fetching articles directly from the source every time, we can check whether the article is present in our cache dictionary and retrieve it from there if available. This saves us the overhead of making repeated server requests for the same content.

Here’s an example of how we can implement a caching mechanism using a Python dictionary:

# Create an empty cache dictionary
cache = {}


def fetch_article(article_id):
    if article_id in cache:
        # Return the article from cache
        return cache[article_id]
    else:
        # Fetch the article from the source
        article_content = fetch_from_source(article_id)


        # Store the fetched article in cache
        cache[article_id] = article_content


        # Return the fetched article
        return article_content

In this example, we have a cache dictionary that stores articles using their unique IDs as keys. Before fetching a new article, we check if it already exists in the cache. If it does, we retrieve it directly from the cache. Otherwise, we fetch it from the source, store it in the cache for future use, and then return it to the user.

By implementing this caching mechanism, we reduce the number of network requests and improve the performance of our newsreader application.

Caching Strategies

Now that we understand the basics of caching and its implementation using a dictionary, let’s explore different caching strategies. Each caching strategy has its own way of determining which data to keep in the cache and when to evict or update it. Some common caching strategies include:

  • Least Recently Used (LRU): This strategy removes the least recently used items from the cache when it reaches its maximum capacity.
  • First-In-First-Out (FIFO): This strategy removes the oldest items from the cache when it reaches its maximum capacity.
  • Least Frequently Used (LFU): This strategy removes the least frequently accessed items from the cache when it reaches its maximum capacity.
  • Time-based expiration: This strategy removes items from the cache after a certain period of time has elapsed.

In this tutorial, we will focus on the LRU caching strategy using the @lru_cache decorator provided by the functools module.

Using @lru_cache to Implement an LRU Cache in Python

The @lru_cache decorator is a powerful tool provided by the functools module in Python. It allows us to easily implement an LRU cache for functions, automatically handling the caching and eviction of function results based on the LRU strategy.

To demonstrate how to use the @lru_cache decorator, let’s consider a simple problem: finding the number of unique ways to climb a staircase with a given number of steps. We can solve this problem using a recursive function, but it can be computationally expensive for large inputs due to redundant calculations.

Playing With Stairs

from functools import lru_cache


@lru_cache(maxsize=None)
def climb_stairs(n):
    if n in (0, 1):
        return 1
    return climb_stairs(n - 1) + climb_stairs(n - 2)

In this example, we define a climb_stairs function that uses recursion to find the number of unique ways to climb a staircase with n steps. The function calculates this by recursively summing the number of ways to climb n-1 steps and n-2 steps.

By applying the @lru_cache decorator to our function, we enable caching of the function results. The decorator automatically handles the caching and eviction of function results based on the LRU strategy.

Timing Your Code

To measure the performance improvement gained from caching, we can time our code before and after implementing caching. We’ll compare the execution time of the climb_stairs function with and without caching for a given number of steps.

import time


# Without caching
start_time = time.time()
result_without_cache = climb_stairs(30)
end_time = time.time()


print("Without caching:")
print("Result:", result_without_cache)
print("Execution time:", end_time - start_time, "seconds")


# With caching
start_time = time.time()
result_with_cache = climb_stairs(30)
end_time = time.time()


print("With caching:")
print("Result:", result_with_cache)
print("Execution time:", end_time - start_time, "seconds")

In this example, we calculate the number of unique ways to climb a staircase with 30 steps both with and without caching. We measure and compare the execution times of both approaches. We expect the cached version to have a significantly faster execution time due to the avoidance of redundant calculations.

Using Memoization to Improve the Solution

Memoization is a technique that involves storing the results of expensive function calls and returning the cached result when the same inputs occur again. It can be used to optimize recursive functions by avoiding redundant calculations.

Let’s modify our climb_stairs function to use memoization alongside the LRU caching strategy provided by the @lru_cache decorator.

from functools import lru_cache


@lru_cache(maxsize=None)
def climb_stairs(n):
    if n in (0, 1):
        return 1


    if n not in cache:
        cache[n] = climb_stairs(n - 1) + climb_stairs(n - 2)


    return cache[n]


# Clear the cache before running the function
climb_stairs.cache_clear()

In this modified version, we check if the result for a given input already exists in the cache. If it does not, we calculate it recursively and store it in the cache for future use.

Using memoization alongside the LRU caching strategy can further improve the performance of our function, especially for larger inputs where redundant calculations are more prevalent.

Unpacking the Functionality of @lru_cache

The @lru_cache decorator provides more functionality than just caching the result of a function. It also exposes some properties and methods that can be useful in certain scenarios.

Here are a few notable properties and methods provided by the @lru_cache decorator:

  • cache_info(): Returns the current state of the cache, including the number of cache hits, cache misses, and the maximum cache size.
  • cache_clear(): Clears the cache, removing all cached results.
  • maxsize: Specifies the maximum size of the cache. By default, it is set to 128, but it can be customized by providing an integer value.

By utilizing these properties and methods, we can gain insights into the caching behavior and manage the cache based on our specific requirements.

Adding Cache Expiration

In some cases, it is essential to remove items from the cache after a certain time to ensure the freshness of the data. The functools module does not provide built-in support for cache expiration, but we can implement our own solution using a combination of caching and time-based checks.

Let’s consider an example where we want to cache articles fetched from a remote source but ensure that the cache automatically expires after a specific time.

Evicting Cache Entries Based on Both Time and Space

from functools import lru_cache
import time


CACHE_EXPIRATION_TIME = 3600  # 1 hour in seconds


@lru_cache(maxsize=None)
def fetch_article(article_id):
    if cache_is_expired(article_id):
        invalidate_cache(article_id)
        article_content = fetch_from_source(article_id)
        update_cache(article_id, article_content)
    elif article_id not in cache:
        article_content = fetch_from_source(article_id)
        update_cache(article_id, article_content)
    return cache[article_id]


def cache_is_expired(article_id):
    cached_time = cache[article_id]["timestamp"]
    current_time = time.time()
    return current_time - cached_time >= CACHE_EXPIRATION_TIME


def invalidate_cache(article_id):
    del cache[article_id]


def update_cache(article_id, article_content):
    cache[article_id] = {
        "content": article_content,
        "timestamp": time.time()
    }

In this example, we define a fetch_article function that fetches articles from a remote source. We use a combination of caching and time-based checks to determine whether to fetch the article from the source or retrieve it from the cache. We store each cached article along with a timestamp to track its age.

The cache_is_expired function checks if the cached article has exceeded the expiration time. If it has, we invalidate the cache entry by removing it from the cache. We then fetch the article from the source and update the cache with the new content and timestamp.

By adding cache expiration support to our caching mechanism, we ensure that the cache remains up-to-date and the stored data does not become stale over time.

Caching Articles with the New Decorator

from functools import lru_cache


def cache(expires_in=None):
    def decorator(func):
        setattr(func, "_cache_expire", expires_in)
        return lru_cache(None)(func)
    return decorator


@cache(expires_in=3600)  # 1 hour expiration time
def fetch_article(article_id):
    # Fetching the article from the source
    return article_content

In this alternative approach, we define a custom cache decorator that accepts an optional expires_in argument, representing the expiration time in seconds. The decorator sets this argument as an attribute on the decorated function object, _cache_expire.

By combining the @cache decorator and the lru_cache decorator, we can cache the function results and specify an expiration time for each cached value. The caching and expiration logic is automatically handled by the lru_cache decorator, while the @cache decorator allows us to specify expiration times for different functions.

Conclusion

In this tutorial, we learned about caching in Python and explored how to implement the LRU (Least Recently Used) Cache strategy using the functools module. Caching is a powerful technique for improving application performance by reducing the load on computing resources and avoiding redundant calculations.

We implemented a cache using a Python dictionary and discussed different caching strategies. We focused on the LRU caching strategy and demonstrated how to use the @lru_cache decorator to cache function results, improving the performance of our code.

We also explored memoization as a technique for optimizing recursive functions and discussed additional functionality provided by the @lru_cache decorator.

Finally, we discussed cache expiration and explained how to combine caching with time-based checks to ensure the freshness of data stored in the cache.

By understanding caching and how to leverage caching techniques in Python, you can significantly improve the performance of your applications and reduce the load on computing resources.