{"id":283504,"date":"2025-06-09T11:28:52","date_gmt":"2025-06-09T11:28:52","guid":{"rendered":"https:\/\/peraltafinancing.com\/analytics\/ml-model-serving-with-fastapi-and-redis-for-faster-predictions\/"},"modified":"2025-06-09T11:28:52","modified_gmt":"2025-06-09T11:28:52","slug":"ml-model-serving-with-fastapi-and-redis-for-faster-predictions","status":"publish","type":"post","link":"https:\/\/fivemor.com\/?p=283504","title":{"rendered":"ML Model Serving with FastAPI and Redis for faster predictions"},"content":{"rendered":"


\n<\/p>\n

\n

Ever waited too long for a model to return predictions? We have all been there. Machine learning models, especially the large, complex ones, can be painfully slow to serve in real time. Users, on the other hand, expect instant feedback. That\u2019s where latency becomes a real problem. Technically speaking, one of the biggest problems is redundant computation when the same input triggers the same slow process repeatedly. In this blog, I\u2019ll show you how to fix that. We will build a FastAPI-based ML service and integrate Redis caching to return repeated predictions in milliseconds.<\/p>\n

What is FastAPI?<\/h2>\n

FastAPI<\/a> is a modern, high-performance web framework for building APIs with Python. It uses Python<\/a>\u2018s type hints for data validation and automatic generation of interactive API documentation using Swagger UI and ReDoc. Built on top of Starlette and Pydantic, FastAPI supports asynchronous programming, making it comparable in performance to Node.js and Go. Its design facilitates rapid development of robust, production-ready APIs, making it an excellent choice for deploying machine learning models as scalable RESTful services.\u00a0<\/p>\n

What is Redis?<\/h2>\n

Redis (Remote Dictionary Server) is an open-source, in-memory data structure store that functions as a database, cache, and message broker. By storing data in memory, Redis offers ultra-low latency for read and write operations, making it ideal for caching frequent or computationally intensive tasks like machine learning model predictions. It supports various data structures, including strings, lists, sets, and hashes, and provides features like key expiration (TTL) for efficient cache management.<\/p>\n

Why Combine FastAPI and Redis?<\/h2>\n
\n
\"\"\/<\/figure>\n<\/div>\n

Integrating FastAPI<\/a> with Redis creates a system that is both responsive and efficient. FastAPI serves as a swift and reliable interface for handling API requests, while Redis acts as a caching layer to store the results of previous computations. When the same input is received again, the result can be retrieved instantly from Redis, bypassing the need for recomputation. This approach reduces latency, lowers computational load, and enhances the scalability of your application. In distributed environments, Redis serves as a centralised cache accessible by multiple FastAPI instances, making it an excellent fit for production-grade machine learning deployments.<\/p>\n

Now, let\u2019s walk through the implementation of a FastAPI application that serves machine learning model predictions with Redis caching. This setup ensures that repeated requests with the same input are served quickly from the cache, reducing computation time and improving response times. The steps are mentioned below:\u00a0<\/p>\n

    \n
  1. Loading a Pre-trained Model<\/li>\n
  2. Creating a FastAPI Endpoint for Predictions<\/li>\n
  3. Setting Up Redis Caching<\/li>\n
  4. Measuring Performance Gains<\/li>\n<\/ol>\n

    Now, let\u2019s see these steps in more detail.<\/p>\n

    Step 1: Loading a Pre-trained Model<\/h3>\n

    First, assume that you already have a trained machine learning model that is ready to deploy. In practice, most of the models are trained offline (like a scikit-learn model, a TensorFlow<\/a>\/Pytorch model, etc), saved to disk, and then loaded into a serving app. For our example, we will create a simple scikit-learn classifier that will be trained on the famous Iris flower dataset and saved using joblib. If you already have a saved model file, you can skip the training part and just load it. Here\u2019s how to train a model and then load it for serving:<\/p>\n

    from sklearn.datasets import load_iris\nfrom sklearn.ensemble import RandomForestClassifier\nimport joblib\n\n# Load example dataset and train a simple model (Iris classification)\nX, y = load_iris(return_X_y=True)\n\n# Train the model\nmodel = RandomForestClassifier().fit(X, y)\n\n# Save the trained model to disk\njoblib.dump(model, \"model.joblib\")\n\n# Load the pre-trained model from disk (using the saved file)\nmodel = joblib.load(\"model.joblib\")\n\nprint(\"Model loaded and ready to serve predictions.\")<\/code><\/pre>\n
    In the above code, we have used scikit-learn\u2019s built-in Iris dataset<\/a>, trained a random forest classifier on it, and then saved that model to a file called model.joblib<\/em>. After that, we have loaded it back using joblib.load. The joblib library is pretty common when it comes to saving scikit-learn models, mostly because it is good at handling NumPy arrays inside models. After this step, we have a model object ready to predict on new data. Just a heads-up, though, you can use any pre-trained model here, the way you serve it using FastAPI, and also cached results would be more or less the same. The only thing is, the model should have a predict method that takes in some input and produces the result. Also, make sure that the model\u2019s prediction stays the same every time you give it the same input (so it\u2019s deterministic). If it\u2019s not, caching would be problematic for non-deterministic models as it would return incorrect results.<\/p>\n
    Step 2: Creating a FastAPI Prediction Endpoint<\/h3>\n
    Now that we have a model, let\u2019s use it via API. We will be using FASTAPI to create a web server that attends to prediction requests. FASTAPI makes it easy to define an endpoint and map request parameters to Python function arguments. In our example, we will assume the model accepts four features. And will create a GET endpoint \/predict<\/code> that accepts these features as query parameters and returns the model\u2019s prediction.<\/p>\n
    from fastapi import FastAPI\nimport joblib\n\napp = FastAPI()\n\n# Load the trained model at startup (to avoid re-loading on every request)\nmodel = joblib.load(\"model.joblib\")  # Ensure this file exists from the training step\n\n@app.get(\"\/predict\")\ndef predict(sepal_length: float, sepal_width: float, petal_length: float, petal_width: float):\n    \"\"\" Predict the Iris flower species from input measurements. \"\"\"\n    \n    # Prepare the features for the model as a 2D list (model expects shape [n_samples, n_features])\n    features = [[sepal_length, sepal_width, petal_length, petal_width]]\n    \n    # Get the prediction (in the iris dataset, prediction is an integer class label 0,1,2 representing the species)\n    prediction = model.predict(features)[0]  # Get the first (only) prediction\n    \n    return {\"prediction\": str(prediction)}\n<\/code><\/pre>\n
    In the above code, we have made a FastAPI app, and upon executing the file, it starts the API server. FastAPI is super fast for Python, so it can handle lots of requests easily. Then we load the model just at the start because doing it again and again on every request would be slow, so we keep it in memory, which is ready to use. We created a \/predict<\/code> endpoint with @app.get<\/code>, GET makes testing easy since we can just pass things in the URL, but in real projects, you will probably want to use POST, especially if sending big or complex input like images or JSON. This function takes 4 inputs: sepal_length<\/code>, sepal_width<\/code>, petal_length<\/code>, and petal_width<\/code>, and FastAPI auto reads them from the URL. Inside the function, we put all the inputs into a 2D list (because scikit-learn accepts only a 2D array), then we call model.predict()<\/code>, and it gives us a list. Then we return it as JSON like { \u201cprediction\u201d: \u201c...\u201d}<\/code>.<\/p>\n
    Therefore, now it works, you can run it using uvicorn main:app --reload<\/code>, hit \/predict<\/code>, endpoint and get results. Even if you send the same input again, it still runs the model again, which is not good, so the next step is adding Redis to cache the previous results and skip redoing them.<\/p>\n
    Step 3: Adding Redis Caching for Predictions<\/h3>\n
    To cache the model output, we will be using Redis. First, make sure the Redis server is running. You can install it locally or just run a Docker container; it usually runs on port 6379<\/strong> by default. We will be using the Python redis library to talk to the server.<\/p>\n
    So the idea is simple: when a request comes in, create a unique key that represents the input. Then check if the key exists in Redis; if that key is already there, which means we already cached this before, so we just return the saved result, no need to call the model again. If not there, we do model.predict<\/code>, get the output, save it in Redis, and send back the prediction.<\/p>\n
    Let\u2019s now update the FastAPI app to add this cache logic.<\/p>\n
    !pip install redis\nimport redis  # New import to use Redis\n\n# Connect to a local Redis server (adjust host\/port if needed)\ncache = redis.Redis(host=\"localhost\", port=6379, db=0)\n\n@app.get(\"\/predict\")\ndef predict(sepal_length: float, sepal_width: float, petal_length: float, petal_width: float):\n    \"\"\"\n    Predict the species, with caching to speed up repeated predictions.\n    \"\"\"\n    # 1. Create a unique cache key from input parameters\n    cache_key = f\"{sepal_length}:{sepal_width}:{petal_length}:{petal_width}\"\n    \n    # 2. Check if the result is already cached in Redis\n    cached_val = cache.get(cache_key)\n    \n    if cached_val:\n        # If cache hit, decode the bytes to a string and return the cached prediction\n        return {\"prediction\": cached_val.decode(\"utf-8\")}\n    \n    # 3. If not cached, compute the prediction using the model\n    features = [[sepal_length, sepal_width, petal_length, petal_width]]\n    prediction = model.predict(features)[0]\n    \n    # 4. Store the result in Redis for next time (as a string)\n    cache.set(cache_key, str(prediction))\n    \n    # 5. Return the freshly computed prediction\n    return {\"prediction\": str(prediction)}<\/code><\/pre>\n
    In the above code, we added Redis now. First, we made a client using redis.Redis()<\/code>. It connects to the Redis server. Using db=0 by default. Then we created a cache key just by joining the input values. Here it works because the inputs are simple numbers, but for complex ones it\u2019s better to use a hash or a JSON string. The key must be unique for each input. We have used cache.get(cache_key)<\/code>. If it finds the same key, it returns that, which makes it fast, and with this, there is no need to rerun the model. But if it is not found in the cache, we need to run the model and get the prediction. Finally, save that in Redis using cache.set()<\/code>. So next time, when the same input comes, it\u2019s already there, and caching would be fast.<\/p>\n
    Step 4: Testing and Measuring Performance Gains<\/h3>\n
    Now that our FastAPI app is running and is connected to Redis, it\u2019s time for us to test how caching improves the response time. Here, I will demonstrate how to use Python\u2019s requests library to call the API twice with the same input and measure the time taken for each call. Also, make sure that you start your FastAPI before running the test code:<\/p>\n
    import requests, time\n# Sample input to predict (same input will be used twice to test caching)\nparams = {\n\"sepal_length\": 5.1,\n\"sepal_width\": 3.5,\n\"petal_length\": 1.4,\n\"petal_width\": 0.2\n}\n\n# First request (expected to be a cache miss, will run the model)\nstart = time.time()\nresponse1 = requests.get(\"http:\/\/localhost:8000\/predict\", params=params)\nelapsed1 = time.time() - start\nprint(\"First response:\", response1.json(), f\"(Time: {elapsed1:.4f} seconds)\")<\/code><\/pre>\n
    \n
    \"Output<\/figure>\n<\/div>\n
    # Second request (same params, expected cache hit, no model computation)\nstart = time.time()\nresponse2 = requests.get(\"http:\/\/localhost:8000\/predict\", params=params)\nelapsed2 = time.time() - start\nprint(\"Second response:\", response2.json(), f\"(Time: {elapsed2:.6f}seconds)\")<\/code><\/pre>\n
    \n
    \"Output<\/figure>\n<\/div>\n
    When you run this, you should see the first request return a result. Then the second request returns the same result, but noticeably faster. For example, you might find the first call took on the order of tens of milliseconds (depending on model complexity), while the second call might be a few milliseconds or less. In our simple demo with a lightweight model, the difference might be small (since the model itself is fast), but the effect is drastic for heavier models. <\/p>\n
    Comparison<\/h3>\n
    To put this into perspective, let\u2019s consider what we achieved:<\/p>\n