Introduction

In the dynamic landscape of e-commerce, discovering the perfect product can be a challenge without the ability to physically showcase a sample.

Enter Smartle Visual Search. Our innovative technology allows users to upload an image of any item they desire, instantly unlocking visually similar options from our extensive inventory. With Smartle, navigating the world of online shopping becomes as effortless as browsing through a physical store.

Image search is a powerful feature that enhances user experience in various applications such as e-commerce platforms, social media networks, and content management systems. By implementing an image search API, users can easily find visually similar images, products, or content based on their preferences.

Are you looking to develope an image search functionality for your application? In this tutorial, we’ll walk through the process of creating a robust image search API using Flask, Celery for asynchronous task processing, and ChromaDB for efficient multimodal search capabilities.

Technologies Used for Smartle Visual search

• Flask: A Python web framework for building web applications.
• Celery: A distributed task queue for asynchronous task processing.
• ChromaDB: A database designed for multimodal search, capable of handling both text and image data efficiently.
• OpenCLIP: An advanced AI model for generating image embeddings using the CLIP (Contrastive Language-Image Pretraining) framework.
• Swagger: Used for API documentation and testing.
• PIL: Python Imaging Library for image processing.

Setting Up the Environment

Before diving into development, we’ll set up our environment by installing the necessary dependencies such as Flask, Celery, ChromaDB, and OpenCLIP. Additionally, we’ll configure Celery to use Redis as the message broker for task queuing, ensuring scalability and performance.

Creating the Image Search API

Our image search API will consist of endpoints for adding images to the database and retrieving suggestions based on a query image. We’ll leverage AI-powered visual search capabilities by integrating the OpenCLIP model to generate image embeddings, allowing for efficient storage and retrieval of visually similar images.

Adding Images Endpoint

The /add_images endpoint enables users to upload images to the database. Images are processed asynchronously using Celery tasks, and their embeddings are generated using the OpenCLIP model, ensuring accurate representation and efficient storage.

To access this endpoint the following params must be there,

parameters:
      - name: images
        in: formData
        type: file
        required: true
        description: The images to add.
      - name: Id
        in: formData
        type: int
        required: true

Getting Suggestions Endpoint

The /get_suggestions endpoint accepts a query image and returns visually similar images from the database. Leveraging ChromaDB’s multimodal search capabilities, we’ll perform efficient search based on the image embeddings generated by the OpenCLIP model, providing users with relevant and personalized recommendations.

To access this endpoint the following params must be there,

parameters:
      - name: image
        in: formData
        type: file
        required: true
        description: The query image.
      - name: Id
        in: formData
        type: int
        required: true
      - name: no_of_results
        in: formData
        type: int
        required: true
        description: The number of image suggestions.e 

The image suggestions will be filtered based on their accuracy

Error Handling

To ensure a seamless user experience, we’ll implement robust error handling to provide informative messages in case of errors.

Challenges we faced in Smartle Visual Search

1. Endpoint Overload: Initially, when attempting to add images to the database directly through the endpoint, we encountered issues with performance and reliability. The standard endpoint couldn’t efficiently manage the process of adding images, leading to delays and potential crashes.
2. Celery Crashes: Introducing Celery for asynchronous task processing seemed like a solution, but we soon faced another challenge. Despite leveraging Celery, we experienced crashes during the image addition process. This posed a significant setback in our efforts to streamline image management.
3. Batch Processing: To address the instability of Celery and the inefficiency of adding images individually, we implemented a batch processing approach. By adding images in batches, we aimed to distribute the workload more evenly and reduce the likelihood of crashes.
4. Efficiency and Speed: Despite our efforts, we realized that our current system still lacked the efficiency and speed required for seamless image management. As we continue to iterate on our solution, we’re exploring ways to optimize the image addition process further, ensuring both efficiency and reliability.
5. Ongoing Improvements: We’re committed to overcoming these challenges and enhancing our image management capabilities. Whether through refining our batch processing strategy or exploring alternative solutions, we’re dedicated to finding the most efficient and reliable approach to adding images to our database.

Architecture

Conclusion

By integrating AI-powered visual search capabilities into your image search API, you can elevate the user experience and drive engagement in your application. In this tutorial, we’ve demonstrated how to harness the power of Flask, Celery, ChromaDB, and OpenCLIP to build a sophisticated image search solution that meets the demands of today’s users.