Chroma

What it does

Chroma is an open-source embedding database designed for building LLM (Large Language Model) applications that require memory. It allows you to store documents and their associated embeddings, along with metadata, enabling semantic search over those documents. Chroma facilitates the creation of Retrieval-Augmented Generation (RAG) pipelines by providing a local or self-hosted vector database solution.

When to use it

• Building RAG applications where an LLM needs access to external knowledge.
• When you require a locally hosted and managed vector database.
• For prototyping LLM application components in notebooks.
• Performing semantic search over collections of documents.
• Storing embeddings alongside associated metadata for filtering and context.

Key capabilities

• Collection Creation: Create named collections to organize your data.
• Document Addition: Add documents, their corresponding embeddings (either auto-generated or custom), and associated metadata.
• Semantic Search/Querying: Perform similarity searches using query texts and filter results based on metadata.
• Metadata Filtering: Filter search results based on complex criteria defined within the where clause.
• Custom Embedding Functions: Integrate with embedding models like OpenAI's text-embedding-3-small.

Example prompts

• "Create a Chroma collection named 'my_collection'."
• "Add two documents to the 'my_collection' along with metadata indicating their source."
• "Perform a similarity search on 'my_collection' for documents related to 'machine learning tutorial', returning the top 5 results."

Tips & gotchas

• You’ll need to install Chroma using pip install chromadb or npm install chromadb.
• To use custom embeddings, you may need an API key (e.g., for OpenAI).
• Chroma is designed as a self-hosted solution; consider alternatives like Pinecone if managed cloud scaling is required.