Fastembed Colbert Reranker #2482

xoaryaa · 2025-10-30T07:10:06Z

Related Issues

fixes feat: support bi-encoder models in TransformerSimilarityRanker haystack#8245

Proposed Changes:

Feature: Add FastembedColbertReranker — a ColBERT late-interaction (MaxSim) reranker inside the FastEmbed integration.

New component: haystack_integrations.components.rankers.fastembed.FastembedColbertReranker
Uses FastEmbed’s LateInteractionTextEmbedding to encode query and documents (token-level).
Implements ColBERT MaxSim scoring: sum_q max_d (sim(q_i, d_j)) with configurable similarity={"cosine","dot"} and optional L2 normalization.
CPU-friendly ONNX backend via FastEmbed; no new core Haystack deps.
Batching support (batch_size) and optional token limits (max_query_tokens, max_doc_tokens).
Stable sorting with deterministic tie-break; parameter validation and robust warm-up tolerant to minor FastEmbed kwargs drift.
Small README note + CHANGELOG entry for the FastEmbed integration.
Example script: examples/fastembed_colbert_reranker.py.

Why: Cross-encoder rankers don’t cover late-interaction/bi-encoder models like ColBERT v2. This adds a first-class ColBERT reranker over ~100–500 retrieved candidates.

How did you test it?

Unit tests
- Pure math checks for MaxSim and L2 normalization.
- Parameter validation (invalid similarity, batch_size, top_k).
- Deterministic ranking & top_k via monkeypatched encoders (no model download required).
Manual verification
1. pip install -e integrations/fastembed
2. Run python examples/fastembed_colbert_reranker.py
  - First run downloads the ColBERT model via FastEmbed (cached afterward).
  - Verified that documents mentioning “late interaction/ColBERT” rank at the top.

Notes for the reviewer

Lives entirely under integrations/fastembed/src/... (no changes to core Haystack).
Warm-up handles minor API differences in FastEmbed by retrying without unknown kwargs.
Intended usage: rerank ~100–500 candidates from a first-stage retriever (BM25 or dense).

Example import path:

from haystack_integrations.components.rankers.fastembed import FastembedColbertReranker

…U, cosine/dot)

…p; add tests

…, CHANGELOG, example

CLAassistant · 2025-10-30T07:10:13Z

All committers have signed the CLA.

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

…rig_idx) Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

…ert-reranker Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in> # Please enter a commit message to explain why this merge is necessary, # especially if it merges an updated upstream into a topic branch. # # Lines starting with '#' will be ignored, and an empty message aborts # the commit.

xoaryaa · 2025-10-30T09:22:03Z

@anakin87 The “Check Integration Format” job is failing due to hyphens found in unrelated files in the repo. My FastEmbed changes don’t add any hyphenated directories under ./integrations/ and pass local checks.

These existing files contain hyphens in their filenames:

./llama_stack/examples/llama-stack-with-tools.py
./opensearch/docker-compose.yml
./amazon_bedrock/tests/test_files/haystack-logo.png
./weaviate/docker-compose.yml
./elasticsearch/docker-compose.yml

anakin87 · 2025-10-30T09:24:43Z

@anakin87 The “Check Integration Format” job is failing due to hyphens found in unrelated files in the repo. My FastEmbed changes don’t add any hyphenated directories under ./integrations/ and pass local checks.

These existing files contain hyphens in their filenames:

./llama_stack/examples/llama-stack-with-tools.py

./opensearch/docker-compose.yml

./amazon_bedrock/tests/test_files/haystack-logo.png

./weaviate/docker-compose.yml

./elasticsearch/docker-compose.yml

The problem lies in this git submodule. Please remove it.

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa · 2025-10-31T08:42:35Z

@anakin87 can you check once and LMK if you want changes?

anakin87

I initially started reviewing this PR, but I noticed that it diverges significantly from other components in this package/repository.

Please take a look at other components and tests in the Fastembed-Haystack integration for inspiration and to align this implementation with the existing patterns.

I'll be away for a few days, but I can pick up the review again once I'm back if there are updates.

anakin87 · 2025-10-31T16:35:34Z

integrations/fastembed/CHANGELOG.md

+## [Unreleased]
+
+### 🚀 Features
+- Add `FastembedColbertReranker`: ColBERT late-interaction (MaxSim) reranker for 1st-stage candidates (100–500). (#2482)
+
+### 📚 Documentation
+- FastEmbed docs: “ColBERT Reranker” section with quickstart + pipeline example. (#2482)
+- Integration README: add bullet for `FastembedColbertReranker` and links. (#2482)
+
+### 🧪 Testing
+- Unit tests for MaxSim math, parameter validation, stable tie-break, and monkeypatched ranking (no model download). (#2482)
+
+### 🧹 Chores
+- Parameter validation, robust warm-up for fastembed API drift, stable sorting on score ties. (#2482)
+
+


this is automatically generated and not meant to be modified

anakin87 · 2025-10-31T16:36:24Z

integrations/fastembed/README.md

+## Components
+
+- **FastembedTextEmbedder / FastembedDocumentEmbedder / Sparse embedders** – embedding components for dense and sparse retrieval (see docs).
+- **FastembedRanker** – cross-encoder reranker (see docs).
+- **FastembedColbertReranker** – *new* ColBERT late-interaction (MaxSim) reranker for reordering 100–500 candidates on CPU via ONNX.  
+  → See the docs for installation and examples.
+
+### Documentation
+
+- Integration page: https://haystack.deepset.ai/integrations/fastembed
+- Component docs:
+  - Fastembed embedders: https://docs.haystack.deepset.ai/docs/fastembedtextembedder
+  - FastembedRanker (cross-encoder): https://docs.haystack.deepset.ai/docs/fastembedranker
+  - **FastembedColbertReranker (ColBERT)**: *[link to your new docs section]*
+
+### Install
+
+```bash
+pip install fastembed-haystack
+


Suggested change

## Components

- **FastembedTextEmbedder / FastembedDocumentEmbedder / Sparse embedders** – embedding components for dense and sparse retrieval (see docs).

- **FastembedRanker** – cross-encoder reranker (see docs).

- **FastembedColbertReranker** – *new* ColBERT late-interaction (MaxSim) reranker for reordering 100–500 candidates on CPU via ONNX.

→ See the docs for installation and examples.

### Documentation

- Integration page: https://haystack.deepset.ai/integrations/fastembed

- Component docs:

- Fastembed embedders: https://docs.haystack.deepset.ai/docs/fastembedtextembedder

- FastembedRanker (cross-encoder): https://docs.haystack.deepset.ai/docs/fastembedranker

- **FastembedColbertReranker (ColBERT)**: *[link to your new docs section]*

### Install

```bash

pip install fastembed-haystack

README should be stay minimal. We'll add this information to docs.

anakin87 · 2025-10-31T16:36:51Z

integrations/fastembed/haystack_integrations/components/rankers/fastembed/colbert_utils.py

@@ -0,0 +1,4 @@
+from __future__ import annotations


remove this file

anakin87 · 2025-10-31T16:39:13Z

integrations/fastembed/src/haystack_integrations/components/rankers/fastembed/__init__.py

@@ -1,3 +1,3 @@
-from .ranker import FastembedRanker


the existing ranker should not be removed from here.

also, please not introduce new init files: integrations/fastembed/src/haystack_integrations/init.py, etc.
This package is a namespace package, so these init files should be avoided.

xoaryaa added 4 commits October 29, 2025 15:10

feat(...)

4404f59

feat(fastembed): implement ColBERT MaxSim reranking via FastEmbed (CP…

7ba9f2e

…U, cosine/dot)

chore(fastembed): parameter validation, stable sorting, robust warm-u…

fdc0de4

…p; add tests

feat(fastembed): add ColBERT late-interaction reranker; tests, README…

b555998

…, CHANGELOG, example

xoaryaa requested a review from a team as a code owner October 30, 2025 07:10

xoaryaa requested review from anakin87 and removed request for a team October 30, 2025 07:10

github-actions bot added integration:fastembed type:documentation Improvements or additions to documentation labels Oct 30, 2025

xoaryaa added 5 commits October 30, 2025 12:43

feat(fastembed): add ColBERT Reranker

c4567f9

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

chore(fastembed): satisfy Ruff EM101/EM102 and other lint rules

bf4b2d1

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

fix(fastembed): deterministic tie-break on equal scores (descending o…

fb77993

…rig_idx) Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

chore(fastembed): 3.9-friendly typing (Optional/List)

6548693

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

chore: remove accidental submodule 'haystack-core-integrations'

6d30a65

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from a302ad2 to 6d30a65 Compare October 30, 2025 09:34

fix lint: remove unused imports in integrations/fastembed

b909590

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from fcf4a21 to b909590 Compare October 30, 2025 09:54

fix: python 3.9 on linux

7198b83

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from 770e875 to 7198b83 Compare October 30, 2025 10:20

fix: trailing whitespaces

fdb1d9e

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from 5663922 to fdb1d9e Compare October 30, 2025 10:30

fix: python 3.9 on linux

3cf8a37

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from 9b4e1bf to 3cf8a37 Compare October 31, 2025 05:42

chore: silence UP045 on Optional for py3.9

217d213

Signed-off-by: Arya Tayshete <avtayshete_b21@et.vjti.ac.in>

xoaryaa force-pushed the feat/fastembed-colbert-reranker branch from 0d0e1e2 to 217d213 Compare October 31, 2025 06:01

anakin87 requested changes Oct 31, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fastembed Colbert Reranker #2482

Fastembed Colbert Reranker #2482

Uh oh!

xoaryaa commented Oct 30, 2025 •

edited by sjrl

Loading

Uh oh!

CLAassistant commented Oct 30, 2025 •

edited

Loading

Uh oh!

xoaryaa commented Oct 30, 2025

Uh oh!

anakin87 commented Oct 30, 2025

Uh oh!

xoaryaa commented Oct 31, 2025

Uh oh!

anakin87 left a comment

Uh oh!

anakin87 Oct 31, 2025

Uh oh!

anakin87 Oct 31, 2025

Uh oh!

anakin87 Oct 31, 2025

Uh oh!

anakin87 Oct 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fastembed Colbert Reranker #2482

Are you sure you want to change the base?

Fastembed Colbert Reranker #2482

Uh oh!

Conversation

xoaryaa commented Oct 30, 2025 • edited by sjrl Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related Issues

Proposed Changes:

How did you test it?

Notes for the reviewer

Uh oh!

CLAassistant commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

xoaryaa commented Oct 30, 2025

Uh oh!

anakin87 commented Oct 30, 2025

Uh oh!

xoaryaa commented Oct 31, 2025

Uh oh!

anakin87 left a comment

Choose a reason for hiding this comment

Uh oh!

anakin87 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

anakin87 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

anakin87 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

anakin87 Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

xoaryaa commented Oct 30, 2025 •

edited by sjrl

Loading

CLAassistant commented Oct 30, 2025 •

edited

Loading