API/behavior changes

New pooling parameter for huggingFace embedders

• The embedder object that can be set in the PATCH /indexes/{:indexUid}/settings and PATCH /indexes/{:indexUid}/settings/embedders routes, is modified as follows:
  • A new parameter pooling is added to the embedder object and allows to override the pooling method of huggingFace embedders.
    • When a huggingFace embedders transforms a text into an embedding, it starts by transforming the text into tokens, then it computes an embedding for each of these tokens. Lastly, it computes a single sentence embedding from the token embeddings by using a pooling method.
    • Models typically indicate in their configuration the expected pooling method.
  • pooling is a string with values "useModel", "forceMean" or "forceCls".
    • "useModel" : fetch the pooling method from the model configuration
    • "forceMean": always use mean pooling
    • "forceCls" always use CLS pooling
  • pooling is optional and defaults to "useModel"
  • For backward compatibility, huggingFace embedders that were created in a previous version of Meilisearch, and imported using a dump or the dumpless upgrade feature will have pooling set to "forceMean" , as this was the behavior of these embedders in previous versions of Meilisearch.
  • pooling is only available for embedders with source huggingFace
  • Changing the value of pooling always triggers a full reindexing.

Composite embedder

• A new experimental feature compositeEmbedders is added to the /experimental-features route.
• The embedder object that can be set in the PATCH /indexes/{:indexUid}/settings and PATCH /indexes/{:indexUid}/settings/embedders routes, is modified as follows:
  • A new value for the source parameter is allowed: "composite" . This value is selectable when the compositeEmbedders feature is set to true.
  • new parameters are added to the embedder object, available when the source is "composite":
    • searchEmbedder: an object whose keys are the same as an embedder object. The embedder it describes will be used at search time.
    • indexingEmbedder: an object whose keys are the same as an embedder object. The embedder it describes will be used at search time.
    • These parameters can be set when the compositeEmbedders feature is set to true.
  • Meilisearch checks that some test embeddings generated by the searchEmbedder and the indexingEmbedder are “similar enough”: Meilisearch computes the angular distance for both embeddings in each test case, and checks that that distance is < 0.01.

This feature allows using different embedders at search and indexing time, which can be used to optimize the embedder to each use case:

• Using a remote embedder for indexing in bulk, as remote embedders provide the highest bandwidth (embeddings/s)
• Using a local embedder for answering search queries, as local embedders provide the lowest latency (time to first embedding)

Example

• Mixing a local model (huggingFace) to a Hugging Face inference endpoint

{
    "embedders": {
        "text": {
            "source": "composite",
            "searchEmbedder": {
                "source": "huggingFace", // locally computed embeddings using a model from the Hugging Face Hub
                "model": "baai/bge-base-en-v1.5",
                "revision": "a5beb1e3e68b9ab74eb54cfd186867f64f240e1a"
            },
            "indexingEmbedder": {
                "source": "rest", // remotely computed embeddings using Hugging Face inference endpoints
                "url": "<https://URL.endpoints.huggingface.cloud>",
                "apiKey": "hf_XXXXXXX",
                "documentTemplate": "Your {{doc.template}}",
                "request": {
                    "inputs": [
                        "{{text}}",
                        "{{..}}"
                    ]
                },
                "response": [
                    "{{embedding}}",
                    "{{..}}"
                ]
            }
        }
    }
}

• Mixing a local model (huggingFace source) with a Cloudflare AI worker

{
    "embedders": {
        "text": {
            "source": "composite",
            "searchEmbedder": {
                "source": "huggingFace",
                "model": "baai/bge-base-en-v1.5",
                "revision": "a5beb1e3e68b9ab74eb54cfd186867f64f240e1a",
                "pooling": "forceMean"
            },
            "indexingEmbedder": {
                "source": "rest",
                "url": "<https://api.cloudflare.com/client/v4/accounts/ACCOUNT_NUMBER/ai/run/@cf/baai/bge-base-en-v1.5>",
                "apiKey": "API_KEY",
                "documentTemplate": "Your {{doc.template}}",
                "request": {
                    "text": [
                        "{{text}}",
                        "{{..}}"
                    ]
                },
                "response": {
                    "result": {
                        "data": [
                            "{{embedding}}",
                            "{{..}}"
                        ]
                    }
                }
            }
        }
    }
}

Telemetry

• The existing embedders.sources can now contain the value composite

Error handling

• The "composite" source cannot be nested inside of a composite embedder, trying to set searchEmbedder.source or indexingEmbedder.source to "composite" will return a 400 invalid_settings_embedder
  • Sample error message:
    • .embedders.test.searchEmbedder.source: Source composite is not available in a nested embedder