API/behavior changes
Describe routes and behavior changes
Add batchStrategy field to batch objects
Motivation
Autobatching enough tasks is very important for indexing performance in Meilisearch.
However, which tasks can be batched together by the autobatcher is an implementation detail, and the inner workings of the autobatcher are not always trivial, resulting in unexpected performance pitfalls for some users.
This task is about making it clear what causes the autobatcher to stop batching tasks. This will allow users (or at least, Meilisearch engineers) to easily troubleshoot indexing performance issues that might be related to autobatching patterns.
Usage
The batch object now contains a new batchStrategy field that displays a text representation of how the autobatcher created this batch of tasks.
- This field is meant for debugging purposes, and not to be used in code.
- SDKs should not create structured data out of the returned strings. Declare the field as a string in the response object.
- in future versions, the texts might change, new texts might appear, some might never be returned again
Examples
- “batched all enqueued tasks”: all enqueued tasks at the time of the batch creation are part of the batch, so no additional task could be taken (because there is none)
- “batched all enqueued tasks for index
{index}": there were more enqueued tasks, but they were for another index
- “stopped after reaching configured batch limit of {task_limit} tasks”: Meilisearch was configured to batch at most
N task, which it did
- “stopped before task with id {id} because it is a settings change in a batch of document operations”: We no longer batch settings and document operations
- “stopped before task with id {id} because it is a deletion by filter in a batch of document operations”: We cannot batch deletion by filter and document operations
- “stopped after task with id {id} which deletes the index”
- …
Telemetry
No new telemetry
Error handling