Summary

Between 16:30 and 21:40 UTC, customers in the Frankfurt region experienced service issues. New projects could not be created starting around 16:30 UTC, and from 17:45 UTC most projects became unreachable. Service was fully restored by 21:40 UTC.

Impact

• Affected environment: Frankfurt only
• Customer experience: requests to Meilisearch returned errors, and instances were not accessible. New project creation also failed.
• Duration: about 3.5 hours of downtime for most customers.
• Data:
  • Analytics data lost between 2025-12-04 and 2025-12-08 since we had to stop the service to mitigate the incident. Capabilities will be resumed on 2025-12-08.
  • Indexed data and configuration related to Meilisearch instances remained intact.

Root cause

Our Frankfurt gateway, which routes traffic to customer projects, stopped accepting configuration updates because of an invalid route definition. This invalid route came from a rare edge case during the rollout of the new version of our Beta Analytics capability. In that case, a duplicated analytics configuration was generated for a small number of projects. The gateway treated this as invalid and then rejected all routing updates, leading to a large outage in the Frankfurt environment.

Resolution

To restore service, we removed the analytics-related gateway plugins from Frankfurt and regenerated all routes. This allowed the gateway to accept configuration again. Routes were progressively recreated and service returned between 21:10 and 21:40 UTC.

Prevention and next steps

We have already:

• Fixed the configuration edge case that could create duplicated analytics settings
• Enforced uniqueness checks when generating new routes
• Removed the gateway plugin overrides that contributed to the issue

We are now working on:

• Enabling stronger validation on route creation to block invalid configurations before they reach the gateway
• Adding proactive alerts when the gateway control plane fails to apply updates
• Improving observability and error reporting around routing changes