Confluent Metrics API experiencing high latencies

Incident Report for Confluent Cloud

Resolved

We have fixed the issue, and are now monitoring to be sure it does not recur. As of May 27 19:00 UTC, customers should see normal operations with Confluent systems.
Posted May 29, 2023 - 13:00 UTC

Update

The fix put in place has been confirmed to help mitigate the issue. We will continue to investigate the root cause while we monitor the performance.
Posted May 28, 2023 - 00:21 UTC

Monitoring

We have put a temporary fix in place which has helped mitigate the issue. We will continue to investigate the root cause while we monitor the performance.
Posted May 27, 2023 - 22:49 UTC

Investigating

We are continuing to see intermittent high latency and error rates for Metrics API. We are investigating the issue and will provide an update soon.
Posted May 27, 2023 - 18:41 UTC

Monitoring

We have implemented a fix as of 16:02 UTC. We are monitoring the systems closely and will provide an update soon.
Posted May 27, 2023 - 17:34 UTC

Investigating

The intermittent high latency and error rates for Metrics API have returned since 10:00 UTC. We are continuing to investigate this issue and are working to resolve it.
Posted May 27, 2023 - 15:53 UTC

Update

Latencies are back to normal from the time the issue was mitigated (00:15 UTC). We are continuing to monitor the systems closely.
Posted May 27, 2023 - 05:55 UTC

Monitoring

We have made the fix and incident has been mitigated as of 00:15 UTC. We are monitoring the systems closely and will provide an update soon.
Posted May 27, 2023 - 01:38 UTC

Investigating

We are currently investigating an issue where Metrics API is intermittently experiencing higher than normal latency and error rates. This issue started at 19:13 UTC.
Posted May 26, 2023 - 21:00 UTC
This incident affected: Confluent Cloud.