We've rolled back to existing the existing infrastructure after the orchestration layer failed to re-establish a quorum when the leader was removed. We will reschedule this maintenance after investigating the factors which caused this failure.
Posted Sep 02, 2019 - 05:25 UTC
Verifying
Rolling back to the existing infrastructure has allowed the orchestration layer to stabilise. Our monitoring indicates the cloud console is responsive for all users again.
Our monitoring does not indicate any impact on cluster availability and we don't believe any incoming data was lost during the incident.
We'll continue to monitor the platform, and will delay the original maintenance tasks until we've investigated a root cause. We'll update this incident within the next hour.
Posted Sep 02, 2019 - 04:51 UTC
Update
We've experienced an unexpected failure whilst undertaking this instance replacement. The Cloud console will be down for all users.
We're currently working on rolling back to the existing infrastructure and will work to restore functionality. We'll update this incident within the next 30 minutes or as the situation changes.
Posted Sep 02, 2019 - 04:27 UTC
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Sep 02, 2019 - 04:00 UTC
Scheduled
We will be replacing a voting member of the Zookeeper ensemble on Monday 2nd September, as a preventative measure. AWS have notified us of an instance retirement which happens to be one of our ensemble members. This happens occasionally and we have done this maintenance many times before without issue. We are not expecting any problems during this replacement but as always if you notice any issues during that time please contact Elastic Support.
Posted Aug 29, 2019 - 02:10 UTC
This scheduled maintenance affected: Global services (Cloud console) and AWS N. Virginia (us-east-1) (Deployment orchestration (Create/Edit/Restart/Delete): AWS us-east-1).