Our monitoring shows routing failures have decreased to normal levels, we expect all cluster connectivity to have been restored.
We've completed the scheduled maintenance and will continue normal monitoring of the platform.
Posted 14 days ago. Jul 08, 2019 - 04:07 UTC
We've successfully replaced the Zookeeper instance which was scheduled for retirement and have verified the ensemble state. We're currently working on restarting cluster orchestration services within us-east-1 which will allow deployment changes to proceed again. We're currently seeing an increase in cluster routing failures which will cause some deployments to have become unavailable. Requests to Kibana and Elasticsearch will be impacted by these routing failures.
We expect cluster connectivity to be fully restored within the next 5 minutes and will update this incident with more details as they are uncovered.
Posted 14 days ago. Jul 08, 2019 - 03:52 UTC
We're stopping some internal services the ensure the Zookeeper replacement goes smoothly. Creating, updating or deleting deployments within us-east-1 may take longer than expected or may time out whilst these services are disabled.
Any failed deployment changes can be retried after the Zookeeper maintenance is complete.
Posted 14 days ago. Jul 08, 2019 - 03:25 UTC
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted 14 days ago. Jul 08, 2019 - 03:00 UTC
We will be replacing a voting member of the Zookeeper ensemble on Monday, 8th of July, as a preventative measure. AWS have notified us of an instance retirement which happens to be one of our ensemble members. This happens occasionally and we have done this maintenance many times before without issue.
We are not expecting any problems during this replacement but as always if you notice any issues during that time please contact Elastic Support.
Posted 20 days ago. Jul 02, 2019 - 02:48 UTC
This scheduled maintenance affected: AWS N. Virginia (us-east-1) (Cluster Connectivity: AWS us-east-1) and Cluster Management (Cluster Orchestration).