We had a cluster issue on the LAX nodes this morning. We now have a temporary fix and all services are online. Some features may not be working as we repair the cluster.
Update 11:50 am.
After hours of trying to repair a corrupted cluster config for HA with no success. We are backing up the servers on nodes L.A2-LA7 and will reinstall the system on those nodes. The proxmox update that came through overnight caused this problem. We will work as fast as we can to get everyone back online asap. We will extend those affected by this due dates 5 days. Please dont open tickets asking how much longer. Because when we have to stop and answer tickets it will take long for use to restore everything.
Update 2:30 pm
We will be restoring servers back to LAX1 shortly.
Update 6:50 pm
We are still restoring servers to LAX 1. We should have been done already but everyone has to open tickets.
