rssLink RSS for all categories
 
icon_orange
icon_green
icon_green
icon_orange
icon_red
icon_green
icon_green
icon_orange
icon_red
icon_red
icon_green
icon_green
icon_green
icon_blue
icon_orange
icon_red
icon_green
icon_green
icon_orange
icon_red
icon_green
icon_red
icon_green
icon_blue
icon_orange
icon_green
icon_green
icon_green
icon_green
icon_blue
icon_green
icon_green
 

FS#40183 — PrivateSQL P19

Attached to Project— Web Hosting / CloudDB
Incident
P19 / Private SQL
CLOSED
100%
Nous rencontrons un problème sur notre infrastructure à P19. Les PrivateSQL hébergés sur ce datacentre sont impactés.

---

We have an issue on our infrastructure on P19. The PrivateSQL hosted on this datacentre are impacted.
Date:  Friday, 23 August 2019, 23:52PM
Reason for closing:  Done
Comment by OVH - Friday, 23 August 2019, 07:51AM

Linked issues:
- http://travaux.ovh.net/?do=details&id=40178

Ceph cluster is being recovered (2 instances of data is needed at least for the Ceph cluster to accept Ios).
When the Ceph cluster will be UP, we will restart the PCI infrastructure.
When the PCI infrastructure will be UP, we will restart the services.


Comment by OVH - Friday, 23 August 2019, 08:17AM

Ceph cluster is still being recovered (about 70%).


Comment by OVH - Friday, 23 August 2019, 08:46AM

Ceph cluster is still being recovered (about 82%).


Comment by OVH - Friday, 23 August 2019, 10:21AM

Ceph cluster is still being recovered (about 94%)


Comment by OVH - Friday, 23 August 2019, 12:10PM

Ceph cluster is back online, servers are recovering.


Comment by OVH - Friday, 23 August 2019, 12:30PM

Almost all server are back online.


Comment by OVH - Friday, 23 August 2019, 21:14PM

Servers impacted by this issue have a high probability of crashing until they have been rebooted in a specific way: the services and the OS have to restart softly, but the "shell" of the VM (qemu process...) has to be rebuilt.
So we will reboot the PrivateSQL of P19. A 1 to 5 minutes downtime is expected.


Comment by OVH - Friday, 23 August 2019, 22:00PM

50% done


Comment by OVH - Friday, 23 August 2019, 23:52PM

Completed. The platform is stable.