24 terabytes of data recovered from RAID 6 array with newly developed toolset.
Feb 20, 2020
The situation
A large UK Government organization had to learn the hard way that even RAID 6 arrays, known for their reliability, are not 100 percent impervious to hardware failure.
Unfortunately, the system failed to rebuild the data after two hard disk drives failed resulting in the loss of access to 24 terabytes of highly critical data. The organization approached the experts at Ontrack for help.
The client was using an Infortrend® EonStor RAID 6 array to run a range of business applications.
They experienced failures on two 2TB SATA drives in the system and replaced both of the failed drives. Even though it was a RAID 6, when the second drive failed, it also caused the array to fail. After the replacement drives were installed, the system failed to rebuild, which meant the business critical data was not accessible. The engineers at Ontrack virtually rebuilt the RAID 6 array with the two missing disks in order to recover the missing data. Due to the manufacturer-unique algorithm in a RAID 6 array, a rebuild of the secondary parity stripe from this specific system had not been completed before.
The solution
Due to the rebuild failure, missing data from two failed drives had not been replicated onto the new drives when the new drives were added.
Being the client was using a RAID 6 array, the missing information could be rebuilt from the existing data on the other drives. The challenge of a RAID 6 recovery is locating the data to be restored; each RAID controller uses different algorithms and a concept called parity to create a RAID 6 configuration. To locate and access the missing data, Ontrack engineers developed a solution to support the Infortrend controller type. The engineering team utilized the specialized toolset to recover and rebuild all 24 terabytes of missing data from the RAID array.
Ontrack assembled a team of three data recovery engineers and two developers in order to provide the fastest possible recovery.
The resolution
The Ontrack developers quickly created the tools needed to improve the success of the recovery. After only a few hours, the first virtual machine was rebuilt allowing for the extraction of the Exchange databases to be returned to the customer. The team continued to rebuild all of the critical virtual machines until the client’s email was back in production. At the end of the project, a total of 15TB of data was recovered with minimal downtime for the client.