New tools made recovery from highly specialized EMC® Isilon® big data storage possible.
Feb 20, 2020
The Situation
The pharmaceutical company lost almost 4 million files of highly critical research and development data when moving files within the Isilon storage system.
The “lost files” were mostly raw data gathered from chemical analysis in ongoing laboratory research work. The firm, together with EMC support, was able to recover 90 percent of the data using a standard rebuild process. To recover the desperately needed remaining files, Ontrack was contacted by the customer for help. Ontrack worked with the EMC support in order to get detailed information on the situation.
The Isilon IQ 6000x, designed for big data, consisted of four nodes with 15 terabytes of data storage in total with 500 gigabyte hard drives installed.
It was set up as a single volume where all of the data was striped across the disks. Unlike a traditional RAID system, Isilon systems are built on the concept of a huge data lake where all data is stored and managed inside one data pool. To manage the system and this “data lake” a unique file system was created by Isilon called OneFS. In this case one of the four nodes inside the system experienced a kernel panic and several disks showed multiple errors. EMC was able to gather most of the files with a rebuild before a consistency check showed that several disks had errors. To recover the missing data, the file system had to be analyzed by the Ontrack team to find out how the data was laid out in the whole storage system.
The Solution
To find out how the data originally was distributed over the disks and determine the data mapping, Ontrack s own R&D department developed a brand new set of tools unique to the OneFS.
With these tools, an existing OneFS volume can be analyzed in depth and missing or faulty data file structures inside the “big” data lake can be discovered even more quickly. With the now gathered information on how the data structure of the Isilon system was setup, the engineers were finally able to rebuild and recover the missing data.
The Resolution
In the end, Ontrack engineers were able to recover almost all of the missing files with only a couple of bad files due to damaged encryption.
The client was delighted that Ontrack handled such a complex recovery so quickly. None of the confidential data was compromised and the solution provided by Ontrack was cost-effective and completed within a very short timeframe. With the newly developed tools for this case, Ontrack is currently the only data recovery service provider with the ability to recover data from Isilon storage systems.