Only Available at Ontrack: IBM Storwize Data Recovery
A client recently experienced a remote ransomware attack that resulted in Ontrack engineers being presented with one of their most extraordinary data recovery efforts to date: restoring 120 damaged HDDs within an IBM SVC Storwize v7000 system…with no backup to rely on.
After noting the critical nature of the project, Ontrack’s data recovery experts proceeded
with a comprehensive process to potentially restore the deleted data:
The Ontrack team was able to join the client’s team and scope the data loss event and the storage systems impacted. Based on the scope set forth, Ontrack was able to determine a project plan, set timing expectations and determine costs for data recovery.
Data recovery engineers used Ontrack’s proprietary tools to analyze the disks, determine the likely array configuration as well as detect evidence” of Storage Spaces and VMware structures within the data.
- R&D Simulation and Software Programming
After the initial diagnosis, engineers analyzed a minimal hardware setup of the IBM SVC Storwize v7000 as a means of detecting the layout of on-disk structures used to map Raid Arrays.
Ontrack then began to work closely with the client’s IT department to get hardware running on a new setup of the IBM SVC Storwize v7000 system.
Simulations were performed to see if the client’s environment could be recreated on the live hardware and if any structure could be found to possibly reconstruct the deleted data. All findings regarding the simulated structures were compared to the structures on the original hard drives.
A positive prediction was formed based on the comparison of the structures, and Ontrack was able to move forward with the creation and modification of proprietary tools to extract functional storage systems and proceed with successful SAN system data recovery.
- Data Recovery
With an enormous challenge ahead of them, Ontrack’s data recovery experts performed extensive research on IBM’s proprietary software which resulted in engineers modifying their recovery tools to allow for the virtual rebuild of the DRAID
that was in use on the IBM system.
Figuring out the distribution patterns for DRAID proved to be the most intricate part
of the recovery process, given that all of the data sitting on the DRAID6 MDisk was combined with a number of other MDisks and dynamically allocated multiple levels of both VDisks and Dynamic Disks.
Once the array was virtually rebuilt, the Ontrack team was able to virtually rebuild the volumes. After the rebuild the team was able to generate reports to show the directory structures and layout of the recoverable data to the customer. The team then completed the IBM Storwize data recovery.
When the client initially introduced the issue, there was little hope for full (if any) recovery given the complex nature of the IBM Storwize data storage system. However, thanks to the diligence of our engineers, an unprecedented Ontrack data recovery solution for all IBM Storwize systems is now solely available via Ontrack.
Ontrack Performs Emergency Raid 5 Data Recovery
A multinational client had fallen victim to the disappearance of crucial financial data and office files, with no backup data plan in place. After realizing the urgency of the data loss situation, they immediately contacted Ontrack’s expert engineers to help with recovery.
Given the importance of the lost data, the client expedited all media over the weekend via courier to an Ontrack engineer who happened to be on call. Per client request, Ontrack provided project updates every 15 minutes as a data recovery plan was being prepared.It was quickly discovered that the client’s hard drives used the Raid 5 system, and in order to have a successful data recovery, engineers would have to virtually rebuild the drives to guarantee perfect quality of the data found on their VMFS VMware ESX 5.1 volumes.
Through a fantastic show of teamwork and diligence, Ontrack’s data recovery engineers were successfully able to bring all drives back to nearly 100% of their data volume within 6 days after rebuilding the system; a testament to the company’s unmatched expertise and dedication to solving even the most complex data challenges. The client expressed enthusiastic gratitude for such quick data recovery and was excited to have their company back up and running in just under a week with the help of Ontrack’s emergency recovery service.
Damaged Mac Laptop? Ontrack to the Rescue.
A customer contacted Ontrack’s experts in need of help with Mac data recovery for a laptop that had been dropped and accidentally run over by a truck, causing its battery to combust.
Ontrack’s recovery experts surveyed the Mac device damage and implemented a 3-step recovery process for data extraction which included:
- Decontamination – A thorough process in which all contaminants that adversely affected the damaged Mac laptop’s operability were removed.
- Micro-Soldering – A small soldering tool was used to replace a pool of damaged capacitors and resistors which were located near the burned battery.
- Diagnostics – The MacBook was placed in a special diagnostic mode where both Apple-provided OEM tools and Ontrack’s own proprietary tools were used to mount the internal storage device as a volume onto lab machines and process the copy out.
Once the process was complete, Ontrack was able to successfully extract and verify all data.
Some accidents aren’t easy to account for. Ontrack’s recovery experts can be trusted to provide satisfactory service when it comes to Mac recovery, as well as data recovery for similar devices, whether damage is due to an occasional drop and break or a truck that comes out of nowhere.
A Deep Dive for Dell/EMC Isilon Data
A client inadvertently ran a command that deleted critical files on a Dell/EMC Isilon storage array containing 270 disks totaling 2 petabytes (PB).
After having the drives in question flown in, Ontrack’s recovery engineers began an in-depth evaluation and determined that the JIT* development team would be needed to assist in data recovery from this version of Isilon. Within eight weeks of working nights, weekends, and holidays to develop a proper solution, an initial set of data was delivered to the client.
While the client was ecstatic to receive the recovered files, they also requested that subsequent tasks be done to prove that no stone was left unturned in their recovery efforts, per regulatory requirements. Ontrack’s JIT development team complied by combing through the 270 drives in search of files that matched regulatory requirements and implementing a process that would search, copy, and deduplicate specific files that were found across each disk.
After months of conducting a thorough secondary search, more than 300 million files and 13 terabytes (TB) of PDF and JPG files were produced.
This project exemplifies the ability of Ontrack’s team of recovery experts to go above and beyond to meet the client’s needs. Our team of engineers is not only well-equipped to restore files lost from Dell/EMC Isilon storage, but they are also prepared to recover your data according to regulatory standards in place.
*JIT stands for "Just In Time". It means that our development team can make changes on our tools on the fly to accommodate custom problems or changes to third party file systems and others. It's a development capability that sets Ontrack apart from other data recovery companies that just buy fixed data recovery tools off the shelf.
Ransomware Recovery – Veeam Agent for Windows
A health care customer was affected by a ransomware attack that not only targeted their server data, but also “Veeam Agent for Windows” backups located on an external HDD. Their IT / managed services provider agreement did not include regular off-site backups, so this was the only copy of the data that existed.
The customer was able to send the affected HDD to Ontrack, where an image of the drive was taken to preserve the original state of the customer media.
Ontrack engineers assessed the damage to the affected Veeam backup files and identified that partial recovery would be possible as the files had not been fully encrypted, meaning there was a chance that some data could be recovered from within the files. However, it was determined that the version of Veeam used was newer than Ontrack could support with current tools and required development assistance.
With a global engineering presence, as well as internal development teams that maintain and improve our proprietary tools, Ontrack was able to research, develop and implement support for the new version quickly. In fact, much of the time-intensive research required had already been completed for similar jobs seen in our European offices. This allowed Ontrack developers to quickly and efficiently modify tools to the level required to be able to support this restore scenario. Rather than building out a fully-fledged tool, Ontrack engineers were able to use the improved version of the tools to complete searches for required structures to allow them to manually rebuild internal components critical to the recovery of data from within the file.
Ransomware attacks server – backup tapes erased
A ransomware attack of a company server encrypted the Microsoft Dynamics 365 data and demanded payment. Recent backups of the server were stored on multiple LTO-6 backup tapes, which had been erased by the malware.
After assessing the extent of the ransomware attack, Ontrack representatives identified the company’s backup tapes as the best option for data recovery—even though the malware had erased them. 23 LTO-6 backup tapes from the backup library were sent to the Ontrack office in Böblingen, Germany. Working in conjunction with the R&D department in the United Kingdom, Ontrack developed a custom solution to recover the data from the erased backup tapes.
Ontrack was able to restore 46TB of data from 18 of the LTO-6 tapes. Due to the type of attack on the tapes, Ontrack had to repair the logical damage, shipping the data and tapes separately back to the customer.
Accidental Deletion of Virtual Machines Results in 15TB Lost.
An accidental deletion at a large wireless provider causes a massive loss of email databases.
The wireless carrier stored all of their Microsoft® Exchange databases spread across 24 separate 2TB LUNs on an EMC® VNX 5400 using VMware® virtual machines. It was also set up so each database had a mirror copy on a different LUN. All of the virtual machines were accidentally deleted resulting in the loss of email for the entire company.
New tools made recovery from highly specialized EMC® Isilon® big data storage possible.
The “lost files” were mostly raw data gathered from chemical analysis in ongoing laboratory research work. The firm, together with EMC support, was able to recover 90 percent of the data using a standard rebuild process. To recover the desperately needed remaining files, Ontrack was contacted by the customer for help. Ontrack worked with the EMC support in order to get detailed information on the situation.
Ontrack is assisted by NetApp’s technology to solve a ransomware infection.
Hospital databases rescued from ransomware.
Ontrack Provides Database and Backup Restores After a Flood.
Missing Dell® EqualLogic™ LUNs Recovered via Remote Data Recovery.
Ontrack recovers over 230 million files from Commvault database.
Accidental wipe command brings down critical production database server.
Four terabytes of data recovered from flood damaged HP EVA SAN
A flooded data center left a client’s servers and storage systems partially submerged in water.
At the center of the damage was a HP Storage Works EVA (Enterprise Virtual Array) 6000 containing business critical SQL database files as well as employee file shares. The EVA sustained substantial physical damage due to the flood water preventing access to the data. The severity of the damage from the flood was increased when an attempt was made to access the data by powering on the drives that were still wet. The customer contacted HP Support for help and they handed the project over to Ontrack.
The SAN consisted of 80 hard disk drives which were divided into 2 EVA disk groups; in total there were 18 virtual RAID volumes consisting of both VRAID1 and VRAID5.
A HP EVA system is fully virtualized and has a unique way to write data which adds to the complexity of any data recovery effort. It works with disk groups and virtual disks instead of normal RAID sets and logical drive volumes. The disk groups consist of physical drives organized in a proprietary manner. LUNs or Virtual disks (vDisks) in an EVA are then distributed over all of the installed HDDs.
Due to the physical damage, all of the drives were sent to one of Ontrack´s cleanroom facilities to be assessed.
Once the 80 drives were decontaminated and cleaned, 55 were found to be fully recoverable. 25 of the drives had severe water damage and were not recoverable. To regain access to the data on the damaged drives, the engineers needed to research how the EVA RAID and file system was structured. After the engineers were able to map the disk groups and determine how the vDisks were distributed, they had to rebuild the whole EVA system. To recover the data included in the vDisks, the R&D team and its software developers had to create completely new tools to extract the data. Once the development was complete, Ontrack engineers virtually assembled the disk groups and virtually rebuilt the vDisks which allowed access to the underlying file systems. The file systems were virtually repaired and the data extracted.
After extensive development, reengineering and recovery work the project successfully ended.
With the newly created tools the data recovery specialists were able to recover four terabytes of sensitive data including the critical SQL database files. In all, approximately 86 percent of the total data lost was recovered. With the HP EVA SAN data mapping knowledge gained and the integration of the newly developed tools, Ontrack is able to quickly recover data from all models of the HP Enterprise EVA storage systems.