Only Available at Ontrack: IBM Storwize Data Recovery
A client recently experienced a remote ransomware attack that resulted in Ontrack engineers being presented with one of their most extraordinary data recovery efforts to date: restoring 120 damaged HDDs within an IBM SVC Storwize v7000 system…with no backup to rely on.
After noting the critical nature of the project, Ontrack’s data recovery experts proceeded with a comprehensive process to potentially restore the deleted data:
The Ontrack team was able to join the client’s team and scope the data loss event and the storage systems impacted. Based on the scope set forth, Ontrack was able to determine a project plan, set timing expectations and determine costs for data recovery.
Data recovery engineers used Ontrack’s proprietary tools to analyze the disks, determine the likely array configuration as well as detect indications of Windows storage space and VMware storage virtual machines.
- R&D Simulation and Software Programming
After the initial diagnosis, engineers analyzed a minimal hardware setup of the IBM SVC Storwize v7000 as a means of detecting the layout of on-disk structures used to map Raid Arrays, including managed disks, SVC pools, virtual disks, and physical disks (=LUN).
Ontrack then began to work closely with the client’s IT department to get the hardware running on a new setup of the IBM SVC Storwize v7000 system.
Simulations were performed to see if the client’s environment could be recreated on the live hardware and if any structure could be found to possibly reconstruct the deleted data. All findings regarding the simulated structures were compared to the structures on the original hard drives.
A positive prediction was formed based on the comparison of the structures, and Ontrack was able to move forward with the creation and modification of proprietary tools to extract functional storage systems and proceed with successful SAN system data recovery.
With an enormous challenge ahead of them, Ontrack’s data recovery experts performed extensive research on IBM’s proprietary software which resulted in engineers modifying their recovery tools to allow for the virtual rebuild of the DRAID that was in use on the IBM system.
Figuring out the distribution patterns for DRAID proved to be the most
intricate part of the recovery process, given that all of the data sitting on the DRAID6 MDisk was combined with a number of other MDisks and dynamically allocated multiple levels of both VDisks and Dynamic Disks.
Once the array was virtually rebuilt, the Ontrack team was able to virtually rebuild the volumes, transforming them into 1,152 devices in order to display the overall layout of available data contained within to generate reports for the client and complete the IBM Storwize data recovery.
When the client initially introduced the issue, there was little hope for full (if any) recovery given the complex nature of the IBM Storwize data storage system. However, thanks to the diligence of our engineers, an unprecedented Ontrack data recovery solution for all IBM Storwize systems is now solely available via Ontrack.
Ontrack Performs Emergency Raid 5 Data Recovery
A multinational client had fallen victim to the disappearance of crucial financial data and office files, with no backup data plan in place. After realizing the urgency of the data loss situation, they immediately contacted Ontrack’s expert engineers to help with recovery.
Given the importance of the lost data, the client expedited all media over the weekend via courier to an Ontrack engineer who happened to be on call. Per client request, Ontrack provided project updates every 15 minutes as a data recovery plan was being prepared.It was quickly discovered that the client’s hard drives used the Raid 5 system, and in order to have a successful data recovery, engineers would have to virtually rebuild the drives to guarantee perfect quality of the data found on their VMFS VMware ESX 5.1 volumes.
Through a fantastic show of teamwork and diligence, Ontrack’s data recovery engineers were successfully able to bring all drives back to nearly 100% of their data volume within 6 days after rebuilding the system; a testament to the company’s unmatched expertise and dedication to solving even the most complex data challenges. The client expressed enthusiastic gratitude for such quick data recovery and was excited to have their company back up and running in just under a week with the help of Ontrack’s emergency recovery service.
Damaged Mac Laptop? Ontrack to the Rescue.
A customer contacted Ontrack’s experts in need of help with Mac data recovery for a laptop that had been dropped and accidentally run over by a truck, causing its battery to combust.
Ontrack’s recovery experts surveyed the Mac device damage and implemented a 3-step recovery process for data extraction which included:
- Decontamination – A thorough process in which all contaminants that adversely affected the damaged Mac laptop’s operability were removed.
- Micro-Soldering – A small soldering tool was used to replace a pool of damaged capacitors and resistors which were located near the burned battery.
- Diagnostics – The MacBook was placed in a special diagnostic mode where both Apple-provided OEM tools and Ontrack’s own proprietary tools were used to mount the internal storage device as a volume onto lab machines and process the copy out.
Once the process was complete, Ontrack was able to successfully extract and verify all data.
Some accidents aren’t easy to account for. Ontrack’s recovery experts can be trusted to provide satisfactory service when it comes to Mac recovery, as well as data recovery for similar devices, whether damage is due to an occasional drop and break or a truck that comes out of nowhere.
A Deep Dive for Dell/EMC Isilon Data
A client inadvertently ran a command that deleted critical files on a Dell/EMC Isilon storage array containing 270 disks totaling 2 petabytes (PB).
After having the drives in question flown in, Ontrack’s recovery engineers began an in-depth evaluation and determined that the JIT development team would be needed to assist in data recovery from this version of Isilon. Within eight weeks of working nights, weekends, and holidays to develop a proper solution, an initial set of data was delivered to the client.
While the client was ecstatic to receive the recovered files, they also requested that subsequent tasks be done to prove that no stone was left unturned in their recovery efforts, per regulatory requirements. Ontrack’s JIT development team complied by combing through the 270 drives in search of files that matched regulatory requirements and implementing a process that would search, copy, and deduplicate specific files that were found across each disk.
After months of conducting a thorough secondary search, more than 300 million files and 13 terabytes (TB) of PDF and JPG files were produced.
This project exemplifies the ability of Ontrack’s team of recovery experts to go above and beyond to meet the client’s needs. Our team of engineers is not only well-equipped to restore files lost from Dell/EMC Isilon storage, but they are also prepared to help your company provide the proper proof of data recovery according to regulatory standards.
Ransomware Recovery – Veeam Agent for Windows
A health care customer was affected by a ransomware attack that not only targeted their server data, but also “Veeam Agent for Windows” backups located on an external HDD. Their IT / managed services provider agreement did not include regular off-site backups, so this was the only copy of the data that existed.
The customer was able to send the affected HDD to Ontrack, where an image of the drive was taken to preserve the original state of the customer media.
Ontrack engineers assessed the damage to the affected Veeam backup files and identified that partial recovery would be possible as the files had not been fully encrypted, meaning there was a chance that some data could be recovered from within the files. However, it was determined that the version of Veeam used was newer than Ontrack could support with current tools and required development assistance.
With a global engineering presence, as well as internal development teams that maintain and improve our proprietary tools, Ontrack was able to research, develop and implement support for the new version quickly. In fact, much of the time-intensive research required had already been completed for similar jobs seen in our European offices. This allowed Ontrack developers to quickly and efficiently modify tools to the level required to be able to support this restore scenario. Rather than building out a fully-fledged tool, Ontrack engineers were able to use the improved version of the tools to complete searches for required structures to allow them to manually rebuild internal components critical to the recovery of data from within the file.
IBM server with a RAID 5 comprised of 5 SCSI hard drives failed
They arrived in the Ontrack office and clean room facility at approximately 3pm in the afternoon, and within two hours the client was given confirmation that the data was recoverable. “From speaking to the client, I knew that his business would be in serious trouble if data was not restored quickly” explains Adrian Briscoe. “Due to our ‘follow the sun’ support capabilities, our local engineers were able to image the hard drives and then send the images to teams in Europe and the US where they pieced the RAID back together. The critical data was then uploaded to the FTP and made available for the client to download.”
The recovery was 100% successful and every single file that was on the server was able to be recovered. When asked about the Ontrack service, the client said “communication was great I would say, there was response and we didn’t have to sit around the resort waiting for communication or having to chase updates ourselves – we were informed about what was happening every step of the way.” They continued on to state that Ontrack scored “10 out of 10 – communication, speed, response, recovery. I would definitely recommend their services to other businesses.”
Ontrack supports an IT service provider to ensure its end-customer can access legacy backup tapes
An IT service provider had to guarantee access to the legacy backup tapes of a new end-customer from the insurance industry.
The end-customer needed to have access to data on a large number of 3592 and 3592/JA tapes for a period of five years, in order to comply with data retention and governance regulations. The backup tapes had been created using Tivoli Storage Manager, however the end customer did not want to incur the costs of maintaining this environment for a five year period for infrequent backup tape restore requests.
Ransomware attacks server – backup tapes erased
A ransomware attack of a company server encrypted the Microsoft Dynamics 365 data and demanded payment. Recent backups of the server were stored on multiple LTO-6 backup tapes, which had been erased by the malware.
After assessing the extent of the ransomware attack, Ontrack representatives identified the company’s backup tapes as the best option for data recovery—even though the malware had erased them. 23 LTO-6 backup tapes from the backup library were sent to the Ontrack office in Böblingen, Germany. Working in conjunction with the R&D department in the United Kingdom, Ontrack developed a custom solution to recover the data from the erased backup tapes.
Ontrack was able to restore 46TB of data from 18 of the LTO-6 tapes. Due to the type of attack on the tapes, Ontrack had to repair the logical damage, shipping the data and tapes separately back to the customer.
Ransomware VBK Recoveries on Tape - Server & NAS Systems
The attacked volume was originally also used to back up data to LTO8 tapes at regular intervals. Most of these backup tapes were also in the tape library at the time of the incident and were quickly formatted by the attackers. However, the customer was able to save an original unformatted tape with a fairly old backup date, which was then completely restored to the now empty Windows volume with a total of 6 TB. Only then was Ontrack commissioned to examine data recovery options. The HP server DL380 with the 55 3TB hard disks were transported to Ontrack in Böblingen Germany.
During the diagnosis, a large number of the searched VEEAM vbk files were successfully found on the Windows volume with Ontrack Tools and 27 records were extracted according to a priority list. The restore of the LTO8 tape partially overwrote some of the data sets and damaged the backup files.
A large part of the data could still be repaired and extracted in several steps.
Later on, 19 significantly older LTO8 quick formatted tape backups were successfully recovered too.The attack also affected numerous European sub offices of the customer. Here were predominantly QNAP NAS systems in use which had stored virtual VMs under VMware, including backup VMs that were partially deleted or internally reformatted with another file system. Ontrack was also able to successfully restore complete backup data in 90% of the seven cases ordered.
Accidental Deletion of Virtual Machines Results in 15TB Lost.
An accidental deletion at a large wireless provider causes a massive loss of email databases.
The wireless carrier stored all of their Microsoft® Exchange databases spread across 24 separate 2TB LUNs on an EMC® VNX 5400 using VMware® virtual machines. It was also set up so each database had a mirror copy on a different LUN. All of the virtual machines were accidentally deleted resulting in the loss of email for the entire company.
Ontrack is assisted by NetApp’s technology to solve a ransomware infection.
Ontrack Successfully Recovers Data from iPhone 5.
Hospital databases rescued from ransomware.
Ontrack Provides Database and Backup Restores After a Flood.
Missing Dell® EqualLogic™ LUNs Recovered via Remote Data Recovery.
24 terabytes of data recovered from RAID 6 array with newly developed toolset.
Unfortunately, the system failed to rebuild the data after two hard disk drives failed resulting in the loss of access to 24 terabytes of highly critical data. The organization approached the experts at Ontrack for help.
The Ontrack developers quickly created the tools needed to improve the success of the recovery. After only a few hours, the first virtual machine was rebuilt allowing for the extraction of the Exchange databases to be returned to the customer. The team continued to rebuild all of the critical virtual machines until the client’s email was back in production. At the end of the project, a total of 15TB of data was recovered with minimal downtime for the client.
Data loss in paradise.
When Uprising Beach Resort in Fiji experienced a problem with their RAID configured server, they knew that they had to act quickly.
Uprising Beach Resort’s IBM server with a RAID 5 comprising of 5 SCSI hard drives failed. When the hard drives arrived at the Brisbane Cleanroom it was found that one of the mirrored operating drives had failed with internal mechanical faults. The second OS was also reporting bad sectors. Ontrack took the three hard drives that constituted the data volume, imaged the hard drives and rebuilt the RAID.
The client originally contacted VMware for support.
According to Alfred Christoffersen, manager of Uprising Beach Resort, the server contained every single record of their operations since the day that they opened. “We did have an external backup” explains Christoffersen, “but it was a month out of date. Restoring it was not an option because it would not have had the last months’ worth of data and reservations - we could have reentered a lot of the missing data manually but it would have taken weeks and we didn’t have that kind of time.” Uprising Beach Resort contacted Datec Fiji Limited who Christoffersen described as being Fiji’s biggest and best IT company. Datec Fiji Limited are a part of Ontrack’s Authorised Partner network, and after working on the server for 8 hours, they referred the case to Ontrack. “I called Ontrack after their business hours and got through to an automated voice service. I left a message and within 30 minutes, Adrian Briscoe, the Managing Director, called me back” says Christoffersen.