Ontrack supports an IT service provider to ensure its end-customer can access legacy backup tapes

The client

An IT service provider had to guarantee access to the legacy backup tapes of a new end-customer from the insurance industry.

The situation

The end-customer needed to have access to data on a large number of 3592 and 3592/JA tapes for a period of five years, in order to comply with data retention and governance regulations. The backup tapes had been created using Tivoli Storage Manager, however the end customer did not want to incur the costs of maintaining this environment for a five year period for infrequent backup tape restore requests.

The solution

Working with the IT service provider, Ontrack came up with a cost effective and efficient solution to allow continued direct access to the end-customer’s backup tapes over the retention period.

The first step was to conduct a Proof of Concept (PoC) where the end-customer sent 5 tapes to Ontrack. The purpose of the PoC was to ensure full support for the client’s tape and backup software combination and confirm the scope of the project and timelines.

Following the PoC, Ontrack extracted the Tivoli Storage Manager catalogue into a standard database format. This allowed the IT service provider to identify the location of folders or specific files and send the relevant tapes for restore to Ontrack when required. The data from the tape restore could then be delivered back to the end-customer on an encrypted USB drive or via secure FTP for low volume or urgent restore requests.

The data restores and maintenance of the required infrastructure are covered by a multi-year tape service agreement allowing for a pre-defined number of tape restores, with incremental restores available upon request.

The resolution

The IT service provider was able to offer a bespoke tape processing service to their end-customer which meant that they no longer had to sustain the significant costs of maintaining a Tivoli Storage Manager environment while still having the ability to extract data from the tapes as and when required.

The multi-year tape service agreement for restoring backup tapes on demand meant that the services were tailored to the client’s needs and ensured greater predictability in expenditure planning and budgeting. The service agreement also allows requests for, and delivery of, tape restores to be processed quickly and efficiently and avoid administrative delays. Under the service agreement Ontrack is able to guarantee the tape extraction capabilities over the full period of the contract.

Ransomware attacks server – backup tapes erased

The client

The situation

A ransomware attack of a company server encrypted the Microsoft Dynamics 365 data and demanded payment. Recent backups of the server were stored on multiple LTO-6 backup tapes, which had been erased by the malware. 

The solution

After assessing the extent of the ransomware attack, Ontrack representatives identified the company’s backup tapes as the best option for data recovery—even though the malware had erased them. 23 LTO-6 backup tapes from the backup library were sent to the Ontrack office in Böblingen, Germany. Working in conjunction with the R&D department in the United Kingdom, Ontrack developed a custom solution to recover the data from the erased backup tapes.

The resolution

Ontrack was able to restore 46TB of data from 18 of the LTO-6 tapes. Due to the type of attack on the tapes, Ontrack had to repair the logical damage, shipping the data and tapes separately back to the customer.

Ransomware VBK Recoveries on Tape - Server & NAS Systems

The client

The situation

The attacked volume was originally also used to back up data to LTO8 tapes at regular intervals. Most of these backup tapes were also in the tape library at the time of the incident and were quickly formatted by the attackers. However, the customer was able to save an original unformatted tape with a fairly old backup date, which was then completely restored to the now empty Windows volume with a total of 6 TB. Only then was Ontrack commissioned to examine data recovery options. The HP server DL380 with the 55 3TB hard disks were transported to Ontrack in Böblingen Germany.

The solution

During the diagnosis, a large number of the searched VEEAM vbk files were successfully found on the Windows volume with Ontrack Tools and 27 records were extracted according to a priority list. The restore of the LTO8 tape partially overwrote some of the data sets and damaged the backup files. 

The resolution

A large part of the data could still be repaired and extracted in several steps.

Later on, 19 significantly older LTO8 quick formatted tape backups were successfully recovered too.The attack also affected numerous European sub offices of the customer. Here were predominantly QNAP NAS systems in use which had stored virtual VMs under VMware, including backup VMs that were partially deleted or internally reformatted with another file system. Ontrack was also able to successfully restore complete backup data in 90% of the seven cases ordered.

Accidental Deletion of Virtual Machines Results in 15TB Lost.

The client

The situation

An accidental deletion at a large wireless provider causes a massive loss of email databases.

The wireless carrier stored all of their Microsoft® Exchange databases spread across 24 separate 2TB LUNs on an EMC® VNX 5400 using VMware® virtual machines. It was also set up so each database had a mirror copy on a different LUN. All of the virtual machines were accidentally deleted resulting in the loss of email for the entire company.

The solution

The client originally contacted VMware for support.

When VMware’s support team realized the extent of the data loss, they immediately contacted Ontrack for assistance. An Ontrack® Data Recovery™ Engineer assessed the situation and determined that a remote data recovery would be the fastest and most cost-effective option for the customer. The engineer fully explained the process and the client agreed and connected the LUNs to Ontrack’s proprietary Remote Data Recovery system for 24 hour emergency service. Ontrack assembled a team of three data recovery engineers and two developers in order to provide the fastest possible recovery.

The resolution

The Ontrack developers quickly created the tools needed to improve the success of the virtual machine data recovery. After only a few hours, the first virtual machine was rebuilt allowing for the extraction of the Exchange databases to be returned to the customer. The team continued to rebuild all of the critical virtual machines until the client’s email was back in production. At the end of the project, a total of 15TB of data was recovered with minimal downtime for the client.

Ontrack is assisted by NetApp’s technology to solve a ransomware infection.

The client

The situation

A single user’s laptop at a large pharmaceutical company was infected with CryptoLocker ransomware.

This malware encrypts the user’s files and withholds the encryption key until you pay the ransom amount. The laptop was connected to the corporate network which allowed the malware to infect a CIFS volume which was set up as a file share on a NetApp FAS. The malware was able to infiltrate the file share and encrypt the majority of the files. The IT team was not notified of the infection until after the backup retention period had expired, meaning that the backup contained only encrypted data. The total impact resulted in inaccessible data on:

■ 46 drives

■ 1 aggregate

■ 1 volume infected on a RAID-DP

To perform the recovery, the aggregate needed to be taken offline, which affected 17 volumes in total.

The solution

The customer brought their 46 drives into our New Jersey lab for evaluation and Ontrack engineers got to work on a solution.

The engineering team from Ontrack:

■ Virtually rebuilt the RAID groups which were strewn across 10 different shelves

■ Virtually rebuilt the aggregate

■ Virtually rebuilt the critical volume

An additional challenge on this recovery was that the aggregate was in use for two weeks after the incident occurred which resulted in some data being overwritten.

The resolution

Ontrack was able to virtually rebuild the volume containing the CIFS share and encrypted data.

Leveraging NetApp’s proprietary OS (OnTap) and file system (WAFL), Ontrack engineers used multiple consistency points to “walk back” in time to find and merge unencrypted copies of the critical data to return to the customer. This type of ransomware recovery is only possible on storage like NetApp’s FAS because of the way the data is stored on the volume.

Ontrack Successfully Recovers Data from iPhone 5.

The client

The situation

Customer Testimonial

“I was taking video of my [then] 7 month-old in the bathtub, and of course, ended up dropping my phone in the tub! It was completely soaked. I tried all the usual tricks like putting it in rice, etc., but nothing worked. So we contacted a couple of data recovery companies for quotes, and ended up sending it to a company other than Ontrack. We were told initially by the other data recovery company that there was an 85-90% chance of recovering the data. We paid a deposit to have it sent to this other company, who we later found out sent the phone to Canada to have it diagnosed. After all that, they came back and said the phone was unrecoverable. We ended up losing the deposit, and having them send the phone back to us. We decided to send it to Ontrack for a second opinion. We wanted to work with a well-known company that absolutely knew what they were doing if we were going to take another chance. We had Ontrack evaluate the phone and they were able to recover all the photos! We were so happy. There were over 2,500 memories that we never would have been able to recreate. We couldn’t have gotten them back without Ontrack.” - Stacy Holm

Case Details

■ iPhone 5 recovery needed due to water damage to the PCB.

■ Ontrack’s expert engineers were able to perform repairs to the PCB utilizing the company’s proprietary data recovery methods.

■ 17GB of data was recovered from the phone which consisted of approximately 2,500 files.

The solution

With the implementation of newer, more secure technology, it is more difficult to recover data from the iPhone 5.

Ontrack had to make extensive repairs to the device. Even after the physical restoration was complete, there were still logical failures within the device. Despite these logical faults, Ontrack was able to recover all of the requested data from the phone. The entire recovery only took two days and Ontrack succeeded where the competition had failed.

The resolution

Hospital databases rescued from ransomware.

The client

The situation

A ransomware attack with the ‘Locky’ virus had severe effects for a large German hospital.

Many servers at the hospital were paralyzed by the virus, limiting operations. Uninfected servers became affected during the panic when their power supplies were disconnected while they were still in operation. In highly complex virtualized storage systems, an improper power shutdown can result in unexpected issues. This was the case for a Dell EqualLogic PS6500ES storage array with a total of 148 professional grade 100-gigabyte hard drives. After the hospital’s IT staff and Dell’s technical support were unable to solve the problem, the specialists at Ontrack were called in to help. All of the drives were delivered to the data recovery laboratory in Germany where they were assessed.

The Dell EqualLogic PS6500ES system typically contains multiple hard drives arranged on 16 or 48 hard drive shelves and are connected together to form RAID 5 or RAID 50 systems (sub-arrays). These sub-arrays in turn are connected to ’members,’ with one or more members belonging to a logical unit (a group). LUNs are created and stored in the group, then fragmented and distributed over all members and sub-arrays. They are ‘tracked’ by a map, which in turn distributes itself to the members or to the various subarrays when it gets proportionally large. In this case our specialists discovered of those seven shelves with 148 hard drives, three shelves with 80 hard drives contained the LUN with the Oracle databases needed. However, many of the links (mappings) of the data fragments (which were distributed over all hard disks) were either corrupted or no longer available, so arranging the fragments proved to be a very difficult task. The mapping of an EqualLogic PS system is also encoded in a specific logic, so the links here aren’t easy to locate either.

The solution

To map the links, specialist engineers from other Ontrack offices developed new software tools to specifically solve the logic and corruption problems regarding the RAID and the LUN mapping.

With the help of the new tools, the engineers were able to recreate the RAID 5 and RAID 50 systems as well as display the LUN. Within this LUN a virtual hard disk (a VMDK file) was located, in which an NTFS file system with two Oracle databases were hidden. Two file layers had to be identified and recovered within the LUN before these databases could be finally exported.

The resolution

The team of ransomware data recovery engineers from several Ontrack offices were finally able to successfully extract and recover the required databases and send the data by courier to the client.

The hospital was very pleased with the mediation support from Dell to Ontrack and the fact that they finally had all their important data available again. In addition, the tools developed for this project can be used again in upcoming data recovery cases of Dell EqualLogic PS Array systems, significantly reducing future data recovery times.

Ontrack Provides Database and Backup Restores After a Flood.

The client

The situation

Dell Equallogic™, Storage Area Network, VMware ESX base and RAID 10 backup server.

A flash flood in Baden-Wurttemberg, Germany in Spring 2016 permeated the walls of a server room in a hobby and art supplies store, severely affecting the IT system.

Vast amounts of water flooded into the server room affecting two Dell devices: an EqualLogic SAN with 96 hard drives and a RAID 10 backup server with 12 hard drives.

The storage capacity of the SAN hard drives was between 300 and 700 gigabytes and the backup server contained 24 terabytes of data. About 30 terabytes of data were lost when the SAN volume and the iSCSI connections for the SAN in a VMware ESX Server were damaged. Due to the importance of the data and that most of it contained critical customer information, an emergency recovery was arranged with Ontrack. Several SAN virtualized LUNs running in a VMware environment were prioritized as critical information, with one of them storing a particularly important Oracle database that needed to be recovered as soon as possible. The client also needed to recover two additional LUNs with important data, as well as all the data on the backup server.

The solution

After working with the client to assess the data loss and prioritize the data that needed recovering, it was determined that the hard drives in both systems needed to be processed simultaneously. All the hard drives were picked up by Ontrack and delivered to the data recovery lab in Boblingen, Germany.

Upon arrival, the hard drives were initially processed in a cleanroom environment to safely remove dirt and inspect the full extent of the damage. The data on each hard drive and its server location were also documented at this time.

Fortunately, the drives in the SAN had no mechanical problems and could be read properly. However, some of the drives from the backup server were faulty and these had to be processed further in the cleanroom in order to extract copies of the data.

The SAN data recovery was very complex, as the water damage interrupted the connection to the VMware ESX Server and the power supply while in operation. This meant that the mapping links to the EqualLogic SAN and the LUNs in the VMFS datastores (as well as the Oracle database and other files) were heavily corrupted.

To perform the recovery, Ontrack’s proprietary data recovery software tools were required to reconstruct the system and the file structures in order to get to the actual files. However, reassembling the Dell backup server was relatively easy since it hardly experienced any data corruption.

The resolution

The engineers from Ontrack’s Boblingen data recovery lab succeeded in reconstructing the main points of both affected devices so that the data could be accessed again, including the critical Oracle database and the full backup.

Overall, the customer was very satisfied with the work of the recovery experts from Ontrack. The recovery efforts were also mediated by the Dell Support Team so the data could be restored into a new storage system, as the information required was urgent and needed as soon as the recovery was complete.

Missing Dell® EqualLogic™ LUNs Recovered via Remote Data Recovery.

The client

The situation

A large municipal event center in the US lost data on a Dell® EqualLogic™ iSCSI SAN configured with in a RAID 50 running VMware® ESXi™ 5.5.

VMware snapshots filled up the datastore causing the system to crash. The customer attempted to delete one of the snapshots, but after four hours of processing without success, they had to give up. Working with VMware support, they were able to get the VMware ESXi 5.5 host to boot, but were missing critical data from six of the iSCSI LUNs. This system was unique because it was using the EqualLogic LUNs as raw device mappings (RDMs) attached to the guest instead of the traditional virtual disks (VMDKs) on VMFS datastores.

The solution

The event center called Ontrack at noon on a Saturday for emergency service.

Highly-trained data recovery engineers connected remotely to the EqualLogic LUNs using their proprietary remote data recovery (RDR) solution and started assessing the damage. During the evaluation, the engineers were able to locate the snapshots containing the missing data and virtually apply them to RDMs. Once the snapshots had been applied, the Ontrack engineering team was able to access the underlying NTFS volume, virtually repair the NTFS corruption, and extract the data.

The resolution

Within 12 hours Ontrack was able to reunite the customer with the lost data which totaled over 250,000 files (-250GB of data).

“I was most impressed with the customer service I received from Ontrack throughout the data recovery process, the speed at which all the data was restored and the fact that during the entire restore process we were able to have our live environment up and running.”

24 terabytes of data recovered from RAID 6 array with newly developed toolset.

The client

The situation

A large UK Government organization had to learn the hard way that even RAID 6 arrays, known for their reliability, are not 100 percent impervious to hardware failure.

Unfortunately, the system failed to rebuild the data after two hard disk drives failed resulting in the loss of access to 24 terabytes of highly critical data. The organization approached the experts at Ontrack for help.

The client was using an Infortrend® EonStor RAID 6 array to run a range of business applications.

They experienced failures on two 2TB SATA drives in the system and replaced both of the failed drives. Even though it was a RAID 6, when the second drive failed, it also caused the array to fail. After the replacement drives were installed, the system failed to rebuild, which meant the business critical data was not accessible. The engineers at Ontrack virtually rebuilt the RAID 6 array with the two missing disks in order to recover the missing data. Due to the manufacturer-unique algorithm in a RAID 6 array, a rebuild of the secondary parity stripe from this specific system had not been completed before.

The solution

Due to the rebuild failure, missing data from two failed drives had not been replicated onto the new drives when the new drives were added.

Being the client was using a RAID 6 array, the missing information could be rebuilt from the existing data on the other drives. The challenge of a RAID 6 recovery is locating the data to be restored; each RAID controller uses different algorithms and a concept called parity to create a RAID 6 configuration. To locate and access the missing data, Ontrack engineers developed a solution to support the Infortrend controller type. The engineering team utilized the specialized toolset to recover and rebuild all 24 terabytes of missing data from the RAID array.

Ontrack assembled a team of three data recovery engineers and two developers in order to provide the fastest possible recovery.

The resolution

The Ontrack developers quickly created the tools needed to improve the success of the recovery. After only a few hours, the first virtual machine was rebuilt allowing for the extraction of the Exchange databases to be returned to the customer. The team continued to rebuild all of the critical virtual machines until the client’s email was back in production. At the end of the project, a total of 15TB of data was recovered with minimal downtime for the client.

Data loss in paradise.

The client

The situation

When Uprising Beach Resort in Fiji experienced a problem with their RAID configured server, they knew that they had to act quickly.


Uprising Beach Resort’s IBM server with a RAID 5 comprising of 5 SCSI hard drives failed. When the hard drives arrived at the Brisbane Cleanroom it was found that one of the mirrored operating drives had failed with internal mechanical faults. The second OS was also reporting bad sectors. Ontrack took the three hard drives that constituted the data volume, imaged the hard drives and rebuilt the RAID.

The client originally contacted VMware for support.

According to Alfred Christoffersen, manager of Uprising Beach Resort, the server contained every single record of their operations since the day that they opened. “We did have an external backup” explains Christoffersen, “but it was a month out of date. Restoring it was not an option because it would not have had the last months’ worth of data and reservations - we could have reentered a lot of the missing data manually but it would have taken weeks and we didn’t have that kind of time.” Uprising Beach Resort contacted Datec Fiji Limited who Christoffersen described as being Fiji’s biggest and best IT company. Datec Fiji Limited are a part of Ontrack’s Authorised Partner network, and after working on the server for 8 hours, they referred the case to Ontrack. “I called Ontrack after their business hours and got through to an automated voice service. I left a message and within 30 minutes, Adrian Briscoe, the Managing Director, called me back” says Christoffersen.

The solution

Ontrack assembled a team of three data recovery engineers and two developers in order to provide the fastest possible recovery.

The Ontrack developers quickly created the tools needed to improve the success of the recovery. After only a few hours, the first virtual machine was rebuilt allowing for the extraction of the Exchange databases to be returned to the customer. The team continued to rebuild all of the critical virtual machines until the client’s email was back in production. At the end of the project, a total of 15TB of data was recovered with minimal downtime for the client.

The resolution

► The recovery was 100% successful and every single file that was on the server was able to be recovered.

When asked about the service, Christoffersen said “communication was great I would say, there was response and we didn’t have to sit around the resort waiting for communication or having to chase updates ourselves - we were informed about what was happening every step of the way.” Christoffersen continues on to state that Ontrack scored “10 out of 10 - communication, speed, response, recovery. I would definitely recommend their services to other businesses.”

German service partner turns to Ontrack to recover data for customer facing loss of business and personal data from external RAID drives.

The client

The situation

A self-employed marketing professional used an Apple Macintosh desktop with two back-up drives to store an extensive image library, ongoing client projects and personal data including photographs.

In common with many marketing professionals, the client used an Apple Macintosh desktop to work on multiple design projects for clients, backing them up to two external RAID drives.

Both drives were attached directly to the Apple Mac but there was no mechanism in place to perform automatic backups. When one day an unidentified issue meant that the drives failed, the client believed that all of their data was lost.

As a marketing freelancer, the client was under pressure to deliver design and photography jobs to their customers on time and to budget. The system failure meant that the client’s entire business was at risk.

The solution

The client approached their IT service provider for help to recover the lost data and they in turn sent the job to Ontrack.

The engineering team at Ontrack evaluated the two external devices and made the surprising discovery that the second external USB device consisted of two internal hard drives.

The client, like so many small and home businesses, had taken advantage of the availability of low-cost external devices to extend the storage capability of their Apple Mac when they ran out of space. Having installed the first one terabyte drive, they implemented a special Span Set to attach the two terabyte drive - with a hidden internal stripe set inside.

No emphasis had been placed on how the external storage should be configured, how often manual backups should be made or whether a larger, dedicated storage drive with automated backups would have been preferable to a more complex RAID array setup than it appeared to be in the first run.

It wasn’t immediately clear to the engineers at Ontrack what had gone wrong with the client’s set-up, but since there was no physical damage to the drives the problem was most likely caused by a power failure or an issue with the cables used to connect the drives to the computer.

In any event, the complexity involved in the way in which the system was backing up data meant that while the set-up had appeared to work well for the client for a certain length of time, it was always at risk of sudden failure and data loss. While Ontrack’s service partner was able to help with an initial survey of the problem drives, it did not have the specialist engineering skills and resources to rebuild the file structures and retrieve the data - and could have potentially made the situation worse.

The engineers at Ontrack were able to rebuild and restore all of the information lost by the client: a total of 423,064 files and almost two trillion bytes of data.

The recovery was 100% successful and the drives were reconfigured correctly so that the same problem would not happen again. Some of the most common reasons for failure of backups to supply lost data are:

■ The external hard drives used by the majority of companies are only connected on an occasional basis, hence backup is not automated and instead performed on demand

■ The computer was not switched on during the scheduled backup nor configured to perform at a different time

■ The backup software failed

■ The backup ran out of destination space

■ The backup profile did not cover all of the device requiring backup

■ File was lost before the scheduled backup

The resolution

Having seen so many cases of critical data loss, Ontrack recommends the following tips to ensure backup success:

■ Take the time to invest in a backup solution and set up an automated backup schedule

■ Ensure backups are running regularly in accordance with the determined schedule

■ Check backup reports for error indications or failure

■ Test backups on a regular basis to ensure data has been accurately captured and files are intact

When things do go wrong, calling a trusted data recovery provider to identify and assess your data recovery options can increase the likelihood of successfully recovering your data.

Four terabytes of data recovered from flood damaged HP EVA SAN.

The client

The situation

A flooded data centre left a client’s servers and storage systems partially submerged in water.

At the centre of the damage was a HP Storage Works EVA (Enterprise Virtual Array) 6000 containing business-critical SQL database files as well as employee file shares. The EVA sustained substantial physical damage due to the flood water preventing access to the data. The severity of the damage from the flood was increased when an attempt was made to access the data by powering on the drives that were still wet. The customer contacted HP Support for help and they handed the project over to Ontrack.

Technical Details

The SAN consisted of 80 hard disk drives which were divided into 2 EVA disk groups; in total there were 18 virtual RAID volumes consisting of both VRAID1 and VRAID5.

A HP EVA system is fully virtualised and has a unique way to write data which adds to the complexity of any data recovery effort. It works with disk groups and virtual disks instead of normal RAID sets and logical drive volumes. The disk groups consist of physical drives organised in a proprietary manner. LUNs or Virtual disks (vDisks) in an EVA are then distributed over all of the installed HDDs.

The solution

Due to the physical damage, all of the drives were sent to one of Ontrack s cleanroom facilities to be assessed.

Once the 80 drives were decontaminated and cleaned, 55 were found to be fully recoverable. 25 of the drives had severe water damage and were not recoverable. To regain access to the data on the damaged drives, the engineers needed to research how the EVA RAID and file system was structured. After the engineers were able to map the disk groups and determine how the vDisks were distributed, they had to rebuild the whole EVA system. To recover the data included in the vDisks, the R&D team and its software developers had to create completely new tools to extract the data. Once the development was complete, Ontrack engineers virtually assembled the disk groups and virtually rebuilt the vDisks which allowed access to the underlying file systems. The file systems were virtually repaired and the data extracted.

The resolution

After extensive development, reengineering and recovery work the project successfully ended.

With the newly created tools the data recovery specialists were able to recover four terabytes of sensitive data including the critical SQL database files. In all, approximately 86 per cent of the total data lost was recovered.

With the HP EVA SAN data mapping knowledge gained and the integration of the newly developed tools, Ontrack is able to quickly recover data from all models of the HP Enterprise EVA storage systems.

Important hospital databases rescued after ransomware attack.

The client

The situation

A ransomware attack with the ‘Locky’ virus had severe effects for a large German hospital.

Not only were many servers paralysed by the virus limiting hospital operations, but also allegedly uninfected servers were affected as during the panic these were separated while the power supply was still operating. The problem, especially in highly complex virtualised storage systems, is that there may be unexpected issues resulting from a power shutdown. This was the case of a Dell EqualLogic PS6500ES storage array with a total of 148 professional 100-gigabyte hard drives. After the hospital’s IT staff and Dell’s technical support were unable to solve the problem, the specialists of Ontrack were called in to help. All disks were delivered to the data recovery laboratory in Germany where they were assessed.

The Dell EqualLogic PS6500ES system contained multiple hard disks, which typically consist of 16 or 48 HDD shelves, which are connected together to form RAID 5 or RAID 50 systems (sub-arrays). These sub-arrays in turn are connected to ’members’, with one or more members belong to a logical unit (a group). There the LUNs are created and stored, fragmented and distributed over all members and sub-arrays. They are ‘tracked’ by a map, which in turn distributes itself to the members or to the various sub-arrays when it gets proportionally large.

In this case our specialists found out that from those 7 shelves with 148 hard disks, 3 shelves with 80 hard disks contained the LUN with the Oracle databases needed. However, many of the links (mappings) of the data fragments (which were distributed over all hard disks) were either corrupted or no longer available, so assigning the fragments proved to be a very difficult task. The mapping of an EqualLogic PS system is also encoded in a specific logic, so the links here aren’t easy to find either.

The solution

To map the links specialist engineers from other Ontrack offices were involved and new software tools had to be developed especially to solve the logic and the corruption problems regarding both the RAID and the LUN addressing.

With the help of the new tools, the experts were able to recreate the RAID 5 and RAID 50 systems as well as display the LUN. Within this LUN a virtual HDD (a VMDK file) was located, in which an NTFS file system with two Oracle databases were hidden. Two file layers had to be identified and recovered within the LUN before these databases could be finally exported.

The resolution

The data recovery engineers from several Ontrack offices were finally able to successfully extract and recover the required databases and send the data by courier to the client.

The hospital was very pleased with the mediation support from Dell to Ontrack and the fact that they finally had all their important data available again. In addition, the tools developed for this project can be used again in upcoming data recovery cases of Dell EqualLogic PS Array systems and therefore significantly reduce future data recovery times.

Ontrack supports the team.

The client

The situation

In the aftermath of a storm, many people were left with flooded homes and businesses.

One of them was a popular sports club, which suffered severe water damage after torrential rains hit the north of England. Over 1.9TB of important data was lost when critical data stored in drives went underwater, bringing the club to an absolute standstill. They searched for support in recovering their data and came across an IT service provider, a Ontrack partner who provides technical support in the area.

The solution

The IT reseller sent the two damaged drives to Ontrack for the recovery.

Once our engineers reviewed the drives they discovered that the electronics (PCB) had suffered water damaged. The club requested an emergency service in order to get the data back quickly and get themselves back up and running as soon as possible.

The resolution

Fortunately the internal parts were intact, facilitating the recovery.

The damage to the electronic components was overcome when the engineers, using specialist technology, were able to clean up the drive and fix the components on the PCB. Once the PCB was functioning, they were able to access the data on the drive and recover 100% of the customer’s data in only a couple of days. Needless to say both the service provider and the club were fully satisfied with the speed and success of the recovery.

Ontrack helps market-leading UK & European insurance company reduce archive costs by approximately 85%

The client

An enterprise backup and archive storage provider wanted to assist it’s third party client - a marketleading UK insurance company - reduce the costs of their legacy archive storage facilities whilst maintaining restoration capabilities.

The situation

The end client was seeking to reduce the costs of maintaining a restore capability for its legacy archived data, which cost approximately £3.5 million over seven years just to maintain a restore capability.

The client then asked their new archive storage provider to seek out a way to help bring this cost down whilst maintaining the ability to restore and extract data from the legacy tapes, as these were not compatible with the new archive storage provider’s systems and required the proprietary software. The archive storage provider contacted Ontrack with their dilemma; so that they could progress with the sale of the modern archive system with a solution for maintaining the accessibility of the legacy archive tapes.

The solution

Upon briefing of the situation, the Ontrack tape services engineers devised a cost effective plan which involved the cataloguing of 3,600 9940B archive tapes which were created by the legacy system.

The archive data would then remain on the old tapes, so, in the event of a restoration requirement, the end client would consult their new archive catalogue for the exact tapes that contain the files that they require and Ontrack would perform the ad hoc restore.

The resolution

Ontrack worked with the client during an 18 month period to complete a pilot project to prove the concept was technically viable and cost effective whilst meeting stringent legal and company policy requirements around data security and protection.

Upon the success of the pilot project, the full tape archive catalogue was delivered in nine weeks, in order to retire the legacy system prior to the software renewal deadline. In the end, the solution developed allowed the end client to retire their legacy system, saving over £3 million in data retention and archiving costs.

Ontrack helps a UK leading IT company recover over 60 VMs.

The client

An IT company with data centre facilities in Europe and USA and supporting almost 2 million customers.

The situation

The customer realised that 60+ Kernel-based Virtual Machines (KVMs) had disappeared following a technical fault in the host.

Critical customer data was lost, preventing hundreds of clients from carrying on with their normal online businesses. As safeguarding their customers’ data and accessibility was the main priority, they needed a trustworthy data recovery partner who could achieve a full and speedy recovery and be back up and running as soon as possible.

The solution

As part of their disaster recovery action plan, the company contacted Ontrack for immediate help.

Knowing they could provide the best recovery services in the market, it was the obvious partnership to get through this situation. Due to the critical nature of the loss and the amount of people affected, once Ontrack was contacted, a senior engineer visited the customer’s premises that same day to assess the situation. Once there, he performed a KVM recovery on-site to assess how much of the data was recoverable. After working on the drive all night, the recovery proved successful.

A further proof of concept exercise was requested by the client so four KVMs were brought back to the Ontrack laboratory in Epsom to perform simultaneous recoveries to discover how much data and how quickly it could be accessed. With several engineers working around the clock to complete the recoveries as soon as possible, these were successfully completed in only 4 days. The client then gave the green light to Ontrack to continue with the recovery of all the missing data from the remaining KVMs.

The resolution

Following the success of the previous five KVM recoveries, Ontrack achieved a full recovery of all the remaining KVMs by assigning a specialist team of engineers to work around the clock on this project.

By running simultaneous recoveries they were able to complete the job in under a month. In total, the Ontrack Data Recovery team fully recovered all the KVMs, including almost 5,000 virtual disks and 200TB+ of data copied out.

“Nobody expects a data loss to happen and even when it is confined to a small area of a business, as it was in this case, it can still have a large impact. Our client was able to restore its drives relatively quickly with the help of our expert engineering team, who worked around the clock to recover the data that had been affected. It is always best to prepare for all contingencies, which is why it’s important to include a trustworthy data recovery partner in any disaster recovery plan”.

— Robin England Senior Research and Development Engineer at Ontrack

Accius® case study -don’t let your cloud burst.

The client

The situation

The Accius development server had become unresponsive and failed to re-boot, so Accius started the process of ‘disassembling’ the virtual machine to prepare for launching a new development server.

However, a software engineer at Accius forcibly detached one of the virtual storage volumes from the unresponsive server, which is the same as pulling a cable from an operational volume. When the volume was re-connected to a Windows host, Windows found the volume to be corrupt. Amazon’s technical support advised that the only way to recover the data was to use a data recovery tool or company - so Accius contacted Ontrack who provided a free consultation.

Accius President Douglas Moore said, “Time was business critical and recreating or re-building the development environment would have added an additional two to three days to the project - time we didn’t have. The three days’ labour we had invested would have also gone to waste. We must do all we can to ensure we successfully meet our clients’ expectations.”

The solution

“When I called Ontrack, I was quickly connected to an engineer who spent an hour assessing our data loss situation, considering such factors as how we store our data, the technology and type of data involved, and how we thought the data loss happened,” explained Douglas.

Accius was quickly connected to Ontrack using their proprietary Ontrack® Remote Data Recovery™ (RDR®) process. The RDR engineer was able to repair the damaged cloud storage volume and recover the critical data.

RDR is a patented technology that consists of three main components:

■ Communications Client—Accius initiated a connection to an RDR Server using the specially designed RDR® QuickStart™ software. After installing the application, Accius then selected its internet connection as its mode of communication.

■ RDR Server—Once Accius established a connection with the RDR Server it was then routed to the next available RDR engineer.

■ RDR work station—A specially designed application allowed the RDR engineer to run advanced data recovery tools on Accius’ computer system that lost data. Before beginning the recovery process, the engineer enabled proprietary technologies that track and backup all changes that could be made to the system. This process provided the engineer with the ability to complete the recovery “virtually” before any changes were made to the system. Any changes made could be reversed or modified in order to provide the most complete recovery possible.

The resolution

David Logue, Lead Remote Data Recovery Engineer at Ontrack commented, “Fortunately, we were able to connect to Accius’ virtual cloud system via the internet connection to perform a lab-quality data recovery. Unless the drive is physically or electronically failing, we can perform the same recoveries in RDR as we can in-lab, thereby ensuring the same level of quality, satisfaction and customer service.”

Ontrack was able to recover the data, repair the damaged cloud storage volume and get the project up and running in two hours. The damaged volume was repaired and was accessible immediately upon completion of the recovery, all within four hours of the data loss occurring.

“Ontrack responded quickly to our call for help and also understood that we needed to restore the data quickly to safeguard Accius’ reputation. Ontrack really understands the data recovery business and as one of only a few companies with the expertise to recover data from the cloud, Ontrack will be our first port of call should we require their services in future.”

“To guard against another data loss, Accius is investigating ways to improve its cloud development processes to reflect the change to their IT infrastructure as a result of using cloud-based servers. Virtual systems have to be treated with the same care as SAN or physical hard disks,” commented Logue.

The data loss has also been a learning experience for Accius’ staff who now understand that cloud computing resources are susceptible to real types of failures. The potential for human error to cause data loss is still present in the cloud. A survey in 2009 by Ontrack underscores this further with the finding that 65 percent of data loss in virtual environments is the result of human error.

“Pinning down the different types of data loss scenarios with your cloud provider before you commit to their storage solution can be valuable in preventing or addressing issues”, said Logue. “Accius is just one of several organisations we have helped who have experienced data loss in the cloud. As more companies choose cloud storage solutions for their data there is the potential for data losses to increase”, said Logue.

A global pharmaceutical company wanted to ensure all data could be restored from 5,000+ tapes with suspected water and corrosion damage.

The client

The company needed to establish whether to restore just the definitely water-damaged tapes or others also as the condition of the tapes was not known. They turned to Ontrack for advice on how to proceed.

The situation

A fire at the company’s premises activated the sprinkler system, and water fell directly onto data archive storage tapes. Out of a total of 5,128 tapes on the premises 346 were identified to definitely have internal water contamination.

The water-damaged tapes required data recovery techniques to restore the data. Although an initial visual inspection of the remaining tapes indicated no water contamination, the ability to restore data from them was unclear. It was difficult to be sure of the condition of the tapes as high humidity environmental conditions subsequent to the fire may have caused moisture to condense inside the tape casings and the metal oxides on the tape media to corrode. Corrosion would prevent the data restoring correctly.

The company contacted Ontrack for data recovery from the water damaged tapes and asked for a quotation for evaluating the condition of the remaining tapes. Once the company had the evaluation results it conducted a risk analysis to decide how to proceed (i.e. whether to restore, re-key the data or leave the data on the tapes).

The solution

The company decided that all of the data on the tapes needed to be restored as the risks of not doing so were too high. Three hundred of the water damage tapes needed to be cleaned and dried before restoring. Also, a number of these tape needed thawing first as they had been stored in a freezer to help slow down any corrosion on the tape media.

There were several different backup software and tape formats used. There were estimated to be: 1,200 DLT; 400 LTO; 1,000 Exabyte 8mm; 600 DDS and 1,928 tapes of undefined format.

Once restored to a central storage device the data was consolidated onto new LTO 5 tape media, using the company’s current backup software - CommVault. This gave the company a single restore infrastructure for all their archives and backups. The project was completed over a 6 month period.

The resolution

Virtual planning data lost by wrong moving of data.

The client

The situation

Since it was not a mechanic defect, a remote data recovery (RDR) could be done. The Dell EqualLogic System with three LUNs was connected with the computer of a Ontrack data recovery engineer via a secure internet connection. All the client had to do was to install a client-software on a PC connected to the Dell EqualLogic System and connect it to Ontrack. The engineer could then start to work. The first step was to check the system data. When this check confirmed that the virtual machines were not deleted but only saved wrongly, we could start the recovery.

The solution

The important Excel planning files could all be recovered by the data recovery specialists. It took the engineer two full days to find his way through the Dell EqualLogic system’s data structure. He found out that the virtual machines with the Excel files were hidden in one of the three LUNs. 49 GB of data were stored on the respective drive. Because of the wrong move of the virtual machines and their snapshots, the engineer had to dig deep into the HyperV configuration to copy it back to the correct location. This was the only way to make the files accessible again. The effort paid off: the planning files’ restoration allowed the continuous smooth operations of the supermarket under the Caribbean sun.

The resolution