A Korean based managed service provider attempted to make configuration changes to their client’s NetApp system when an engineer incorrectly started a ‘dd’ command on some LUNs, effectively wiping the data that was part of the of end user’s production Sybase server
Without access to the data, the managed service provider potentially faced loss of contract from their client, as well as potential liability costs.
The client had a NetApp FAS8060 system containing 161 x 900GB SAS HDDs, arranged into two separate aggregates (68 drives + 93 drives). The customer was presenting 3 x 468GB FC LUNs from each aggregate out to a Sybase server. The 6 total LUNs were combined into a single Disk Pool, with three logical volumes carved out of the Pool. An incorrect ‘dd’ command had written zeroes to approximately 45GB of one of the logical volumes, and this volume was no longer visible to the Sybase server.