Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Sector-Disk (SD) Erasure Codes for Mixed Failure Modes in RAID Systems

Sector-Disk (SD) Erasure Codes for Mixed Failure Modes in RAID Systems Traditionally, when storage systems employ erasure codes, they are designed to tolerate the failures of entire disks. However, the most common types of failures are latent sector failures, which only affect individual disk sectors, and block failures which arise through wear on SSDs. This article introduces SD codes, which are designed to tolerate combinations of disk and sector failures. As such, they consume far less storage resources than traditional erasure codes. We specify the codes with enough detail for the storage practitioner to employ them, discuss their practical properties, and detail an open-source implementation. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ACM Transactions on Storage (TOS) Association for Computing Machinery

Sector-Disk (SD) Erasure Codes for Mixed Failure Modes in RAID Systems

Plank, James S.; Blaum, Mario
ACM Transactions on Storage (TOS) , Volume 10 (1): 17 – Jan 1, 2014
Download PDF
17 pages

Loading next page...
 
/lp/association-for-computing-machinery/sector-disk-sd-erasure-codes-for-mixed-failure-modes-in-raid-systems-09fEfuZwjv
Publisher
Association for Computing Machinery
Copyright
Copyright © 2014 ACM
ISSN
1553-3077
eISSN
1553-3093
DOI
10.1145/2560013
Publisher site
See Article on Publisher Site

Abstract

Traditionally, when storage systems employ erasure codes, they are designed to tolerate the failures of entire disks. However, the most common types of failures are latent sector failures, which only affect individual disk sectors, and block failures which arise through wear on SSDs. This article introduces SD codes, which are designed to tolerate combinations of disk and sector failures. As such, they consume far less storage resources than traditional erasure codes. We specify the codes with enough detail for the storage practitioner to employ them, discuss their practical properties, and detail an open-source implementation.

Journal

ACM Transactions on Storage (TOS) – Association for Computing Machinery

Published: Jan 1, 2014

Keywords: Erasure codes

References

  • Practical scrubbing: Getting to the bad sector at the right time
    Amvrosiadis, ; G., ; Oprea, ; A., ; Schroeder, ; B.,
  • An analysis of data corruption in the storage stack
    Bairavasundaram, ; N., L.; Goodson, ; G., ; Schroeder, ; B., ; Arpaci-Dusseau, ; C., A.; Arpaci-Dusseau, ; H., R.
  • Differential RAID: Rethinking RAID for SSD reliability
    Balakrishnan, ; M., ; Kadav, ; A., ; Prabhakaran, ; V., ; Malkhi, ; D.,
  • Construction of two SD codes
    Blaum, ; M., ; Plank, ; S., J.
  • On lowest density MDS codes
    Blaum, ; M., ; Roth, ; M., R.
  • EVENODD: An efficient scheme for tolerating double disk failures in RAID architectures
    Blaum, ; M., ; Brady, ; J., ; Bruck, ; J., ; Menon, ; J.,
  • Partial-MDS codes and their application to RAID type of architectures
    Blaum, ; M., ; Hafner, ; L., J.; Hetzler, ; S.,
  • An XOR-based erasure-resilient coding scheme
    Blomer, ; J., ; Kalfane, ; M., ; Karpinski, ; M., ; Karp, ; R., ; Luby, ; M., ; Zuckerman, ; D.,
  • HAIL: A high-availability and integrity layer for cloud storage
    Bowers, ; K., ; Juels, ; A., ; Oprea, ; A.,
  • Compound codes for optimal repair in MDS code based distributed storage systems
    Cadambe, ; V., ; Huang, ; C., ; Li, ; J., ; Mehrotra, ; S.,
  • Windows Azure Storage: A highly available cloud storage service with strong consistency
    Calder, ; B., ; Wang, ; J., ; Ogus, ; A., ; Nilakantan, ; N., ; Skjolsvold, ; A., ; McKelvie, ; S., ; Xu, ; Y., ; Srivastav, ; S., ; Wu, ; J., ; Simitci, ; H., ; Haridas, ; J., ; Uddaraju, ; C., ; Khatri, ; H., ; Edwards, ; A., ; Bedekar, ; V., ; Mainali, ; S., ; Abbasi, ; R., ; Agarwal, ; A., ; Haq, ul; F., M.; Haq, ul; I., M.; Bhardwaj, ; D., ; Dayanand, ; S., ; Adusumilli, ; A., ; McNett, ; M., ; Sankaran, ; S., ; Manivannan, ; K., ; Rigas, ; L.,
  • Remote data checking for network coding-based distributed storage systems
    Chen, ; B., ; Curtmola, ; R., ; Ateniese, ; G., ; Burns, ; R.,
  • Row diagonal parity for double disk failure correction
    Corbett, ; P., ; English, ; B., ; Goel, ; A., ; Grcanac, ; T., ; Kleiman, ; S., ; Leong, ; J., ; Sankar, ; S.,
  • A new intra-disk redundancy scheme for high-reliability RAID storage systems in the presence of unrecoverable errors
    Dholakia, ; A., ; Eleftheriou, ; E., ; Hu, ; Y., X.; Iliadis, ; I., ; Menon, ; J., ; Rao, ; K., K.
  • A survey on network codes for distributed storage
    Dimakis, ; G., A.; Ramchandran, ; K., ; Wu, ; Y., ; Suh, ; C.,
  • FlexVol: Flexible, efficient file volume virtualization in WAFL
    Edwards, ; K., J.; Ellard, ; D., ; Everhart, ; C., ; Fair, ; R., ; Hamilton, ; E., ; Kahn, ; A., ; Kanevsky, ; A., ; Lentini, ; J., ; Prakash, ; A., ; Smith, ; A., K.; Zayas, ; E.,
  • A highly accurate method for assessing reliability of redundant arrays of inexpensive disks
    Elerath, ; G., J.; Pecht, ; M.,
  • The Google file system
    Ghemawat, ; S., ; Gobioff, ; H., ; Leung, ; T., S.
  • On the locality of codeword symbols
    Gopalan, ; P., ; Huang, ; C., ; Simitci, ; H., ; Yekhanin, ; S.,
  • Optimizing galois field arithmetic for diverse processor architectures and applications
    Greenan, ; K., ; Miller, ; E., ; Schwartz, ; J., T.
  • Building flexible, fault-tolerant flash-based storage systems
    Greenan, ; M., K.; Long, ; D., D.; Miller, ; L., E.; Schwarz, ; E., T. J.; Wildani, ; A.,
  • Flat XOR-based erasure codes in storage systems: Constructions, efficient recovery and tradeoffs
    Greenan, ; M., K.; Li, ; X., ; Wylie, ; J., J.
  • HoVer erasure codes for disk arrays
    Hafner, ; L., J.
  • Undetected disk errors in RAID arrays
    Hafner, ; L., J.; Deenadhayalan, ; V., ; Belluomini, ; W., ; Rao, ; K.,
  • NCCloud: Applying network coding for the storage repair in a cloud-of-clouds
    Hu, ; Y., ; Chen, ; H., H. C.; Lee, ; C., P. P.; Tang, ; Y.,
  • Pyramid codes: Flexible schemes to trade space for access efficienty in reliable data storage systems
    Huang, ; C., ; Chen, ; M., ; Li, ; J.,
  • Erasure coding in Windows Azure storage
    Huang, ; C., ; Simitci, ; H., ; Xu, ; Y., ; Ogus, ; A., ; Calder, ; B., ; Gopalan, ; P., ; Li, ; J., ; Yekhanin, ; S.,
  • DFS: A file system for virtualized flash storage
    Josephson, ; K., W.; Bongo, ; A., L.; Flynn, ; D., ; Li, ; K.,
  • REO: A generic RAID engine and optimizer
    Kenchammana-Hosekote, ; D., ; He, ; D., ; Hafner, ; L., J.
  • Rethinking erasure codes for cloud file systems: Minimizing I/O for recovery and degraded reads
    Khan, ; O., ; Burns, ; R., ; Plank, ; S., J.; Pierce, ; W., ; Huang, ; C.,
  • Storage architecture with integrity, redundancy and encryption
    Klein, ; H., ; Keller, ; J.,
  • A new class of array codes for memory storage
    Lastras-Montaño, ; A., L.; Meaney, ; J., P.; Stephens, ; E., ; Trager, ; M., B.; O’Connor, ; J., ; Alves, ; C., L.
  • GRID codes: Strip-based erasure codes with high fault tolerance for storage systems
    Li, ; M., ; Shu, ; J., ; Zheng, ; W.,
  • Efficient eventual consistency in Pahoehoe, an erasure-coded key-blob archive
    Li, ; X., ; Marchant, ; A., ; Shah, ; A., M.; Smathers, ; K., ; Tucek, ; J., ; Uysal, ; M., ; Wylie, ; J., J.
  • LT codes
    Luby, ; M.,
  • Efficient software implementations of large finite fields GF(2n) for secure storage applications
    Luo, ; J., ; Bowers, ; D., K.; Oprea, ; A., ; Xu, ; L.,
  • The Theory of Error-Correcting Codes, Part I
    MacWilliams, ; J., F.; Sloane, ; A., N. J.
  • Caching less for better performance: Balancing cache size and update cost of flash memory cache in hybrid storage systems
    Oh, ; Y., ; Choi, ; J., ; Lee, ; D., ; Noh, ; H., S.
  • Java FEC Library v1
    Networks, Onion
  • A clean-slate look at disk scrubbing
    Oprea, ; A., ; Juels, ; A.,
  • Beating the I/O bottleneck: A case for log-structured file systems
    Ousterhout, ; K., J.; Douglis, ; F.,
  • Schifra Reed-Solomon ECC Library
    Partow, ; A.,
  • Error-Correcting Codes 2nd Ed
    Peterson, ; W., W.; Weldon, ; Jr,
  • A tutorial on Reed-Solomon coding for fault-tolerance in RAID-like systems
    Plank, ; S., J.
  • Tutorial: Erasure coding for storage applications
    Plank, ; S., J.; Huang, ; C.,
  • A performance evaluation and examination of open-source erasure coding libraries for storage
    Plank, ; S., J.; Luo, ; J., ; Schuman, ; D., C.; Xu, ; L., ; Wilcox-O’Hearn, ; Z.,
  • SD codes: Erasure codes designed for how storage systems really fail
    Plank, ; S., J.; Blaum, ; M., ; Hafner, ; L., J.
  • Screaming fast Galois Field arithmetic using Intel SIMD instructions
    Plank, ; S., J.; Greenan, ; M., K.; Miller, ; L., E.
  • AONT-RS: Blending security and performance in dispersed storage systems
    Resch, ; K., J.; Plank, ; S., J.
  • Pond: The OceanStore prototype
    Rhea, ; S., ; Eaton, ; P., ; Geels, ; D., ; Weatherspoon, ; H., ; Zhao, ; B., ; Kubiatowitz, ; J.,
  • Disk failures in the real world: What does an MTTF of 1,000,000 mean to you? In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST’07)
    Schroeder, ; B., ; Gibson, ; G.,
  • Understanding latent sector errors and how to protect against them
    Schroeder, ; B., ; Damouras, ; S., ; Gill, ; P.,
  • An implementation of a log-structured file system for UNIX
    Seltzer, ; M., ; Bostic, ; K., ; McKusick, ; M., ; Staelin, ; C.,
  • Pergamum: Replacing tape with energy efficient, reliable, disk-based archival storage
    Storer, ; W., M.; Greenan, ; M., K.; Miller, ; L., E.; Voruganti, ; K.,
  • POTSHARDS -- A secure, long-term storage system
    Storer, ; W., M.; Greenan, ; M., K.; Miller, ; L., E.; Voruganti, ; K.,
  • Exact regeneration codes for distributed storage repair using interference alignment
    Suh, ; C., ; Ramchandran, ; K.,
  • Rebuilding for array codes in distributed storage systems
    Wang, ; Z., ; Dimakis, ; G., A.; Bruck, ; J.,
  • Tahoe: A secure distributed filesystem
    Warner, ; B., ; Wilcox-O’Hearn, ; Z., ; Kinninmont, ; R.,
  • Scalable performance of the Panasas parallel file system
    Welch, ; B., ; Unangst, ; M., ; Abbasi, ; Z., ; Gibson, ; G., ; Mueller, ; B., ; Small, ; J., ; Zelenka, ; J., ; Zhou, ; B.,
  • Optimal recovery of single disk failure in RDP code storage systems
    Xiang, ; L., ; Xu, ; Y., ; Lui, ; S., J. C.; Chang, ; Q.,
  • Hydra: A platform for survivable and secure data storage systems
    Xu, ; L.,