Download directly for more content description... We have designed and implemented the Google File Sys- tem, a scalable distributed file . It is widely deployed within Google as the storage platform . The Google File System demonstrates the qualities essentialfor supporting large-scale data processing workloadson commodity hardware. While some design decisions arespecific to our unique setting, many may apply to data processingtasks of a similar magnitude and cost consciousness.We started by reexamining traditional file system assumptionsin light of our current and anticipated applicationworkloads and technological environment. Our observationshave led to radically different points in the design space.We treat component failures as the norm rather than theexception, optimize for huge files that are mostly appendedto (perhaps concurrently) and then read (usually sequentially),and both extend and relax the standard file systeminterface to improve the overall system. Our system provides fault tolerance by constant monitoring,replicating crucial data, and fast and automatic recovery.Chunkrep lication allows us to tolerate chunkserverfailures. The frequency of these failures motivated a novelonline repair mechanism that regularly and transparently repairsthe damage and compensates for lost replicas as soonas possible. Additionally, we use checksumming to detectdata corruption at the disko r IDE subsystem level, whichbecomes all too common given the number of disks in thesystem.

» Free PDF eBooks Download
» Search/Search Engine Google