A Hand Book on Hadoop : Real World Approach for Infinite Scalability

In this Exa byte scale era, data increases at an exponential rate. This is in turn generating a massive amount of metadata in the file system. Hadoop is the most widely used framework to deal with big data. Due to this growth of huge amount of metadata, however, the efficiency of Hadoop is questioned numerous times by many researchers. Therefore, it is essential to create an efficient and scalable metadata management for Hadoop. The book principally deals with the Single Point of Failure (SPOF) of Hadoop by providing a suitable distributed metadata management. This book presents a circular Metadata management mechanism named Dynamic Circular Metadata Splitting (DCMS). DCMS preserves metadata locality using consistent hashing as well as locality-preserving hashing, keeps replicated metadata for excellent reliability, as well as dynamically distributes metadata among the NameNodes to keep load balancing. DCMS removes Hadoop's SPOF as well as provides an efficient and scalable metadata management. This book demonstrates the various experiments with DCMS, and shows how the new framework is an improved solution compared to the existing one, both mathematically and theoretically.

Dipayan Dev is adept in big data, cloud computing and especially in Hadoop framework. He has completed his M.Tech from National Institute of Technology Silchar, and now working as software professional in Bangalore, India. Dipayan is currently the author of various research papers, which are published by IEEE and Springer including SCI(E) journal.