Addressing Name Node Scalability Issue in Hadoop Distributed File System using Cache Approach

Abstract

Hadoop is a distributed batch processing infrastructure which is currently being used for big data management. At the foundation of Hadoop lies Hadoop Distributed File System (HDFS). HDFS presents a client-server architecture comprised of a NameNode and many DataNodes. The NameNode stores the metadata for the DataNodes and DataNode stores application data. The NameNode holds file system metadata in memory, and thus the limit to the number of files in a file system is governed by the amount of memory on the NameNode. Thus when the memory on NameNode is full there is no further chance of increasing the cluster capacity. In this paper we have used the concept of cache memory for handling the issue of NameNode scalability.