How to analyze hadoop cluster?

This article is all about to find capacity of each component in a hadoop cluster. Hit below command to check hdfs report  [cloudera@quickstart bin]$ hdfs dfsadmin -report Below are the parameters for my hadoop cluster Configured Capacity: 58479091712 (54.46 GB) It is the total capacity available to HDFS for storage. Present Capacity: 45443014656 (42.32 GB) Read More

How to setup Hadoop cluster using cloudera vm?

Cloudera released their version of hadoop for UI ,security and monitoring They also provide commercial support for hadoop distributions. Cloudera was first to offer hadoop version as package. Lets see how to install hadoop using cloudera. Visit below site to download cloudera on virtual box sign up here, after that downloading will be started Read More

What is federation in Hadoop?

Before explaining a federation concept we must know what are NameNodes, DataNodes and their functionality. HDFS follows master- slave architecture. HDFS cluster consists of a single Master called as NameNode. A master server manages the file system namespace and contains metadata information. Metadata information means mapping of data blocks to DataNodes anywhere in Rack. In Read More

Tuning advisors

How to use SQL Access Advisor?

SQL Access Advisor suggests proper set of materialized views,partitioned tables,indexes for given workload. Complex tasks such as partition of an unpartitioned table needs a go through for access advisors. Here I will suggest how to use access advisor for each component. In case of views, as a virtual table does not hold any data. Every Read More