Red Nodes -- Hadoop Cluster

The Red Nodes were part of the general purpose storage allocation in 2013. The red nodes were designed as data movers into and out of the general purpose storage system. There are fifty of these nodes. Each node has a Sandy Bridge EP "E5-2620" processor with 6 cores @2 GHz, QDR infiniband, 8 GB of memory and a 500 GB hard drive. MSI has configured 40 of these compute nodes into two Hadoop clusters each with 20 nodes.   

 

User Guide

Please email help@msi to get access to this new resource. The red nodes and the Hadoop infrastructure are accessible from the MSI login nodes. For more information to get started, please read the quickstart guide.

Hardware and Configuration

Each red node has an Intel Sandy Bridge EP "E5-2620" processor @2 GHz. Each processor has six cores and 15 MB cache, and it communicates with memory on a dual QuickPath Interconnect (QPI) interface ( 2 x 7.2 Giga-Transfers/sec). The system includes 20 nodes which participate in the Hadoop cluster environment.   

  • 19 compute nodes
  • 1 head node
  • 4.3 TB Hadoop HDFS filesystem (replication factor is 2)
  • 8 GB memory for each node
  • QDR IB fabric
  • Centos 6.5 Systems
  • Suitable for: Big data analytics

Network

All of the systems within the red nodes are interconnected with a 40-gigabit InfiniBand fabric. The InfiniBand fabric is a high-bandwidth, low-latency network, the intent of which is to accommodate high-bandwidth communication for data movement and analytics. The nodes are also interconnected with a single 1-gigabit ethernet network for administration.

Home Directories and Disks

Red home directories are as described on the MSI Home Directories page. The hdfs filesystem is described in the quickstart guide.