Hadoop Solutions For Big Data

Supermicro and Cloudera Solutions

The Challenge

There is a tremendous amount of information driven by the ever changing applications, from structured, unstructured, to semi-structure data. Conventional IT infrastructure is not built to handle the variety, velocity and volume of the data produced by social media networks, mobile applications, machine sensors and scientific researches, etc. For Enterprises, utilizing big data analytics is no longer a question of when, it is a question of how. Hadoop, designed for the cost effective storage and processing of large volumes of data, is born for this purpose. It can linearly scale up to thousands of servers and petabytes of storage.

How to take advantage of Hadoop technology and gain competitive edge is on the mind of almost every corporate CIO. For enterprises, how to deploy the Hadoop infrastructure efficiently means winning or losing in the big battle of market share. Enterprises deploying Hadoop solutions often spend large amount of resource searching for the best architecture and the most capable solution provider. This is where Supermicro comes in to help.

The Solution

Introducing Supermicro Hadoop clusters, a series of optimized big data solutions that provide high performance, high reliability and high scalability. Supermicro Hadoop solutions are fully integrated, fully optimized and completely tested turnkey clusters with flexible support packages available to meet customer specific requirements.

Supermicro Hadoop clusters feature industry proven high density compute and storage servers populated with best of breed components selected through extensive engineering design, validation and testing. Certified configurations take the guess work out of designing and deploying a truly scalable Big Data compute and storage infrastructure that meets the most demanding enterprise IT and data center environments.

Supermicro Advantage

  • Designed from ground up with optimal server and cluster configurations that meet a variety of workloads
  • Proven solutions based on extensive lab testing and large scale production deployments
  • Achieve the best price/performance and the best price/capacity with industry leading server and storage platforms
  • End-to-end turnkey Hadoop clusters with completely integrated HW, SW and Global Support
  • Advanced architecture based on the latest technologies such as SkyLake CPUs, SAS3, NVMe, Optane Drives 
  • Enhanced networking performance and redundancy with dual 10GbE / 25GbE / 40GbE / 100GbE options
  • Automated full cluster testing guarantees build quality and delivery schedule

Fully Integrated Hadoop Cluster

Key Features and benefits:

  • Purpose built cluster configurations optimized for capacity, compute or IO performance
  • High availability Name Node design with no single point of failure
  • Large memory options designed specifically for Spark and other in memory, low latency computations
  • Hyper-Scale server platforms designed for extremely large deployments
  • High density compute, storage and memory design to achieve the best efficiency and lowest TCO
  • Flexible network switch options with 1 or 2x 10G / 25G / 100G switches per rack.
  • Cost effective 14U rack design, ideal for Proof of Concept testing environment
  • Standard 42U rack design and flexible PDU options that meet any data center environment
  • Up to Titanium Level (96%+) Efficiency - Redundant Power Supplies with PMBus
  • Built in with IPMI and SMC OOB (out of band management) suite for automated cluster management
  • Fully integrated, fully configured and completely tested with Hadoop distributions of your choice
  • Proof of Concept testing cluster available for risk free purchasing experience
Supermicro fully integrated Hadoop cluster solution rack
  • 1 or 2x 48 port 10G SFP+ / 10GBase-T / 25GbE
    1 or 2x 32 port 100GbE, 1x 48 port Switch, GbE
  • 1x Management Node 1U UP Skylake 41xx / 51xx
  • 3x Name Nodes 1U DP Xeon Skylake 41xx / 51xx / 61xx
  • Optimized Data nodes 2U SSG, 2U BigTwin or 4U FatTwin with Skylake 41xx / 51xx / 61xx / 81xx processors, dual 10G / 25G / 40G per node, 2.5" and 3.5" HDD options
  • Standard 42U rack with metered PDUs, rack customization options available
  • Integration service includes full cluster Burn-in and testing, BIOS and FW update, networking configuration, Pre-install Hadoop distribution of choice, and full cluster

Hadoop Cluster Technical Specifications

 High CapacityIO OptimizedHigh Density ComputeBalanced
Model SKUSRS-HCSG18-HADP-01SRS-IOBT32-HADP-01SRS-HDFT36-HADP-01SRS-BLSG37-HADP-01
Data NodeSSG-6029P-E1CR24LSYS-2029BT-HNRSYS-F629P3-RC0BSSG-6019P-ACR12L
Data Node (Qty)18323637
Form Factor2U SuperStorage2U BigTwin4U FatTwin1U SuperStorage
2x SKL 4114 2P 10C/20T 2.2G 85W2x SKL 5118 4/2P 12C/24T 2.3G 105W2x SKL 6130 4/2P 16C/32T 2.1G 125W2x SKL 5118 4/2P 12C/24T 2.3G 105W
128GB192GB256GB128GB
24 Bay 3.5"6 Bay 2.5"8 Bay 3.5"12 Bay 3.5"
Total Data Drive432448 (NVMe)*288444
Total Cores3607681152888
Total Memory2.3TB6.144TB9.2TB4.7TB
Total Storage3.45PB (8TB)1.792PB (4TB)1.15PB (4TB)2.66PB (6TB)
Name Node3x 1U WIO3x 1U WIO3x 1U WIO3x 1U WIO
Switches1x 48PT 25GBase-T2x 32PT 40G1x 48PT GbE1x 48PT10G SFP+
Cabinet (W x H x D)42U
23.5 x 82.4 x 48
PDU2x 50A 208 3-Phase Metered PDU
*ADD 32 Bay 2.5" Drives x8 JBOF = 256 NVMe

Server Specs

SSG-6029P-E1CR24L
SSG-6029P-E1CR24L
  • Corporate Database, Database Processing & Storage, Enterprise Server, HPC, Data Center, iSCSI SAN
  • Dual Socket P (LGA 3647) support: Intel® Xeon® Scalable processors; 3 UPI up to 10.4GT/s
  • 24 DIMM slots; up to 3TB ECC 3DS LRDIMM or RDIMM DDR4 2400MHz
  • 24 Hot-swap 3.5" SAS3/SATA3 drive bays, 2 Hot-swap 2.5" drive bays (rear); SAS3 via Broadcom 3008 AOC
  • 2 PCI-E 3.0 x16 and 1 PCI-E 3.0 x8 slots
  • 1 Dedicated IPMI port
  • 1 VGA, 1 COM, 2 USB 3.0, 1 Type A
  • 5 Hot-swap 80mm redundant PWM cooling fans
  • 1600W Redundant Titanium Level (96%) Power Supplies
SYS-2029BT-HNR
SYS-2029BT-HNR
  • 4 DP nodes in 2U; depth 28.5", Each Node:
  • Dual Socket P (LGA 3647) support: Intel® Xeon® Scalable processors; dual UPI up to 10.4GT/s
  • 24 DIMMs; up to 3TB ECC 3DS LRDIMM or RDIMM DDR4‑2666MHz
  • 6 Hot-swap 2.5" NVMe drive bays per node
  • 2 PCI-E 3.0 x16 Low-profile slots, 1 Networking SIOM card support
  • 1 Dedicated IPMI port, 1 VGA, 2 USB 3.0, 1 SuperDOM
  • 4 heavy duty 80mm fans w/ optimal fan speed control
  • 2200W Redundant Titanium Level (96%) Power Supplies
SYS-F629P3-RC0B
SYS-F629P3-RC0B
  • FatTwin™ Server; EACH NODE (x4 Hot-plug System Nodes):
  • Dual Socket P (LGA 3647) support: Intel® Xeon® Scalable processors; dual UPI up to 10.4GT/s
  • 12 DIMMs; up to 1.5TB ECC 3DS LRDIMM, up to DDR4‑2666MHz
  • 8x 3.5" SAS3 or 6 SAS3 + 2 opt. NVMe U.2; SAS3 with Broadcom 3008
  • 1 PCI-E 3.0 x16 (LP), 1 PCI-E 3.0 x16 (SIOM: flexible network opt.)
  • 1 Dedicated IPMI port, 1 VGA, 2 USB 3.0, 1 M.2 support
  • 2x 8cm 14k RPM middle fans
  • 1200W Redundant Titanium Level (96%) Power Supplies
SSG-6019P-ACR12L
SSG-6019P-ACR12L
  • High Density Storage, Object Storage 1U, Ceph / Hadoop, Scale-out Storage, Big Data Analytics
  • Dual Socket P (LGA 3647) support: 2nd Gen. Intel® Xeon® Scalable processors; dual UPI up to 10.4GT/s
  • 12 Hot-swap 3.5" SAS3/SATA3 drive bays, 4 Hot-swap 2.5" NVMe/SATA drive bays
  • 12 DIMMs; up to 3TB 3DS ECC DDR4-2933 MHz RDIMM/LRDIMM
  • Supports Intel® Optane™ DCPMM*
  • 2 PCI-E 3.0 x16, 1 PCI-E 3.0 x8
  • 2x 10GBase-T ports via Intel C622, 1 Dedicated IPMI port
  • 1 VGA, 4 USB 3.0, 1 TPM header
  • 6x 4cm 20.5K RPM cooling fans
  • 600W Redundant Platinum Level Power Supplies

*Contact your Supermicro sales rep for more info.

1Gb & 10Gb Ethernet Switches
1Gb & 10Gb Ethernet Switches
  • Layer 2/3 Ethernet Switching, link aggregation (LACP), Jumbo frames, VLAN support
  • Configuration Flexibility:
    • SSE-G24-TG4 - 24 ports of RJ45 1Gbps Ethernet with 4 SFP Combo ports, 4 ports 10Gbps Ethernet (CX4, XFP, or SFP+), Stacking
    • SSE-G48-TG4 - 48 ports of RJ45 1Gbps Ethernet with 4 SFP Combo ports, 4 ports 10Gbps Ethernet (CX4, XFP, or SFP+), Stacking
    • SSE-X24S / SSE-X3348S(R) - 24 ports 10Gbps Ethernet (SFP+), Reverse airflow option
  • Management: Web-based management interface, Industry standard CLI with telnet SSH or local management port, Scripting, Logging
  • Routing, Multicast:
    • Quality of service and DiffServ - 8 priority queues per port; Adjusted WRR and Strict Priority Scheduling
    • Security - 802.1x port based authentication, RADIUS and TACACS+ authentication, SSH / SSL encryption
  • Dimensions: 17.3"W x 15.2"D x 1.7"H (440mm x 386mm x 43mm)
SSE-C3632S
SSE-C3632S

/ SSE-C3632SR (32 ports)

  • 32x One-Hundred-Gigabit Ethernet ports
    (Each optionally configured for forty-gigabit or 4x twenty-five gigabit)
  • Wire-speed Layer 3 Routing
  • 1:1 Non-blocking connectivity
  • 1U form factor for flexible installation
  • Data-Center friendly - regular and reverse airflow models
  • Redundant hot-pluggable power supplies
  • Bare metal - ONIE Installed
  • Cumulus Linux Capable
IPMI Server Management
IPMI Server Management
  • HW fault alerts - E-mail or SNMP
  • Redundant access for management
  • Support for open standards (SMASH 2.0, LDAP, WSMAN)
  • VLAN support
  • KVM and Virtual Media and SOL (no additional license needed)
  • Easy maintenance (Command line flash tools for firmware update)

Supermicro Integration and Proof of Concept Laboratory Services

See Details

Other Resources