OpenHPC (v1.0) Cluster Building Recipes

1 OpenHPC ( ) Cluster Building Base OSBase Linux* EditionDocument Last Update: 2015-11-12 Document Revision: bf8c471 Install Guide - Version ( )Legal NoticeCopyright 2015, Intel Corporation. All rights and use of the enclosed documentation ( Documentation ), with or without modification, arepermitted provided that the following conditions are met: Redistributions of this Documentation in whatever format ( PDF, HTML, RTF, shell script,PostScript, etc.), must reproduce the above copyright notice, this list of conditions and the followingdisclaimer in the documentation and/or other materials provided with the distribution. Neither the name of Intel Corporation nor the names of its contributors may be used to endorse orpromote products derived from this Documentation without specific prior written as expressly provided above, no license (express or implied, by estoppel or otherwise) to any intel-lectual property rights is granted by this DOCUMENTATION IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANYEXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.

2 IN NO EVENT SHALLTHE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPE-CIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OFSUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOW-EVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENTA-TION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH , the Intel logo, and other Intel marks are trademarks of Intel Corporation in the and/or other countries.*Other names and brands may be claimed as the property of : bf8c471 Install Guide - Version ( )Contents1 Target Audience.

3 Requirements/Assumptions .. Bring your own license .. Inputs ..62 Install Base Operating System (BOS)73 Install OpenHPC Enable OpenHPC repository for local use .. Installation template .. Add provisioning services onmasternode .. Add resource management services onmasternode .. Add InfiniBand support services onmasternode .. Complete basic Warewulf setup formasternode .. Definecomputeimage for provisioning .. initial BOS image .. OpenHPC components .. system configuration .. Customizations (optional) .. locked memory limits .. ssh control via resource manager .. Cluster Checker .. Lustre client.

4 Nagios monitoring .. Ganglia monitoring .. forwarding of system logs .. files .. Finalizing provisioning configuration .. bootstrap image .. Virtual Node File System (VNFS) image .. nodes for provisioning .. Boot compute nodes ..174 Install OpenHPC Development Development Tools .. Compilers .. Performance Tools .. MPI Stacks .. Setup default development environment .. 3rd Party Libraries and Tools ..195 Resource Manager Startup206 Run a Test Interactive execution .. Batch execution ..223 Rev: bf8c471 Install Guide - Version ( )Appendices24 AInstallation Template ..24 BPackage Manifest.

5 25 CPackage Signatures ..344 Rev: bf8c471 Install Guide - Version ( )1 IntroductionThis guide presents a simple Cluster installation procedure using components from the OpenHPC softwarestack. OpenHPC represents an aggregation of a number of common ingredients required to deploy and man-age an HPC Linux* Cluster including provisioning tools, resource management, I/O clients, developmenttools, and a variety of scientific libraries. These packages have been pre-built with HPC integration in mindand represent a mix of open-source components combined with Intel development and analysis tools ( Parallel Studio XE). The documentation herein is intended to be reasonably generic, but uses theunderlying motivation of a small, 4-node stateless Cluster installation to define a step-by-step process.

6 Sev-eral optional customizations are included and the intent is that these collective instructions can be modifiedas needed for local site Linux Edition: this edition of the guide highlights installation without the use of a companion con-figuration management system and directly uses distro-provided package management tools for componentselection. The steps that follow also highlight specific changes to system configuration files that are requiredas part of the Cluster install Target AudienceThis guide is targeted at experienced Linux system administrators for HPC environments. Knowledge ofsoftware package management, system networking, and PXE booting is assumed.

7 Command-line inputexamples are highlighted throughout this guide via the following syntax:[sms]# echo " OpenHPC hello world"Unless specified otherwise, the examples presented are executed with elevated (root) privileges. Theexamples also presume use of the BASH login shell, though the equivalent commands in other shells canbe substituted. In addition to specific command-line instructions called out in this guide, an alternateconvention is used to highlight potentially useful tips or optional configuration options. These tips arehighlighted via the following format:TipIf you don t know where you are going, you might wind up someplace else.

8 Yogi Requirements/AssumptionsThis installation recipe assumes the availability of a single head nodemaster, and fourcomputenodes. Themasternode serves as the overall system management server (SMS) and is provisioned with and issubsequently configured to provision the remainingcomputenodes with Warewulf in a stateless termsmasterand SMS are used interchangeably in this guide. For power management, we assume thatthe compute node baseboard management controllers (BMCs) are available via IPMI from the chosen masterhost. For file systems, we assume that the chosen master server will host an NFS file system that is madeavailable to the compute nodes.

9 Installation information is also discussed to optionally include a Lustre filesystem mount and in this case, the Lustre file system is assumed to exist outline of the physical architecture discussed is shown in Figure 1 and highlights the high-levelnetworking configuration. Themasterhost requires at least two Ethernet interfaces witheth0connected to5 Rev: bf8c471 Install Guide - Version ( )eth1eth0 Data Center Networkhigh speed networktcp networkingto compute eth interfaceto compute BMC interfacecompute nodesLustre* storage systemMaster(SMS)Figure 1: Overview of physical Cluster local data center network andeth1used to provision and manage the Cluster backend (note that theseinterface names are examples and may be different depending on local settings and OS conventions).

10 Twological IP interfaces are expected to each compute node: the first is the standard Ethernet interface thatwill be used for provisioning and resource management. The second is used to connect to each host s BMCand is used for power management and remote console access. Physical connectivity for these two logicalIP networks is often accommodated via separate cabling and switching infrastructure; however, an alternateconfiguration can also be accommodated via the use of a shared NIC, which runs a packet filter to divertmanagement packets between the host and addition to the IP networking, there is a high-speed network (InfiniBand in this recipe) that is alsoconnected to each of the hosts.