IBM Spectrum Scale (formerly GPFS)

1 Front cover IBM Spectrum Scale ( formerly gpfs ). Dino Quintero Luis Bolinches Puneet Chaudhary Willard Davis Steve Duersch Carlos Henrique Fachim Andrei Socoliuc Olaf Weiser Redbooks International Technical Support Organization IBM Spectrum Scale ( formerly gpfs ). May 2015. SG24-8254-00. Note: Before using this information and the product it supports, read the information in Notices on page ix. First Edition (May 2015). This edition applies to IBM Spectrum Scale ( formerly gpfs ) (TL1), SLES 11 SP2, SLES11 SP3, RHEL ppc64 , AIX TL3, Tivoli Storage Manager Server, Client and Storage Agent (LAN-free), Tivoli Storage Manager for Space Management , and TSM for Space Management (HSM) Copyright International Business Machines Corporation 2015. All rights reserved. Note to Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

2 Contents Notices .. ix Trademarks ..x IBM Redbooks promotions .. xi Preface .. xiii Authors .. xiii Now you can become a published author, too! .. xv Comments welcome.. xv Stay connected to IBM Redbooks .. xv Chapter 1. Introduction.. 1. Overview .. 2. IBM Spectrum Scale major components and terminology .. 3. IBM Spectrum Scale cluster topologies .. 5. IBM Spectrum Scale new features and enhancements .. 5. Enhanced security .. 6. Performance improvements .. 6. New usability features .. 7. Reliability, availability, and serviceability.. 10. IBM Spectrum Scale competitive strategy .. 10. IBM Spectrum Scale licensing .. 12. IBM Spectrum Scale on cross-platform environments .. 15. IBM Spectrum Scale for AIX .. 15. IBM Spectrum Scale for Windows operating systems .. 16. IBM Spectrum Scale for Linux.

3 16. IBM Spectrum Scale for Linux on System z .. 17. IBM Spectrum Scale Server solution .. 17. IBM Spectrum Scale and virtualization .. 18. Virtualization on IBM Power Systems .. 19. Virtualization on x86 Linux operating systems .. 26. Virtualization on Windows operating systems.. 27. Contact information .. 28. Chapter 2. Infrastructure planning and considerations .. 29. IBM Spectrum Scale cluster topologies .. 30. Network-based Spectrum Scale client .. 30. Direct attached storage .. 30. Mixed NSD access: server and clients .. 31. IBM Spectrum Scale File Placement Optimizer .. 32. Network design .. 33. Ethernet adapters options .. 33. NIC teaming configurations.. 34. InfiniBand networks.. 36. RDMA over Converged Enhanced Ethernet .. 39. IBM data center networking products.

4 45. Storage design .. 45. Host bus adapter.. 46. Multipath driver .. 47. Copyright IBM Corp. 2015. All rights reserved. iii Storage subsystem considerations .. 50. Tape library .. 54. IBM Spectrum Scale supported platforms.. 56. IBM Spectrum Scale on AIX .. 57. IBM Spectrum Scale for Linux (x86 and POWER) .. 57. IBM Spectrum Scale for Linux on System z .. 59. IBM Spectrum Scale on Windows operating systems.. 61. Summary of IBM Spectrum Scale support functions on operating systems .. 64. GPT partition table .. 66. SCSI-3 Persistent Reservation .. 66. Security considerations for IBM Spectrum Scale clusters.. 69. Remote shell with ssh .. 70. Remote cluster: Subnets, firewall rules, and TCP port numbers .. 70. SELinux configuration with IBM Spectrum Scale .. 71. IBM Spectrum Scale configuration planning.

5 72. High availability .. 72. Network Shared Disk creation considerations .. 78. Planning the IBM Spectrum Scale file system .. 85. Planning for IBM Spectrum Scale FPO.. 93. Chapter 3. Scenarios .. 97. IBM Spectrum Scale advantages over Network File System .. 98. IBM Spectrum Scale in active-passive and mutual takeover clusters .. 98. IBM Spectrum Scale in active-active clusters .. 99. Two-node Linux IBM Spectrum Scale cluster .. 99. Installing IBM Spectrum Scale .. 99. Configure auxiliary tools .. 100. Building the IBM Spectrum Scale portability layer on Linux nodes .. 102. Create IBM Spectrum Scale cluster .. 104. Create NSD disks .. 105. Starting the IBM Spectrum Scale cluster .. 107. Quorum configuration .. 108. Start IBM Spectrum Scale at boot.. 109. Create the IBM Spectrum Scale file system.

6 110. Cluster NFS.. 110. NFS setup .. 111. Configuring cNFS .. 111. Windows operating system-only cluster .. 113. Configuring Windows operating system .. 114. Static IP address .. 114. Active Directory domain .. 114. UAC .. 114. Disable IPv6 .. 116. Windows operating system firewall .. 118. IBM Spectrum Scale traces auxiliary tools .. 120. Installing Cygwin .. 121. Install IBM Spectrum Scale .. 121. Configure IBM Spectrum Scale embedded remote shell .. 128. Create IBM Spectrum Scale cluster .. 129. Creating NSD disks.. 130. Start the IBM Spectrum Scale cluster .. 132. Quorum configuration .. 133. Start IBM Spectrum Scale at boot.. 134. iv IBM Spectrum Scale ( formerly gpfs ). Create the IBM Spectrum Scale file system .. 134. Adding a Linux node to the Windows operating system cluster.

7 135. Adding a Windows operating system node to the cluster .. 148. Oracle Real Application Cluster with IBM Spectrum Scale .. 152. IBM Spectrum Scale integration with IBM Spectrum Protect ( formerly Tivoli Storage Manager) .. 159. Sample Spectrum Scale FPO configuration .. 170. Integrating with OpenStack Swift .. 175. Chapter 4. Management and maintenance .. 177. IBM Spectrum Scale and gpfs migration and update .. 178. gpfs and Spectrum Scale migration considerations .. 178. Migrating to Spectrum Scale from gpfs (Rolling update) .. 179. Migrating to Spectrum Scale from gpfs or gpfs .. 180. Migrating to Spectrum Scale from gpfs or earlier releases of gpfs .. 182. Completing the gpfs or IBM Spectrum Scale migration .. 183. Applying corrective fixes to IBM Spectrum Scale .

8 185. Managing IBM Spectrum Scale cluster .. 187. Managing cluster repository .. 187. Changing cluster manager nodes .. 190. Managing IBM Spectrum Scale nodes .. 192. Managing IBM Spectrum Scale file systems .. 196. Listing file systems .. 197. Mounting a file system .. 198. Unmounting a file system .. 199. Creating an IBM Spectrum Scale file system .. 199. Removing a file system .. 201. Repairing a file system .. 201. Reducing file system fragmentation .. 204. Listing file system attributes .. 205. Changing file system attributes .. 206. Optimizing extended attributes: The fastea option .. 207. Managing IBM Spectrum Scale disks .. 209. Displaying disks .. 209. Creating NSDs .. 209. Adding a disk to a file system .. 212. Deleting disks from a file system.. 214. Replacing disks in an IBM Spectrum Scale file system.

9 215. Managing IBM Spectrum Scale data migration .. 216. Migrating data using NSDs in gpfs file systems .. 216. Migrating data using AFM-based NFS Migration .. 219. Managing the IBM Spectrum Scale network .. 221. Managing IBM Spectrum Scale remote cluster .. 223. Planning and preparation .. 223. Adding a remote cluster .. 224. Enabling gpfs replication .. 228. Exporting or importing a file system .. 232. Configuring IBM Spectrum Scale callback .. 238. Monitoring IBM Spectrum Scale with SNMPD protocol.. 240. SSH configuration .. 242. Chapter 5. Information lifecycle management .. 245. Explaining the ILM concept .. 246. Contents v Fileset .. 246. Snapshot .. 251. Snapshot at the file system level .. 251. Snapshot at the fileset level .. 256. Snapshot at file level .. 259.

10 Storage pools .. 260. Internal storage pools .. 260. External storage pools .. 262. Immutability and appendOnly attributes .. 265. Quotas.. 267. Enabling and editing quotas .. 267. Creating quota reports .. 269. Policies and rules .. 270. Policies .. 270. Rules .. 271. Example of polices with internal pools only.. 271. Example of policies with one external pool only .. 277. Access control list .. 280. Traditional Spectrum Scale ACL administration .. 280. NFSv4 ACL administration .. 283. The mmfind policy sample .. 285. Differences with IBM EasyTier .. 288. Chapter 6. Active File Management.. 291. Active file management fundamentals .. 292. AFM mode .. 293. Minimum requirements and basic operating system tuning .. 294. Using NFS for data movement .. 296. AFM single-writer.