1. DL manuals
  2. IBM
  3. Server
  4. System x3950 X6
  5. Planning And Implementation Manual

IBM System x3950 X6 Planning And Implementation Manual

Summary of System x3950 X6

  • Page 1

    Ibm.Com/redbooks front cover ibm system x3850 x6 and x3950 x6 planning and implementation guide david watts rani doughty ilya solovyev covers the sixth generation enterprise x-architecture servers provides technical information about all server features explains what planning you need to do.

  • Page 3

    International technical support organization ibm system x3850 x6 and x3950 x6 planning and implementation guide september 2014 sg24-8208-00

  • Page 4

    © copyright international business machines corporation 2014. All rights reserved. Note to u.S. Government users restricted rights -- use, duplication or disclosure restricted by gsa adp schedule contract with ibm corp. First edition (september 2014) this edition applies to ibm system x3850 x6 and x...

  • Page 5: Contents

    © copyright ibm corp. 2014. All rights reserved. Iii contents notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vii trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....

  • Page 6

    Iv ibm system x3850 x6 and x3950 x6 planning and implementation guide 2.6.1 storage book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2.6.2 ibm exflash memory-channel storage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...

  • Page 7

    Contents v 3.22 power subsystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 3.23 fans and cooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 3.24 upgrading to an 8-socket x6...

  • Page 8

    Vi ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.8.5 ibm electronic service agent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 5.8.6 problem determination and service guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 chapte...

  • Page 9: Notices

    © copyright ibm corp. 2014. All rights reserved. Vii notices this information was developed for products and services offered in the u.S.A. Ibm may not offer the products, services, or features discussed in this document in other countries. Consult your local ibm representative for information on th...

  • Page 10: Trademarks

    Viii ibm system x3850 x6 and x3950 x6 planning and implementation guide trademarks ibm, the ibm logo, and ibm.Com are trademarks or registered trademarks of international business machines corporation in the united states, other countries, or both. These and other ibm trademarked terms are marked on...

  • Page 11: Summary of Changes

    © copyright ibm corp. 2014. All rights reserved. Ix summary of changes this section describes the technical changes made in this edition of the book and in previous editions. This edition might also include minor corrections and editorial changes that are not identified. 26 september 2014 changed in...

  • Page 12

    X ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 13: Preface

    © copyright ibm corp. 2014. All rights reserved. Xi preface the increasing demand for cloud computing and business analytical workloads by enterprises to meet business needs drives innovation to find new ways to build informational systems. Clients are looking for cost-optimized fit-for-purpose it s...

  • Page 14: Authors

    Xii ibm system x3850 x6 and x3950 x6 planning and implementation guide authors this book was produced by a team of specialists from around the world working at the international technical support organization, raleigh center. Thanks to the following people for their contributions to this project: ib...

  • Page 15: Comments Welcome

    Preface xiii loc nguyen bill stevens tim schlude ibm redbooks: deana coble rich conway tamikia barrow ilya krutov debbie willmschen others who helped us: matthew archibald, data center services simon casey, ibm chris cook, diablo technologies chris noonan, sandisk now you can become a published auth...

  • Page 16

    Xiv ibm system x3850 x6 and x3950 x6 planning and implementation guide stay connected to ibm redbooks find us on facebook: http://www.Facebook.Com/ibmredbooks follow us on twitter: http://twitter.Com/ibmredbooks look for us on linkedin: http://www.Linkedin.Com/groups?Home=&gid=2130806 explore new re...

  • Page 17: Introduction

    © copyright ibm corp. 2014. All rights reserved. 1 chapter 1. Introduction the ibm x6 family of scalable rack servers consists of two servers: ibm system x3850 x6, a four-socket 4u rack-mount server ibm system x3950 x6, an eight-socket 8u rack-mount server these servers are the sixth generation of s...

  • Page 18: 1.1  Target Workloads

    2 ibm system x3850 x6 and x3950 x6 planning and implementation guide 1.1 target workloads the ibm x6 servers introduce new levels of fault tolerance with advanced reliability, availability, and serviceability (ras) features implemented in hardware and software, simplified servicing and upgrades with...

  • Page 19

    Chapter 1. Introduction 3 in x6, flash-based storage choices are as follows: – ibm exflash dimms: where flash based storage modules are installed in memory dimm sockets, thereby having the lowest possible latency and maximized performance. – ibm exflash ssds: based on packs of 1.8-inch solid-state d...

  • Page 20

    4 ibm system x3850 x6 and x3950 x6 planning and implementation guide hypervisors struggle to manage and share the heavy i/o loads typical of performance-intensive workloads. As a result, performance-intensive databases used for core enterprise workloads, such as crm, erp and scm, are left to run on ...

  • Page 21: 1.2  Key Features

    Chapter 1. Introduction 5 1.2 key features the ibm x6 system is fast, agile, and resilient and makes meeting the business needs of your enterprise easier: fast application performance means immediate access to actionable information. Ibm x6 delivers fast application performance, thanks to an innovat...

  • Page 22: 1.3  Positioning

    6 ibm system x3850 x6 and x3950 x6 planning and implementation guide – survive a processor failure: the server is designed to recover from a failed processor and restart automatically. Even if the primary processor (the one used for booting the operating system) fails, the x6 system is designed so i...

  • Page 23

    Chapter 1. Introduction 7 1.4 storage versus in-memory data main memory (ram) is the fastest storage type that can hold a significant amount of data. Data in main memory can be accessed more than a hundred thousand times faster than data on a spinning hard disk, and even flash technology storage is ...

  • Page 24: 1.5  Flash Storage

    8 ibm system x3850 x6 and x3950 x6 planning and implementation guide in database technology, atomicity , consistency , isolation , and durability (acid) is a set of requirements that guarantees that database transactions are processed reliably: a transaction must be atomic. That is, if part of a tra...

  • Page 25: 1.6  Energy Efficiency

    Chapter 1. Introduction 9 ibm flashcache storage accelerator: ibm flashcache storage accelerator is an all-in-one flash-caching product that leverages the speed, management, capacity, and breadth of the ibm high iops adapters and qualified ssds and integrates them into a high speed server-side cachi...

  • Page 26: 1.7  Services Offerings

    10 ibm system x3850 x6 and x3950 x6 planning and implementation guide 1.7 services offerings the x6 systems fit into the services offerings that are already available from ibm global technology services® for system x and bladecenter. More information about these services is available at the followin...

  • Page 27: Technology

    © copyright ibm corp. 2014. All rights reserved. 11 chapter 2. Technology in this chapter, we give an overview of the technologies that ibm includes in the ibm system x3850 x6 and x3950 x6 servers. We describe system architecture and new chassis design with modular structure and the latest intel xeo...

  • Page 28: 2.1  Modular Design

    12 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2.1 modular design ibm x6 rack family consists of the new flagship servers of the ibm x86 server family: ibm system x3850 x6: 4u rack-optimized server scalable to four sockets ibm system x3950 x6: 8u rack-optimized server scalable...

  • Page 29

    Chapter 2. Technology 13 the x6 systems offer a new “bookshelf” design concept that is based on a fixed chassis mounted in a standard rack cabinet. There is no need to pull the chassis in or out of the rack to access components because all components can be accessed either from the front or from the...

  • Page 30

    14 ibm system x3850 x6 and x3950 x6 planning and implementation guide – full-length i/o book. This hot-swap book provides three optional full-length pcie slots, and two of them are capable of hosting a graphics processing unit (gpu) or co-processor adapters with a total power consumption of 300 w. –...

  • Page 31

    Chapter 2. Technology 15 in the middle of the server, there is a passive midplane that connects all modular components to each other. Figure 2-6 shows the x3850 x6 midplane: figure 2-6 x3850 x6 midplane (front side showing compute book connections) each compute book contains one intel xeon processor...

  • Page 32

    16 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2.1.2 storage book the storage book is accessible from the front of the server. The storage book has bays for 1.8-inch (up to 16) or 2.5-inch drives (up to 8), two slots for raid controllers to connect to those drive bays, a front...

  • Page 33

    Chapter 2. Technology 17 figure 2-9 shows the internals of the primary i/o book, as viewed from the front of the server. The black air baffle is raised to show internal components. Figure 2-9 the primary i/o book internals the x3850 x6 has one primary i/o book. The x3950 x6 has two primary i/o books...

  • Page 34

    18 ibm system x3850 x6 and x3950 x6 planning and implementation guide these i/o books are accessible from the rear of the server. Figure 2-10 shows the two i/o books. Figure 2-10 additional i/o books the x3850 x6 server supports up to two additional i/o books of any type; the x3950 x6 server support...

  • Page 35: 2.2  System Architecture

    Chapter 2. Technology 19 for more information about the additional power supplies, see 3.22, “power subsystem” on page 101. 2.2 system architecture this section shows the overall architecture of both the x3850 x6 and x3950 x6. 2.2.1 x3850 x6 figure 2-12 shows the system architecture of the x3850 x6 ...

  • Page 36

    20 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 2-13 shows how the four processors of the x3850 x6 are connected via qpi links. In this configuration, each processor is directly connected to all other processors. Figure 2-13 qpi links between processors in the compute bo...

  • Page 37

    Chapter 2. Technology 21 if you want to take advantage of this recovery, it is important to carefully plan your network and storage connectivity, because if cpu 1 fails, pcie slots 9, 10, and 12 become unavailable. 2.2.2 x3950 x6 the x3950 x6 server has the same architecture adapted to 8-socket conf...

  • Page 38

    22 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 2-15 x3950 x6 system architecture (top half) the 8-socket configuration is formed using the native qpi scalability of the intel xeon processor e7 family. Figure 2-16 shows how the processors are connected together using qpi...

  • Page 39: 2.3  Processors

    Chapter 2. Technology 23 the x3950 x6 is an eight-socket server but also supports four and six processors installed. Figure 2-17 shows the x3950 x6 with only 4 processors installed. On the right of the figure, the effective connectivity is shown, where each processor is connected either directly to ...

  • Page 40

    24 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2.3.1 intel xeon processor e7-4800/8800 v2 product family there are two groups of the intel xeon processor e7 family that are used in the x6 servers and support scaling to separate levels: the intel xeon processor e7-4800 v2 produ...

  • Page 41

    Chapter 2. Technology 25 the new intel xeon processors feature advanced technologies described next. Intel advanced encryption standard - new instructions advanced encryption standard (aes) is an encryption standard that is widely used to protect network traffic and sensitive data. Advanced encrypti...

  • Page 42

    26 ibm system x3850 x6 and x3950 x6 planning and implementation guide turbo boost technology 2.0 the intel xeon e7-8800/4800 v2 processor family brings enhanced capabilities of changing processor speed with new generation of intel turbo boost 2.0 technology. Intel turbo boost technology dynamically ...

  • Page 43

    Chapter 2. Technology 27 each processor has some amount of memory, connected directly to the processor. To access memory connected to another processor, each processor uses qpi links through another processors. This design creates a non-uniform memory access (numa) system. Similarly, i/o can be loca...

  • Page 44

    28 ibm system x3850 x6 and x3950 x6 planning and implementation guide machine check architecture recovery the intel xeon processor e7 family also features machine check architecture (mca) recovery, a ras feature that enables the handling of system errors that otherwise require that the operating sys...

  • Page 45

    Chapter 2. Technology 29 2.3.2 compute books the core modular element of the x6 design is a compute book. It contains the following components: one intel xeon processor e7-4800 v2/8800 v2 product families processor 24 memory dimm slots two dual-motor fans figure 2-20 shows the compute book. Figure 2...

  • Page 46: 2.4  Memory

    30 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 2-21 and figure 2-22 show the left and right sides of the compute book, respectively. On the left side, there are one processor and 12 dimm slots. On the right side, there are 12 dimm slots for a total of 24 dimms per compu...

  • Page 47

    Chapter 2. Technology 31 figure 2-23 shows the processor’s memory architecture. Figure 2-23 intel xeon processor e7-4800/8800 v2 memory architecture 2.4.1 operational modes two memory modes are supported by the intel xeon processor e7-4800 v2/8800 v2 product families: performance mode: in this opera...

  • Page 48

    32 ibm system x3850 x6 and x3950 x6 planning and implementation guide since data is moved using both channels at once, additional advanced memory protection schemes can be implemented to provide protection against both single-bit and multi-bit errors: – chipkill, also known as single device data cor...

  • Page 49

    Chapter 2. Technology 33 2.4.2 memory mirroring and rank sparing in addition to performance and ras modes, the memory subsystem has additional reliability, availability, and serviceability (ras) features that can be enabled from the unified extensible firmware interface (uefi): memory mirroring rank...

  • Page 50

    34 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 2-25 shows how memory mirroring is implemented in performance mode (left) and ras mode (right). Figure 2-25 memory mirroring with used with performance mode (left) and ras mode (right) the following memory mirroring rules a...

  • Page 51

    Chapter 2. Technology 35 when this threshold is reached, the content is copied to its spare rank. The failed rank is then taken offline, and the spare counterpart is activated for use. In rank sparing mode, one rank per memory channel is configured as a spare. The spare rank must have identical or l...

  • Page 52

    36 ibm system x3850 x6 and x3950 x6 planning and implementation guide the following configuration rules apply for rank sparing: memory rank sparing is not supported if memory mirroring is enabled. The spare rank must have identical or larger memory capacity than all the other ranks on the same ddr3 ...

  • Page 53: 2.5  Pcie 3.0

    Chapter 2. Technology 37 the algorithm uses short and long term thresholds per memory rank with leaky bucket and automatic sorting of memory pages with the highest correctable error counts. First, it leverages hardware recovery features, followed by software recovery features, to optimize recovery r...

  • Page 54: 2.6  Internal Storage

    38 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2.6 internal storage in this section, we give an overview of different technologies used in the storage subsystem of the x6 servers family: 2.6.1, “storage book” on page 38 2.6.2, “ibm exflash memory-channel storage” on page 38 2....

  • Page 55

    Chapter 2. Technology 39 figure 2-28 shows an ibm exflash dimm. Figure 2-28 ibm exflash dimm the ibm exflash dimm modules are installed into ddr3 slots and use memory channels of the processors. Data transfers between processors and exflash dimms run directly without any additional controllers such ...

  • Page 56

    40 ibm system x3850 x6 and x3950 x6 planning and implementation guide ibm exflash dimms have the following key characteristics: provides ultrahigh overall performance using array of exflash dimms in a parallel manner. Offers up to 12.8 tb total flash storage capacity per server with ibm 200 gb and 4...

  • Page 57

    Chapter 2. Technology 41 each exflash unit can accommodate up to eight 1.8-inch hot-swap front-accessible ssds, and it occupies four 2.5-inch sas hard disk drive bays. You can install the following number of exflash units: the x3850 x6 can have up to 16x 1.8-inch ssds with up to two exflash units (u...

  • Page 58

    42 ibm system x3850 x6 and x3950 x6 planning and implementation guide unacceptable. These reliability features provide a predictable lifetime and up to 25 years of data retention. Figure 2-31 shows the ibm 2.4tb high iops mlc duo adapter: figure 2-31 ibm 2.4tb high iops mlc duo adapter for more info...

  • Page 59

    Chapter 2. Technology 43 figure 2-32 shows the typical flashcache deployment scenario: figure 2-32 ibm flashcache storage accelerator deployment the flashcache storage accelerator also works in virtualized environment, transparently caching data of virtual machines. Figure 2-33 shows the typical fla...

  • Page 60: 2.7  Uefi

    44 ibm system x3850 x6 and x3950 x6 planning and implementation guide for more information, see the following reference: the benefits of ibm flashcache storage accelerator in the enterprise solutions, redp5080 http://www.Redbooks.Ibm.Com/abstracts/redp5080.Html?Open 2.7 uefi the unified extensible f...

  • Page 61: 2.9  Scalability

    Chapter 2. Technology 45 imm2 provides these features: an embedded web server, which gives you remote control from any standard web browser with oracle java plug-in installed. A command-line interface (cli), which the administrator can use from a telnet or ssh session. Secure sockets layer (ssl) and...

  • Page 62

    46 ibm system x3850 x6 and x3950 x6 planning and implementation guide the x3850 x6 and x3950 x6 servers utilize native qpi scaling capabilities to achieve 4-socket and 8-socket configurations. Unlike ex5 systems, there are no external connectors and cables for x6 systems, all interconnects are integ...

  • Page 63: Product Information

    © copyright ibm corp. 2014. All rights reserved. 47 chapter 3. Product information in this chapter, we describe the details of the components in the ibm system x3850 x6 and the ibm system x3950 x6. This chapter contains the following topics: 3.1, “product features” on page 48 3.2, “specifications” o...

  • Page 64: 3.1  Product Features

    48 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.1 product features the increasing demand for cloud computing and analytics workloads by enterprises to meet social, mobile, and big data requirements drives innovation to find new ways to build informational systems. Clients are...

  • Page 65

    Chapter 3. Product information 49 intel virtualization technology integrates hardware-level virtualization hooks that allow operating system vendors to better utilize the hardware for virtualization workloads. Intel advanced vector extensions (avt) significantly improve floating-point performance fo...

  • Page 66

    50 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.1.3 resilient platform the server provides many features to simplify serviceability and increase system uptime: upward integration modules for standard hypervisors enable the creation and management of policies to maintain high ...

  • Page 67: 3.2  Specifications

    Chapter 3. Product information 51 ibm systems director provides proactive systems management. It offers comprehensive systems management tools that help to increase uptime, reduce costs, and improve productivity through advanced server management capabilities. Solid-state drives (ssds) offer signifi...

  • Page 68

    52 ibm system x3850 x6 and x3950 x6 planning and implementation guide maximum internal storage x3850 x6: up to 12.8 tb with 1.6 tb 2.5" sas ssds, or up to 9.6 tb with 1.2 tb 2.5" sas hdds, up to 8 tb with 1 tb 2.5" nl sas/sata hdds, up to 12.8 tb with 800 gb 1.8" sata ssds. X3950 x6: up to 25.6 tb w...

  • Page 69

    Chapter 3. Product information 53 the x3850 x6 and x3950 x6 servers are shipped with the following items: rail kit cable management bracket kit (2 brackets for x3850 x6, 4 brackets for the x3950 x6) 2.8 m (9.18 in) c13-c14 power cord (one for each power supply) statement of limited warranty importan...

  • Page 70

    54 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.3 standard models of x6 servers table 3-2 lists the standard models of x3850 x6. Table 3-2 x3850 x6 models model intel xeon processor a a. Each processor is installed in a compute book. Detail: quantity, model, number of cores, ...

  • Page 71

    Chapter 3. Product information 55 table 3-3 lists the standard models of x3950 x6. Table 3-3 x3950 x6 models see table 3-1 on page 51 for information about standard features of the server. C. Models with 6 slots have the primary i/o book (4 slots) and storage book (2 slots) standard. Models with 9 s...

  • Page 72: 3.4  Physical Design

    56 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.4 physical design the x3850 x6 and x3950 x6 servers are based on a new design where all major server components are located in easy-to-service modules called “books”, and each book can be inserted into the server chassis from ei...

  • Page 73

    Chapter 3. Product information 57 figure 3-3 shows the front view of the x3950 x6 server. Figure 3-3 front view of the x3950 x6 as shown in figure 3-3, the x3950 x6 is the equivalent of two x3850 x6 servers. It can have up to eight compute books (four compute books minimum), and two storage books, e...

  • Page 74: 3.5  Ports and Controls

    58 ibm system x3850 x6 and x3950 x6 planning and implementation guide the rear of the x3850 x6 has up to three i/o books: one primary i/o book with 3 pcie 3.0 slots, one ml2-slot for ethernet adapters, two hot-swappable fans, one imm2 ethernet port, 4 usb 2.0 ports, video port, and serial port. Two ...

  • Page 75

    Chapter 3. Product information 59 figure 3-6 shows ports, controls, and leds on the front operator panel. Figure 3-6 ports, controls and leds located on the front operator panel the front operator panel contains the following controls: power button/led: press the power button to turn the server on a...

  • Page 76

    60 ibm system x3850 x6 and x3950 x6 planning and implementation guide usb 2.0 port: connect a usb device, such as a usb mouse, keyboard, or other device, to this connector. Usb 3.0 ports: connect a usb device, such as a usb mouse, keyboard, or other device, to any of these connectors. Video connecto...

  • Page 77

    Chapter 3. Product information 61 figure 3-8 shows the available options that can be shown on the lcd system information panel. Figure 3-8 system properties shown on the lcd system information panel figure 3-9 shows an example of the information that you may see on the display panel. Figure 3-9 lcd ...

  • Page 78

    62 ibm system x3850 x6 and x3950 x6 planning and implementation guide the lcd system information display panel displays the following types of information about the server: imm system error log system vpd information: – machine type and serial number – universal unique identifier (uuid) string syste...

  • Page 79

    Chapter 3. Product information 63 here we describe the components in the diagram: 3v battery error led: when this led is lit, it indicates that a standard i/o book battery error has occurred. I/o board error led: when this led is lit, it indicates that an error has occurred on the primary i/o book b...

  • Page 80: 3.6  Compute Book

    64 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.6 compute book in this section, we describe the central component of the x6 family servers, the compute book. We discuss the compute book’s design and its main components, explaining compute books placement order for the 4u and ...

  • Page 81

    Chapter 3. Product information 65 the internals of the compute book are shown in figure 3-12. The system board of the compute book has two sides on which all components reside. Figure 3-12 compute book with the side covers removed figure 3-13 shows the compute book from the front where two hot-swap ...

  • Page 82

    66 ibm system x3850 x6 and x3950 x6 planning and implementation guide the system board of the compute book also contains a set of error leds for each memory module, which show failed modules when the light path diagnostic button is pressed. Figure 3-14 shows a diagram with dimm error leds of the com...

  • Page 83: 3.7  Processor Options

    Chapter 3. Product information 67 figure 3-15 shows how the compute books are numbered. Figure 3-15 compute book numbering the installation sequence for compute books is as follows: x3850 x6: 1 compute book - bay 1 2 compute books - bays 1, 2 4 compute books - all bays x3950 x6: 4 compute books - ba...

  • Page 84

    68 ibm system x3850 x6 and x3950 x6 planning and implementation guide table 3-4 processor options part number feature code intel xeon processor (installed in a compute book) a a. Processor detail: model, core count, core frequency, l3 cache, tdp power rating memory bus speed b b. Ras mode / performa...

  • Page 85: 3.8  Memory

    Chapter 3. Product information 69 3.8 memory ibm ddr3 memory is compatibility tested and tuned for optimal system x performance and throughput. Ibm memory specifications are integrated into the light path diagnostics for immediate system performance feedback and optimum system uptime. From a service...

  • Page 86

    70 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.8.1 memory options in this section, we describe available memory options, memory characteristics, and speeds depending on the selected memory mode. Table 3-5 lists the memory options that are available for x3850 x6 and x3950 x6 ...

  • Page 87

    Chapter 3. Product information 71 3.8.2 memory population order the x3850 x6 and x3950 x6 servers have two main memory operation modes: performance mode and ras (lockstep) mode, and also two additional modes: mirroring and rank sparing. The main operation modes can be combined with additional modes,...

  • Page 88

    72 ibm system x3850 x6 and x3950 x6 planning and implementation guide table 3-7 dimms placement order as shown in table 3-7, in mirroring mode dimms must be installed in pairs (minimum of one pair per cpu). Both dimms in a pair must be identical in type and size. Chipkill is supported only in ras mo...

  • Page 89

    Chapter 3. Product information 73 figure 3-16 shows the dimm slot numbering on each side of the compute book. Figure 3-16 dimm slot numbering for rank sparing, the spare rank must have identical or larger memory capacity than all the other ranks (sparing source ranks) on the same channel. A minimum ...

  • Page 90

    74 ibm system x3850 x6 and x3950 x6 planning and implementation guide this new and innovative technology brings storage electrically closer to the processor subsystem, thereby improving performance considerably. Table 3-8 shows the available ibm exflash dimm options. Table 3-8 ibm exflash memory cha...

  • Page 91: 3.10  Storage Subsystem

    Chapter 3. Product information 75 3.10 storage subsystem in this section, we describe the storage subsystem of x6 servers, including storage book, supported raid adapters, hard drives, and exflash ssd drives. We also discuss high performance ibm high iops adapters, pcie ssd storage adapters, designe...

  • Page 92

    76 ibm system x3850 x6 and x3950 x6 planning and implementation guide the storage book supports up to eight drives (2.5-inch format) or up to 16 1.8-inch solid-state drives (using two 8-bay exflash units) in the following combinations: 4x 2.5-inch hot-swap drive bays 8x 2.5-inch hot-swap drive bays ...

  • Page 93

    Chapter 3. Product information 77 figure 3-20 shows the location of sas backplanes in the storage book as well as the raid controllers that connect to the backplanes. Figure 3-20 sas backplanes location in the storage book table 3-9 shows the two backplanes. Table 3-9 internal storage expansion opti...

  • Page 94

    78 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.10.3 raid controllers the storage book contains two pcie 3.0 x8 slots for raid controllers or sas hbas for use with the internal drives. If two exflash ssd units (16x 1.8-inch ssds) are used, then two controllers are required. F...

  • Page 95

    Chapter 3. Product information 79 the serveraid m5210 sas/sata controller has the following specifications: eight internal 12 gbps sas/sata ports two x4 hd mini-sas internal connectors (sff-8643) supports connections to sas/sata drives and sas expanders supports raid levels 0, 1, and 10 supports rai...

  • Page 96

    80 ibm system x3850 x6 and x3950 x6 planning and implementation guide for more information about 1.8-inch enterprise ssds, see 2.6.3, “ibm exflash ssd technology” on page 40. 00aj217 a4uc ibm 800gb sas 2.5" mlc g3hs enterprise ssd 8 16 00aj222 a4ud ibm 1.6tb sas 2.5" mlc g3hs enterprise ssd 8 16 2.5...

  • Page 97

    Chapter 3. Product information 81 3.10.5 ibm high iops adapter the ibm high iops ssd pcie adapters are high performance ssd storage adapters with very low latency. Depending on the workload, the high iops adapter can deliver throughput of up to 900,000 iops and latency less than 100 microseconds. Ta...

  • Page 98

    82 ibm system x3850 x6 and x3950 x6 planning and implementation guide the serveraid m5120 sas/sata controller has the following specifications: eight external 6 gbps sas/sata ports up to 6 gbps throughput per port two external x4 mini-sas connectors (sff-8088) supports raid 0, 1, and 10 supports rai...

  • Page 99

    Chapter 3. Product information 83 table 3-17 lists the drives supported by exp2512 external expansion enclosures. Table 3-17 drive options for exp2512 external expansion enclosures table 3-18 lists the hard disk drives supported by exp2524 external expansion enclosures. Table 3-18 drive options for ...

  • Page 100: 3.11  I/o Subsystem

    84 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.11 i/o subsystem in this section, we describe the third type of modular component used in the x3850 x6 and x3950 x6, i/o books. The following topics are covered: 3.12, “primary i/o book” on page 85 3.13, “half-length i/o books a...

  • Page 101: 3.12  Primary I/o Book

    Chapter 3. Product information 85 the pcie lanes used in the i/o books and storage book are connected to installed processors as follows: the slots in the primary i/o book and storage i/o book connect either to processor 1 or processor 2. The slots in the optional i/o book in bay 1 connect to proces...

  • Page 102

    86 ibm system x3850 x6 and x3950 x6 planning and implementation guide the primary i/o book also contains core logic: integrated management module ii (imm2) two trusted platform modules (tpms) two hot-swap fan modules internal usb port for an embedded hypervisor dedicated 1 gigabit ethernet port dedi...

  • Page 103

    Chapter 3. Product information 87 figure 3-25 depicts the primary i/o book removed from the server and shows the internal components. Figure 3-25 the primary i/o book removed showing the internal components as shown in figure 3-25, the primary i/o book has an internal usb port for the embedded hyper...

  • Page 104

    88 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 3-26 optional i/o book locations the part numbers for the optional i/o books are listed in table 3-19. The table also lists maximums supported. The x3850 x6 supports, at most, two optional i/o books, and they can be both ha...

  • Page 105

    Chapter 3. Product information 89 as we show in the system block diagrams in 2.2, “system architecture” on page 19, the first optional i/o book connects to the processor in compute book 3 and the second optional i/o book connects to the processor in compute book 4. This bus connectivity results in t...

  • Page 106

    90 ibm system x3850 x6 and x3950 x6 planning and implementation guide the combined power consumption of all adapters installed in the full-length i/o book cannot exceed 300w. Figure 3-29 shows the full-length i/o book. Figure 3-29 full-length i/o book figure 3-30 shows the full-length i/o book with ...

  • Page 107

    Chapter 3. Product information 91 figure 3-31 shows a full-length i/o book and a half-length i/o book installed in the server. Figure 3-31 i/o books installed in the x3850 x6 the full-length i/o book supports hot-swap pcie adapters. See 3.14, “hot-swap adapter support” on page 91 for specifics. 3.14...

  • Page 108: 3.15  Network Adapters

    92 ibm system x3850 x6 and x3950 x6 planning and implementation guide table 3-20 adapters that support hot-swap 3.15 network adapters the x3850 x6 and x3950 x6 servers support ml2 adapters installed in the custom ml2 slot. This slot supports adapters with either two 10gb ports or four gigabit ports ...

  • Page 109

    Chapter 3. Product information 93 table 3-21 ml2 adapter options the server also supports a variety of other ethernet and infiniband network adapters as listed in table 3-22. The maximum quantity listed is for configurations with all processors and i/o books installed. Table 3-22 network adapters pa...

  • Page 110

    94 ibm system x3850 x6 and x3950 x6 planning and implementation guide for more information, see the list of ibm redbooks product guides in the networking adapters category: http://www.Redbooks.Ibm.Com/portals/systemx?Open&page=pg&cat=networkadapters 3.16 storage host bus adapters the following table...

  • Page 111

    Chapter 3. Product information 95 3.17 gpu adapters and co-processors the server supports co-processors and graphics processing units (gpus) that are listed in table 3-24. Each is installed in a full-length i/o book. No other adapter can be installed in the full-length i/o book. The full-length i/o ...

  • Page 112: 3.18  Partitioning

    96 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.18 partitioning the 8-socket x3950 x6 server has ability to form a 2-node configuration using a feature called partitioning . The x3950 x6 server electronically splits to two independent servers or nodes which use their own set ...

  • Page 113

    Chapter 3. Product information 97 remote console by way of a serial over lan remote access to the system event log predictive failure analysis and integrated alerting features (for example, by using simple network management protocol, snmp) remote presence, including remote control of server by usin...

  • Page 114

    98 ibm system x3850 x6 and x3950 x6 planning and implementation guide for details about this technology, see the trusted computing group (tcg) tpm main specification at the following website: http://www.Trustedcomputinggroup.Org/resources/tpm_main_specification 3.19.4 light path diagnostics light pa...

  • Page 115

    Chapter 3. Product information 99 figure 3-35 shows the location of the dimms connectors leds on the one side of the compute book. Figure 3-35 dimms connectors location in addition to the light path diagnostic leds, you can use the lcd system information display located on the front operator panel, ...

  • Page 116: 3.21  Hot-Swap Capabilities

    100 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3.20 integrated virtualization the x6 servers support vmware esxi installed on a usb memory key. The key is installed in a usb port that is located on the primary i/o book. Figure 3-36 shows the location of the usb port for the e...

  • Page 117: 3.22  Power Subsystem

    Chapter 3. Product information 101 table 3-26 shows which components are hot-swap and which are not. Table 3-26 hot-swap capabilities for additional information about hot-swap capabilities of the optional i/o books, see 3.14, “hot-swap adapter support” on page 91. 3.22 power subsystem the x3850 x6 s...

  • Page 118

    102 ibm system x3850 x6 and x3950 x6 planning and implementation guide it is not advised that you mix the two 1400 w options (44x4152 for low altitude and 44x4150 for high altitude), as you will nullify the high-altitude capabilities of 44x4150. You cannot mix the ac power supplies with the dc power...

  • Page 119: 3.23  Fans and Cooling

    Chapter 3. Product information 103 the x3850 x6 server supports three modes of redundancy based on the power supply configuration, system load, and the power policy configuration controlled by the imm: non-redundant fully system redundant redundant with reduced performance (throttling) the imm must ...

  • Page 120

    104 ibm system x3850 x6 and x3950 x6 planning and implementation guide fan zone 3 = fan packs 3 and 4 controlled by the temperature readings near compute book 2, and pcie slots 7 and 8 fan zone 4 = fan packs 5 and 6 controlled by the temperature readings near compute book 3, and pcie slots 4, 5, and...

  • Page 121

    Chapter 3. Product information 105 when the need arises to upgrade the server to six or eight processors, simply purchase more compute books with the same processor model, plus additional power supplies (as determined by using the power configurator), i/o books, adapters and drives as needed. Upgrad...

  • Page 122

    106 ibm system x3850 x6 and x3950 x6 planning and implementation guide there are key considerations for this upgrade: processor support: intel xeon e7-4800 v2 processors cannot be used in an x3950 x6. If your x3850 x6 has compute books with e7-4800 v2 processors, then these must be replaced with com...

  • Page 123: Infrastructure Planning

    © copyright ibm corp. 2014. All rights reserved. 107 chapter 4. Infrastructure planning the ibm system x3850 x6 and x3950 x6 are enterprise-class intel processor-based servers for mission-critical applications. These servers have the potential to replace an entire rack of conventional servers. This ...

  • Page 124

    108 ibm system x3850 x6 and x3950 x6 planning and implementation guide 4.1 physical and electrical specifications the x3850 x6 and x3950 x6 servers have the following physical, electrical, and environmental specifications: x3850 x6 dimensions: – height: 173 mm (6.8 in) – width: 482 mm (19.0 in) – de...

  • Page 125

    Chapter 4. Infrastructure planning 109 • maximum configuration: 1.7 kva btu output: – minimum configuration: 546 btu/hr (160 watts) – maximum configuration: 10,912 btu/hr (3,200 watts) noise level: – 6.6 bels (operating) – 6.4 bels (idle) 4.2 rack selection and rack options the server supports the r...

  • Page 126

    110 ibm system x3850 x6 and x3950 x6 planning and implementation guide for more information, see the list of ibm redbooks product guides in the rack cabinets and options category: http://www.Redbooks.Ibm.Com/portals/systemx?Open&page=pg&cat=rack the server supports the rack console switches and moni...

  • Page 127: 4.3  Floor Clearance

    Chapter 4. Infrastructure planning 111 shipping requirements for x6 the full-length i/o book cannot be shipped while installed in the x3850 x6 or x3950 x6 server. This i/o book and its adapters must be shipped separately from the server. 4.3 floor clearance the server is a traditional rack design an...

  • Page 128

    112 ibm system x3850 x6 and x3950 x6 planning and implementation guide the doors have no moving parts; the fans in the equipment move air though the heat exchanger as easily as a standard rack door. In the event of a disruption in water flow, the rack reverts to standard air cooling. Each door is ca...

  • Page 129: 4.5  Power Advice

    Chapter 4. Infrastructure planning 113 4.5 power advice in this section, we provide power planning considerations, explanations of the power subsystem in the x3850 x6 and x3950 x6 servers and examples of best practices for configuring power connections and power distribution. We also describe advice...

  • Page 130

    114 ibm system x3850 x6 and x3950 x6 planning and implementation guide 4.5.2 power supply redundancy the four x3850 x6 power supply bays are divided into two power groups to support n+n power supply and power feed redundancy (where n = 1 or 2 depending on your system configuration and load). The pow...

  • Page 131

    Chapter 4. Infrastructure planning 115 four 900w four 1400w four 750w dc there are two different 1400w power supply units available, a standard 1400w psu and a high altitude 1400w psu. The high altitude psu should be used when the server’s location is at an altitude higher than 5000 meters from sea ...

  • Page 132

    116 ibm system x3850 x6 and x3950 x6 planning and implementation guide an example in figure 4-3 shows a correctly balanced and redundant system with four 900w power supplies installed, connected to 2 separate feeds. Figure 4-3 correctly balanced and redundant power if using a mix of 900w and 1400w p...

  • Page 133

    Chapter 4. Infrastructure planning 117 the following rules apply when installing power supplies in the x3950 x6: for four power supply configurations, install the power supplies in bays 2, 6, 3, and 7 with each power supply on separate input power feeds. Power supplies must be all ac or all dc. Pair...

  • Page 134

    118 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2. To set the power policy of your system, ensure that you are on the policies tab and click the change button next to current policy power supply redundancy. Figure 4-5 displays the power policies page in the imm2 for an x3850 x...

  • Page 135

    Chapter 4. Infrastructure planning 119 4.5.6 additional power settings in the imm2 in this section, we discuss the power capping and power allocation information you can set and view in the server power management page via the imm2 to help you manage your systems power consumption. Power capping fro...

  • Page 136

    120 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 4-7 displays the power allocation page in the imm2. Figure 4-7 power allocation in the server power management page.

  • Page 137

    Chapter 4. Infrastructure planning 121 the power history tab will also give you a representation of your server’s power consumption over a selected period of time as seen in figure 4-8. Figure 4-8 history of power consumption 4.5.7 examples of power connections the ibm system x3850 x6 and x3950 x6 p...

  • Page 138: 4.6  Cooling Advice

    122 ibm system x3850 x6 and x3950 x6 planning and implementation guide 4.6 cooling advice it is important to maintain sufficient temperature and humidity control in the data center environment to prevent thermal failures. Use the following information to help plan for proper cooling of the x6 server...

  • Page 139

    Chapter 4. Infrastructure planning 123 4.7 uninterruptible power supply units there are several rack mounted uninterruptible power supply (ups) units that can be used with the x3850 x6 and x3950 x6 servers. The size and type of the ups you select will depend on your countries voltage, type, and numb...

  • Page 140

    124 ibm system x3850 x6 and x3950 x6 planning and implementation guide 4.8 pdu and line cord selection there are a number of power distribution units (pdus) that can be used to power the x6 system. The size and type of the pdus that you select will depend on your country voltage, type, and number of...

  • Page 141

    Chapter 4. Infrastructure planning 125 4.8.2 pdu and line cord options some pdus have attached line cords while others require a line cord to be ordered separately based on your requirement of three-phase power or single-phase power. See table 4-7 for pdu, line cord, and phase options for both north...

  • Page 142

    126 ibm system x3850 x6 and x3950 x6 planning and implementation guide 46m4004 ibm 1u 12 c13 aem dpi pdu 40k9612 1ph 220v-240v 32a iec 309 p+n+g 12 / c13 40k9613 1ph 220v-240v 63a iec 309 p+n+g 40k9617 1ph 230v-240v 32a aus/nz 3112 40k9618 1ph 220v 30a ksc 8305 40k9611 3ph y 380v-415v 32a (32a/ph) i...

  • Page 143

    Chapter 4. Infrastructure planning 127 71762mx ibm ultra density enterprise pdu c19 pdu+ (ww) 40k9612 1ph 220v-240v 32a iec 309 p+n+g 9 / c19 3 / c13 40k9613 1ph 220v-240v 63a iec 309 p+n+g 40k9617 1ph 230v-240v 32a aus/nz 3112 40k9618 1ph 220v 30a ksc 8305 40k9611 3ph y 380v-415v 32a (32a/ph) iec 3...

  • Page 144

    128 ibm system x3850 x6 and x3950 x6 planning and implementation guide front-end pdus - international 39y8934 dpi 32 amp front-end pdu included 1ph 220v-240v 32a iec 309 p+n+g 3 / c19 39y8935 dpi 63 amp front-end pdu included 1ph 220v-240v 63a iec 309 p+n+g 3 / c19 universal rack pdus 39y8951 dpi un...

  • Page 145

    Chapter 4. Infrastructure planning 129 39y8956 dpi universal rack pdu with south africa lc a included 1ph 220v-250v 16a south africa sabs 164 7 / c13 optional line cord 2.5m 39m5290 optional line cord 4.3m 39m5291 39y8957 dpi universal rack pdu with uk lc b included 1ph 230v 13a uk bs 1363/a 7 / c13...

  • Page 146

    130 ibm system x3850 x6 and x3950 x6 planning and implementation guide for additional information about pdus and cables, see one of the following guides: for the north america and japan guide, see this website: http://www.Ibm.Com/support/techdocs/atsmastr.Nsf/webindex/wp101526 for the international ...

  • Page 147: Preparing The Hardware

    © copyright ibm corp. 2014. All rights reserved. 131 chapter 5. Preparing the hardware this chapter provides assistance for making configuration, monitoring, and maintenance decisions when implementing an x3850 x6 or x3950 x6 server. Some of the topics described in this section are only guidelines. ...

  • Page 148

    132 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.1 configuring the imm2 settings for any successful server implementation, you should provide access to perform troubleshooting or routine maintenance. The x3850 x6 and x3950 x6 servers ship standard with the integrated manageme...

  • Page 149

    Chapter 5. Preparing the hardware 133 part of the information that is stored in the imm2 can be accessed with f1-setup by selecting system settings integrated management module. Figure 5-1 shows the first panel of the imm2 configuration panel. Figure 5-1 integrated management module panel the next s...

  • Page 150

    134 ibm system x3850 x6 and x3950 x6 planning and implementation guide imm2 default configuration the default network connection for the imm2 on the x3850 x6 is through the system management port on the back of the server. See figure 5-3 for the location of the port. Here are the default settings of...

  • Page 151

    Chapter 5. Preparing the hardware 135 5.1.4 imm2 dedicated versus shared ml2 ethernet port when configured as dedicated , you are connecting to the network via the system management port. As shown in figure 5-3, the system management port is located from the rear of the server to the right side of t...

  • Page 152

    136 ibm system x3850 x6 and x3950 x6 planning and implementation guide to maintain separation between public and management traffic, network teaming software must be used to establish a vlan to be used by the server to send public-tagged traffic to the network switch. The switch port must be configu...

  • Page 153

    Chapter 5. Preparing the hardware 137 5.1.6 imm2 communications troubleshooting the integrated management module user guide is an excellent guide to help you with every aspect of troubleshooting the imm. Download the guide from this website: http://ibm.Com/support/entry/portal/docdisplay?Lndocid=mig...

  • Page 154

    138 ibm system x3850 x6 and x3950 x6 planning and implementation guide system status the first panel that you will see after completing the login credentials and setting your time-out period, is the system status and hardware health page, as shown in figure 5-6. This panel provides a quick summary r...

  • Page 155

    Chapter 5. Preparing the hardware 139 virtual light path diagnostics if you are physically in front of the server, it is easy to track hardware problems by noticing the first tier of light path diagnostics, using the led panel at the front of the server shown in figure 5-7. Figure 5-7 led light path...

  • Page 156

    140 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 5-9 shows the imm event log for various failures. Figure 5-9 imm event log for various errors remote control certain problems require that you get into the operating system or f1-setup to detect them or fix them. For remot...

  • Page 157

    Chapter 5. Preparing the hardware 141 figure 5-10 shows the options available for starting a remote control session. Figure 5-10 integrated management module remote control session start-up panel imm2 remote control provides the following features: the remote control provides you with the same capab...

  • Page 158

    142 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.2 uefi settings for performance the unified extensible firmware interface (uefi) is the interface between the operating system (os) and platform firmware. Uefi provides a modern, well-defined environment for booting an os and r...

  • Page 159

    Chapter 5. Preparing the hardware 143 you can change uefi settings to meet your system requirements. In this section, we provide an overview of the uefi settings for tuning your system for performance. Figure 5-12 uefi settings main panel choose system settings to access the system settings options ...

  • Page 160

    144 ibm system x3850 x6 and x3950 x6 planning and implementation guide table 5-1 provides an overview of the most common uefi performance optimizing settings and their default state. Table 5-1 uefi default settings tuning the x3850 x6 server for performance is a complicated topic, because it depends...

  • Page 161

    Chapter 5. Preparing the hardware 145 5.2.1 operating modes ibm x6 servers are designed to provide optimal performance with reasonable power consumption, which depends on the operating frequency and voltage of the processors and memory subsystem. The operating frequency and voltage of the processors...

  • Page 162: 5.3  Uefi Common Settings

    146 ibm system x3850 x6 and x3950 x6 planning and implementation guide the operating modes are as follows; the default mode is efficiency - favor performance : minimal power: select this choice to minimize the absolute power consumption of the system during operation. Server performance in this mode...

  • Page 163

    Chapter 5. Preparing the hardware 147 5.3.1 system power settings power settings include basic power-related configuration options: ibm systems director active energy manager (default: capping enabled): the active energy manager option enables the server to use the power capping feature of active en...

  • Page 164

    148 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.3.2 processor settings processor settings control the various performance and power features that are available on the installed xeon processor. Figure 5-16 shows the uefi processor system settings window with the default value...

  • Page 165

    Chapter 5. Preparing the hardware 149 intel virtualization technology (default: enable) this option enables the processor hardware acceleration feature for virtualization. See “intel virtualization technology” on page 25 for more information. Enable smx (default: disabled) this option enables safer ...

  • Page 166

    150 ibm system x3850 x6 and x3950 x6 planning and implementation guide the options in the memory settings panel are as follows: memory mode (default: independent): this option enables independent or lockstep memory mode. In independent mode, all channels operate independently with smi links operatin...

  • Page 167

    Chapter 5. Preparing the hardware 151 5.3.4 serveraid m5210 raid controller configuration the uefi provides access to the lsi megaraid configuration utility as used by the serveraid m5210 sas/sata controller. Access the menu in uefi by selecting system settings storage lsi megaraid configuration uti...

  • Page 168

    152 ibm system x3850 x6 and x3950 x6 planning and implementation guide hardware components: displays the battery status and the status of temperature sensors, fans, and power supplies. You can also view additional properties and perform additional operations using the advanced link. Some options wil...

  • Page 169

    Chapter 5. Preparing the hardware 153 3. Access the controller management panel, as seen in figure 5-22. Figure 5-22 controller management main menu panel 4. From the controller management main menu panel, access the advanced option, as seen in figure 5-23. Figure 5-23 advanced controller management...

  • Page 170

    154 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5. From the advanced controller management menu, select the manage megaraid advanced software options as seen in figure 5-24. Figure 5-24 manage megaraid advanced software options 6. Ensure that the megaraid advanced software opt...

  • Page 171

    Chapter 5. Preparing the hardware 155 9. Return back to the main menu as seen in figure 5-21 on page 152. From here, access the configuration management panel as seen in figure 5-27. Figure 5-27 configuration management panel 10.From the configuration management, select create virtual drive as seen ...

  • Page 172

    156 ibm system x3850 x6 and x3950 x6 planning and implementation guide 12.Select the save configuration option and confirm your intentions to create your raid array as seen in figure 5-30. Figure 5-30 warning and confirmation of raid array creation 13.Select the confirm option to mark an x in the bo...

  • Page 173

    Chapter 5. Preparing the hardware 157 pcie adapter slots table 5-3 shows the associated cpu with each pcie adapter slot in the system. You cannot install an adapter in an i/o book that does not have an associated compute book. Table 5-3 processor and pcie associations figure 5-32 shows the numbering...

  • Page 174

    158 ibm system x3850 x6 and x3950 x6 planning and implementation guide pcie performance the x3850 x6 and x3950 x6 offer a large quantity of pcie slots. To ensure the best possible performance, it is important to understand the design of the servers and how the pcie slots connect to each processor. F...

  • Page 175: 5.5  Hot-Swap Procedures

    Chapter 5. Preparing the hardware 159 5.5 hot-swap procedures the x3850 x6 and x3950 x6 hardware supports the ability to hot-swap certain components of the server. Hot-swap refers to adding and removing certain hardware components while the server is running. The following resources can be hot-swapp...

  • Page 176

    160 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.5.2 hot-swapping an i/o book as described in 3.14, “hot-swap adapter support” on page 91, the full-length i/o books and half-length i/o books are hot-swap components. The following requirements are for hot-swap: only adapters i...

  • Page 177

    Chapter 5. Preparing the hardware 161 figure 5-36 location of the attention button and leds on an optional i/o book 3. Wait for all three power indicators to turn off (see figure 5-36). It is now safe to remove the i/o book. 4. Open the i/o book cam handle. 5. Slide the i/o book out and away from th...

  • Page 178

    162 ibm system x3850 x6 and x3950 x6 planning and implementation guide hot add of an i/o book follow these steps to add an i/o book while the operating system is running: 1. Install the adapters into the i/o book. For the full-length i/o book, connect the supplied auxiliary power cables to the adapt...

  • Page 179

    Chapter 5. Preparing the hardware 163 table 5-4 describes the meaning of the i/o book attention led. See figure 5-36 on page 161 for the location of this led. Table 5-4 attention indicator for optional i/o books table 5-5 describes the meaning of the i/o book power led. See figure 5-36 on page 161 f...

  • Page 180

    164 ibm system x3850 x6 and x3950 x6 planning and implementation guide here we list information regarding partitioning an x3950 x6: by default, a single partition will be created out of both halves of the x3950 x6 (that is, to form a single system image using all eight processors). Each half indepen...

  • Page 181

    Chapter 5. Preparing the hardware 165 unassigned node: the servers are not a part of the assigned group and need to be added. The processors cannot be accessed and the node will not function in an unassigned state. After it has been added, the node can be assigned as a partition mode or assigned as ...

  • Page 182

    166 ibm system x3850 x6 and x3950 x6 planning and implementation guide partition actions: – remove partition: creates an unassigned node. An unassigned node can at any point be re-deployed back into an assigned node. A node needs to be assigned to function. – activate stand-alone mode: turns the ser...

  • Page 183

    Chapter 5. Preparing the hardware 167 figure 5-43 displays the scalable complex window pane with one x3950 x6 server, which contains 2 cpus and 32gb of ram in each partition. Figure 5-43 imm2 web interface scalable complex window 3. Before separating the x3950 x6 into two stand-alone servers, ensure...

  • Page 184

    168 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5. After the server has been turned off, select partition actions activate stand-alone mode as seen in figure 5-46. Figure 5-46 partition options in the scalable complex window via the imm2 6. Click activate stand-alone mode on t...

  • Page 185: 5.7  Updating Firmware

    Chapter 5. Preparing the hardware 169 you can restore partition mode on the servers by highlighting the server with the check box and selecting partition actions restore partition mode as seen in figure 5-50. By restoring it to partition mode, you are restoring the server back to a full x3950 x6 ser...

  • Page 186

    170 ibm system x3850 x6 and x3950 x6 planning and implementation guide you can also perform firmware updates using ibm systems director or upward integration module (uim) with a hypervisor. Ibm systems director under certain operating systems, ibm systems director allows you to acquire, install, and...

  • Page 187

    Chapter 5. Preparing the hardware 171 updatexpress system pack installer (uxspi) the ibm updatexpress is a tool that allows the ibm system x firmware and drivers to be updated via the os. The updatexpress system pack installer (uxspi) gives you the ability to update the firmware and device drivers o...

  • Page 188

    172 ibm system x3850 x6 and x3950 x6 planning and implementation guide 4. Accept the default update the local machine, as shown in figure 5-53. Click next. Figure 5-53 selecting the update task 5. Accept the default check the ibm web site, as shown in figure 5-54. Click next. Figure 5-54 selecting t...

  • Page 189

    Chapter 5. Preparing the hardware 173 8. Enter the settings for an http proxy server, if necessary, or leave the check box unchecked, as shown in figure 5-57. Click next. Figure 5-57 http proxy settings 9. A message displays showing that the uxspi acquired the possible updates for the machine, as sh...

  • Page 190

    174 ibm system x3850 x6 and x3950 x6 planning and implementation guide perform these steps to create an update media: 1. Create a folder named bomc . 2. Download the latest version of bomc from the following website and save it in the bomc folder: http://ibm.Com/support/entry/portal/docdisplay?Lndoc...

  • Page 191

    Chapter 5. Preparing the hardware 175 4. The welcome window opens, as shown in figure 5-61. Click next. Figure 5-61 welcome page 4. Check updates and click next, as shown in figure 5-62. Figure 5-62 selected update task 5. Select latest available individual updates and click next, as shown in figure...

  • Page 192

    176 ibm system x3850 x6 and x3950 x6 planning and implementation guide 6. Enter the settings for an http proxy server, if necessary, or check do not use proxy, as shown in figure 5-64. Click next. Figure 5-64 http proxy settings 7. Select one or more machine types that are contained on the bootable ...

  • Page 193

    Chapter 5. Preparing the hardware 177 10.Select do not use unattended mode (figure 5-67) and click next. Figure 5-67 unattended mode panel 11.Review the selections and confirm that they are correct. You can click save to save this configuration information to a file. Click next. 12.Bomc acquires the...

  • Page 194

    178 ibm system x3850 x6 and x3950 x6 planning and implementation guide ibm fastsetup fastsetup allows users to create mass deployments of firmware updates and includes features to make the setup process as simple and as fast as possible, including automation templates made easy for mass deployments ...

  • Page 195

    Chapter 5. Preparing the hardware 179 5. When the installation is complete, you will receive a success message as seen in figure 5-71. Click done. Figure 5-71 successful installation 6. Launch the fastsetup utility from the location you installed it in by clicking ibm fastsetup.Exe. 7. Select your l...

  • Page 196

    180 ibm system x3850 x6 and x3950 x6 planning and implementation guide integrated management module (imm) you can perform certain firmware updates using the imm. See 7.2, “integrated management module ii (imm2)” on page 201 for information about how to connect to the imm. The imm2 will allow you to ...

  • Page 197: 5.8  Troubleshooting

    Chapter 5. Preparing the hardware 181 5.8 troubleshooting this section provides an overview of tools available to assist with problem resolution for the x6 servers in any given configuration. It also provides considerations for extended outages. Use the following tools when troubleshooting problems ...

  • Page 198

    182 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5.8.2 lcd system information panel you can use the lcd information panel to diagnose system errors quickly. The lcd system information display panel is attached to the storage book on the front of the server. The lcd system infor...

  • Page 199

    Chapter 5. Preparing the hardware 183 5.8.4 post event log this log contains the three most recent error codes and messages that were generated during post. You can view the post event log through the uefi by pressing f1 at system start-up and selecting system event logs post event viewer. 5.8.5 ibm...

  • Page 200

    184 ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 201

    © copyright ibm corp. 2014. All rights reserved. 185 chapter 6. Operating system installation this chapter provides an overview of the options that you have when installing an operating system (os) on the x6 family servers. This chapter contains the following topics: 6.1, “installing without a local...

  • Page 202

    186 ibm system x3850 x6 and x3950 x6 planning and implementation guide 6.1 installing without a local optical drive the x6 family servers do not have a local optical drive. To install an os, you can use any of the following methods: 6.1.1, “imm” on page 186 6.1.2, “local usb port” on page 188 6.1.3,...

  • Page 203

    Chapter 6. Operating system installation 187 4. The java application window should open, as shown in figure 6-2. Figure 6-2 video viewer window 5. To mount an existing image to the remote server as virtual media, you should open virtual media windows first as shown in figure 6-3. Figure 6-3 virtual ...

  • Page 204

    188 ibm system x3850 x6 and x3950 x6 planning and implementation guide 7. After adding a desired image, select the check box next to the drive that you want to map and click mount selected in figure 6-5. Figure 6-5 selected image mounting 8. The image drive is now accessible by the system. 6.1.2 loc...

  • Page 205: 6.2  Ibm Serverguide

    Chapter 6. Operating system installation 189 6.2 ibm serverguide ibm serverguide is an installation assistant for windows installations that simplifies the process of installing and configuring ibm system x and bladecenter servers. The wizard guides you through the setup, configuration, and operatin...

  • Page 206

    190 ibm system x3850 x6 and x3950 x6 planning and implementation guide 9. Next, select the operating system that you want to install and click next, as shown in figure 6-7. Figure 6-7 selecting the operating system 10.Enter the current date and time as shown in figure 6-8 and click next. Figure 6-8 ...

  • Page 207

    Chapter 6. Operating system installation 191 11.Create a raid configuration. Select a raid configuration and click next, as shown in figure 6-9. Figure 6-9 raid selection 12.A confirmation window opens indicating that raid configuration is complete, similar to figure 6-10. Click next. Figure 6-10 ra...

  • Page 208

    192 ibm system x3850 x6 and x3950 x6 planning and implementation guide 13.You must now create and format a partition. Choose your selection and click next to start the process, as shown in figure 6-11. Figure 6-11 selection for format and partition 14.When the process has completed, click next. You ...

  • Page 209

    Chapter 6. Operating system installation 193 15.Review the configuration, as shown in figure 6-13. Click next. Figure 6-13 summary report 16.Serverguide copies the necessary files to the disk in preparation for the operating system installation as shown in figure 6-14. Figure 6-14 serverguide copyin...

  • Page 210

    194 ibm system x3850 x6 and x3950 x6 planning and implementation guide 18.Mount your windows iso image via the imm and click next, as shown in figure 6-15. Serverguide searches for the disk. Figure 6-15 prompt to insert the os dvd 19.The windows setup installation procedure starts. Follow the micros...

  • Page 211

    Chapter 6. Operating system installation 195 updatexpress system packs installation integrated with scripted network operating system (nos) deployment ibm director agent installation, integrated with scripted nos deployment the serverguide scripting toolkit, windows edition supports the following ve...

  • Page 212

    196 ibm system x3850 x6 and x3950 x6 planning and implementation guide vmware esxi supports booting from the unified extensible firmware interface (uefi). To ensure that you can boot esxi successfully, you must change the boot order. The first boot entry must embedded hypervisor. Follow these steps:...

  • Page 213: 6.5  Booting From San

    Chapter 6. Operating system installation 197 6.5 booting from san boot from san (or san boot) is a technique used when the node in the chassis has no local disk drives. It uses an external storage system lun to boot the operating system. Both the operating system and data are on the san. This techni...

  • Page 214

    198 ibm system x3850 x6 and x3950 x6 planning and implementation guide you can also check the documentation for the operating system that is used for boot from san support and requirements and storage vendors. See the following sources for more san boot-related information: red hat enterprise linux ...

  • Page 215: Management

    © copyright ibm corp. 2014. All rights reserved. 199 chapter 7. Management in the information technology sector, server systems management has received greater focus over recent years. The ability to maintain and manage systems efficiently is essential to business it operations. In this chapter, we ...

  • Page 216: 7.1  Introduction

    200 ibm system x3850 x6 and x3950 x6 planning and implementation guide 7.1 introduction ibm provides a number of tools to successfully deploy, manage, and maintain the x3850 x6 and x3950 x6 servers. The collective name for these tools is ibm toolscenter. You can access these tools at the following l...

  • Page 217

    Chapter 7. Management 201 7.2 integrated management module ii (imm2) the integrated management module ii (imm2) offers these overall features and functions: provides diagnostics, virtual presence, and remote control to manage, monitor, troubleshoot, and repair from anywhere. Securely manages servers...

  • Page 218

    202 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 7-1 shows the location of the imm2 and ml2 ports on the rear of the server. Figure 7-1 ports over which the imm2 can be managed out-of-band on x6. By default, the ml2 slot is not shared for use with the imm2. You must enab...

  • Page 219

    Chapter 7. Management 203 5. Select network configuration from the integrated management module. This menu allows you to configure the network settings for the imm2. Also, you can configure the imm2 to share the use of ml2 or out-of-band management from this menu, as shown in figure 7-2. Figure 7-2 ...

  • Page 220

    204 ibm system x3850 x6 and x3950 x6 planning and implementation guide after the imm2 is available on the network, you can log in into the imm2 web interface by typing its ip address in a supported web browser as shown in figure 7-3. Figure 7-3 imm2 login web page enter the default user name, which ...

  • Page 221: 7.3  Remote Control

    Chapter 7. Management 205 7.3 remote control the ability to control the x6 systems remotely is provided by the imm2. You can perform the following common tasks with the remote control function: control the power of the systems mount remote media, which includes cd/dvd-roms, supported iso and firmwar...

  • Page 222

    206 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3. To protect sensitive disk and kvm data during your session, click the encrypt disk and kvm data during transmission check box before starting remote control. For complete security, use remote control in conjunction with ssl. Y...

  • Page 223: 7.4  Ibm Systems Director

    Chapter 7. Management 207 7.4 ibm systems director ibm systems director is a platform manager that offers the following benefits: enables the management of x6 physical servers and virtual servers that are running on the x6 platform. Helps to reduce the complexity and costs of managing x6 platforms. ...

  • Page 224

    208 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2. After you log in to the console, the main page appears, figure 7-6. Figure 7-6 ibm systems director pane 3. Navigate to inventory system discovery in the left navigation pane as seen in figure 7-7. Figure 7-7 ibm systems direc...

  • Page 225

    Chapter 7. Management 209 5. Enter the ip address of the imm2 in the space that is provided under the ip address as seen in figure 7-8. Figure 7-8 ibm systems director discovery pane 6. An imm2 is considered a server object in ibm systems director. To specify the imm2 as a server object, click the s...

  • Page 226

    210 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 7-10 renaming an imm2 10.You will need to authenticate to the imm2 before you can perform any tasks. To authenticate to the imm2, right-click no access under the access column and click request access as shown in figure 7-...

  • Page 227

    Chapter 7. Management 211 13.In the view and collect inventory pane, click browse and select your server. You will need to check the tick box and click add as seen in figure 7-14, then click ok, figure 7-14 select server for inventory collection 14.Your server will be displayed in the target systems...

  • Page 228

    212 ibm system x3850 x6 and x3950 x6 planning and implementation guide figure 7-16 learn more about ibm systems director 7.4.2 service and support manager service and support manager is a no-charge automatically installed plug-in to ibm systems director. When enabled, it automatically captures hardw...

  • Page 229

    Chapter 7. Management 213 collect inventory for the systems before you start the installation. This task is not mandatory, but it is advised. Configuring service and support manager the following section discusses the steps you need to take to set up electronic service agent (esa) from the ibm syste...

  • Page 230

    214 ibm system x3850 x6 and x3950 x6 planning and implementation guide 3. Enter your company contact information in the provided fields. The more information that you provide, the easier it will be for ibm support to assist you. Pay particular attention to ensuring that the country or region field i...

  • Page 231

    Chapter 7. Management 215 6. Provide the authorized ids of the personnel who need access to the service information that is transmitted to ibm. This information is not a requirement to activate service and support manager. Click next. If you have not already created the ids, you can do so by clickin...

  • Page 232

    216 ibm system x3850 x6 and x3950 x6 planning and implementation guide 2. Right-click a system and select properties as seen in figure 7-22. Figure 7-22 accessing target system properties 3. Click location on the properties pane as seen in figure 7-23. Figure 7-23 changing the location of a server.

  • Page 233

    Chapter 7. Management 217 4. Click edit to provide the location details. Click ok when finished as seen in figure 7-24. Figure 7-24 changing the location of a target system 7.5 upward integration modules the ibm unique upward integration modules (uims) for virtualization provide hardware visibility ...

  • Page 234

    218 ibm system x3850 x6 and x3950 x6 planning and implementation guide uims features unique to ibm provide ease of management in the following ways: concurrent firmware update: uim provides the ability to concurrently update all system software in a virtualized environment with a single command with...

  • Page 235

    Chapter 7. Management 219 2. Any virtual machines (vms) running on the first server are evacuated and moved off to another server (figure 7-26). Figure 7-26 uim evacuates first servers vms for updating 3. The server is taken off line and updates are applied. 4. After the updates are applied, the ser...

  • Page 236

    220 ibm system x3850 x6 and x3950 x6 planning and implementation guide reliability, availability, serviceability (ras) upward integration module (uim) now has upward integration with ras events. Uim undergoes the following procedure when a ras event is detected on a server: 1. A ras event is detecte...

  • Page 237

    Chapter 7. Management 221 7.5.1 machine check architecture (mca) error recovery hardware errors can now be presented to the operating system (os) to allow for recovery. This allows what used to be fatal errors to now be recoverable. The following operating systems were due to be supported at the tim...

  • Page 238

    222 ibm system x3850 x6 and x3950 x6 planning and implementation guide you can perform the following tasks by using the asu: modify the uefi cmos settings without the need to restart the system and access the f1 menu. Modify the imm2 setup settings. Modify a limited set of vpd settings. Modify the i...

  • Page 239

    Chapter 7. Management 223 3. Unpack the utility: – for windows, double-click ibm_utl_asu_asutxxx_windows_yyyy.Exe, which decompresses the following files into the asu folder. – for linux, open a terminal session and run the following command from the asu directory: tar -zxf ibm_utl_asu_asut xxx _lin...

  • Page 240

    224 ibm system x3850 x6 and x3950 x6 planning and implementation guide show all uefi settings on the command line, enter asu show uefi . See example 7-2. Example 7-2 output of the asu show uefi command uefi.Operatingmode=custom mode uefi.Quietboot=enable uefi.Turbomodeenable=enable uefi.Turboboost=p...

  • Page 241

    Chapter 7. Management 225 you can use msm to manage local or remote raid controllers and configure msm for remote alert notifications. Also, a command-line interface is available. To download the latest megaraid storage manager software and obtain the installation and user’s guide, select the server...

  • Page 242

    226 ibm system x3850 x6 and x3950 x6 planning and implementation guide offline: a drive that is part of a virtual drive but which has invalid data as far as the raid configuration is concerned. 7.7.3 virtual drive states a virtual drive is a partition in a drive group that is made up of contiguous d...

  • Page 243

    Chapter 7. Management 227 example 7-6 output from command megacli -cfgldadd -r1[252:1,252:2] -a0 adapter 0: created vd 1 adapter 0: configured the adapter!! Exit code: 0x00 3. The virtual drive is successfully created. Additional command examples the following command examples use the megacli comman...

  • Page 244: 7.9  Serial Over Lan

    228 ibm system x3850 x6 and x3950 x6 planning and implementation guide system failure logs part feature codes, part number, part serial number, and part locations software inventory, including operating system applications program temporary fixes (ptfs), including the maintenance levels and configur...

  • Page 245

    Chapter 7. Management 229 in this section, we describe these topics: 7.9.1, “enabling sol in uefi” on page 229 7.9.2, “enabling sol in the operating system” on page 229 7.9.3, “how to start a sol connection” on page 233 7.9.1 enabling sol in uefi to enable sol from the uefi interface at boot time, p...

  • Page 246

    230 ibm system x3850 x6 and x3950 x6 planning and implementation guide windows server 2008 to enable the microsoft emergency messaging service (ems) and the special administration console (sac), use the following procedure. You must have administrator privileges to perform these steps. 1. Start a co...

  • Page 247

    Chapter 7. Management 231 imm setting to change the cli mode for the com port for use with ems, use the following procedure: 1. Log in to the web interface of the imm2. 2. Navigate to imm management imm properties, as seen in figure 7-32. Figure 7-32 imm2 properties pane 3. From the imm2 properties ...

  • Page 248

    232 ibm system x3850 x6 and x3950 x6 planning and implementation guide 5. Click apply to save the changes. For more information about microsoft emergency messaging service and the special administration console, see the following documents: boot parameters to enable ems redirection http://msdn.Micro...

  • Page 249

    Chapter 7. Management 233 /etc/inittab add the parameter that is highlighted in bold at the end of the /etc/inittab file, as shown in example 7-11. Example 7-11 the /etc/inittab file id:3:initdefault: co:2345:respawn:/sbin/agetty ttys1 115200 vt100-nav [root@localhost etc]# 7.9.3 how to start a sol ...

  • Page 250

    234 ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 251: Abbreviations and Acronyms

    © copyright ibm corp. 2014. All rights reserved. 235 ac alternating current acid atomicity, consistency, isolation, and durability acpi advanced control and power interface aes advanced encryption standard aes-ni advanced encryption standard new instructions amm advanced management module ansi ameri...

  • Page 252

    236 ibm system x3850 x6 and x3950 x6 planning and implementation guide kvm keyboard video mouse kw kilowatts lan local area network lc line cord lcd liquid crystal display ldap lightweight directory access protocol led light emitting diode ler live error recovery lp low profile lrdimm load-reduced d...

  • Page 253

    Abbreviations and acronyms 237 uxspi updatexpress system packs installer vac volts ac vd virtual drive vfa virtual fabric adapter vga video graphics array vlan virtual lan vm virtual machine vpd vital product data vpi virtual protocol interconnect vt virtualization technology ww world wide.

  • Page 254

    238 ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 255: Related Publications

    © copyright ibm corp. 2014. All rights reserved. 239 related publications the publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this book. Ibm redbooks publications the following ibm redbooks publications provide addition...

  • Page 256: Other Publications

    240 ibm system x3850 x6 and x3950 x6 planning and implementation guide books and papers workload optimization with ibm x6 servers, redp-5058 http://www.Redbooks.Ibm.Com/abstracts/redp5058.Html?Open ibm x6 servers: technical overview, redp-5059 http://www.Redbooks.Ibm.Com/abstracts/redp5059.Html?Open...

  • Page 257: Help From Ibm

    Related publications 241 http://ibm.Com/support/entry/portal/overview/hardware/systems/system_x/system_x 3950_x6 ibm system storage interoperation center http://www.Ibm.Com/systems/support/storage/ssic help from ibm ibm support and downloads ibm.Com/support ibm global services ibm.Com/services.

  • Page 258

    242 ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 259

    (0.5” spine) 0. 475”0.87 3 ” 250 459 pages ibm system x3850 x6 and x3950 x6 pl anning and implementation guide ibm system x3850 x6 and x3950 x6 planning and implementation guide ibm system x3850 x6 and x3950 x6 planning and implementation guide ibm system x3850 x6 and x3950 x6 planning and implement...

  • Page 260

    Ibm system x3850 x6 and x3950 x6 planning and implementation guide ibm system x3850 x6 and x3950 x6 planning and implementation guide.

  • Page 262: Ibm System X3850 X6 and

    ® sg24-8208-00 isbn 0738439746 international technical support organization building technical information based on practical experience ibm redbooks are developed by the ibm international technical support organization. Experts from ibm, customers and partners from around the world create timely te...