1. DL manuals
  2. IBM
  3. Server
  4. z13s
  5. Technical Manual

IBM z13s Technical Manual - Node 0

Summary of z13s

  • Page 1

    Red books in partnership with ibm academy of technology front cover ibm z13s technical guide octavian lascu barbara sannerud cecilia a. De leon edzard hoogerbrug ewerson palacio franco pinto jin j. Yang john p. Troy martin soellig.

  • Page 3

    International technical support organization ibm z13s technical guide june 2016 sg24-8294-00

  • Page 4

    © copyright international business machines corporation 2016. All rights reserved. Note to u.S. Government users restricted rights -- use, duplication or disclosure restricted by gsa adp schedule contract with ibm corp. First edition (june 2016) this edition applies to ibm z13s servers. Note: before...

  • Page 5: Contents

    © copyright ibm corp. 2016. All rights reserved. Iii contents figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Xv notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ....

  • Page 6

    Iv ibm z13s technical guide 2.2.4 cpc drawer power . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.3 single chip modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 2.3.1 processor units...

  • Page 7

    Contents v 3.4.12 instruction fetching, decoding, and grouping . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 3.4.13 extended translation facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 3.4.14 instruction set extensions . . . . . . . . . . . . . . ...

  • Page 8

    Vi ibm z13s technical guide 4.6.6 fanout considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 4.7 i/o features (cards) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 4.7.1 i/o feature ca...

  • Page 9

    Contents vii 6.6.4 tke workstation with licensed internal code 8.1. . . . . . . . . . . . . . . . . . . . . . . . 223 6.6.5 tke hardware support and migration information. . . . . . . . . . . . . . . . . . . . . . . . 224 6.7 cryptographic functions comparison . . . . . . . . . . . . . . . . . . . ....

  • Page 10

    Viii ibm z13s technical guide 7.3.38 ficon link incident reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 7.3.39 fcp provides increased performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 7.3.40 n_port id virtualization. . . . . . ....

  • Page 11

    Contents ix 7.7 ibm z advanced workload analysis reporter (zaware) . . . . . . . . . . . . . . . . . . . . . . . 291 7.7.1 z appliance container infrastructure mode lpar . . . . . . . . . . . . . . . . . . . . . . . 292 7.8 coupling facility and cfcc considerations . . . . . . . . . . . . . . . . . ...

  • Page 12

    X ibm z13s technical guide 8.5.5 termination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 8.5.6 ibm z/os capacity provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 8.6 capacity for planned event...

  • Page 13

    Contents xi 10.4.3 system activity display and monitors dashboard. . . . . . . . . . . . . . . . . . . . . . . 382 10.4.4 ibm systems director active energy manager . . . . . . . . . . . . . . . . . . . . . . . . . 383 10.4.5 unified resource manager: energy management . . . . . . . . . . . . . . . ...

  • Page 14

    Xii ibm z13s technical guide 12.3.2 instruction complexity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 12.3.3 memory hierarchy and memory nest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440 12.4 relative nest intensity . . . . . ....

  • Page 15

    Contents xiii d.3.4 virtual pci function (vpci adapter). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492 d.3.5 planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 495 d.3.6 hardware configuration definitions . . . . . . ...

  • Page 16

    Xiv ibm z13s technical guide i.3.1 planned disk outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 i.3.2 unplanned disk outage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548 i.3.3 disaster recovery . . ....

  • Page 17: Figures

    © copyright ibm corp. 2016. All rights reserved. Xv figures 2-1 z13s frame: rear and front view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 2-2 z13s (one cpc drawer) i/o drawer configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 2-3 z1...

  • Page 18

    Xvi ibm z13s technical guide 4-5 pcie i/o drawer with 32 pcie slots and four i/o domains . . . . . . . . . . . . . . . . . . . . . 146 4-6 infrastructure for pcie and infiniband coupling links . . . . . . . . . . . . . . . . . . . . . . . . . 148 4-7 om3 50/125 µm multimode fiber cable with mpo conn...

  • Page 19

    Figures xvii 11-5 configure backup ftp server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 11-6 backup critical console data destinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 11-7 backup critical data destinations of ses . . ....

  • Page 20

    Xviii ibm z13s technical guide d-8 reduced latency and improved wall clock time with smc-r . . . . . . . . . . . . . . . . . . . 485 d-9 sysplex distributor before roce . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 487 d-10 sysplex distributor after roce. . . . . . ...

  • Page 21: Notices

    © copyright ibm corp. 2016. All rights reserved. Xix notices this information was developed for products and services offered in the us. This material might be available from ibm in other languages. However, you may be required to own a copy of the product or product version in that language in orde...

  • Page 22: Trademarks

    Xx ibm z13s technical guide trademarks ibm, the ibm logo, and ibm.Com are trademarks or registered trademarks of international business machines corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of ibm or other companies. A current list of i...

  • Page 23: Find and Read Thousands of

    Ibm redbooks promotions find and read thousands of ibm red books publications search, bookmark, save and organize favorites get personalized notifications of new content link to the latest redbooks blogs and videos download now get the latest version of the red books mobile app ios android place a sp...

  • Page 24

    This page intentionally left blank.

  • Page 25: Preface

    © copyright ibm corp. 2016. All rights reserved. Xxiii preface digital business has been driving the transformation of underlying information technology (it) infrastructure to be more efficient, secure, adaptive, and integrated. It must be able to handle the explosive growth of mobile clients and em...

  • Page 26

    Xxiv ibm z13s technical guide cecilia a. De leon is a certified it specialist in the philippines. She has 15 years of experience in the z systems field. She has worked at ibm for 7 years. She holds a degree in computer engineering from mapua institute of technology. Her areas of expertise include z ...

  • Page 27: Comments Welcome

    Preface xxv patty driever diana henderson anuja deedwaniya lisa schloemer christine smith martin ziskind romney white jerry stevens anthony sofia ibm poughkeepsie leslie geer iii ibm endicott monika zimmerman carl mayer angel nunez mencias ibm germany parwez hamid ibm poughkeepsie now you can become...

  • Page 28

    Xxvi ibm z13s technical guide mail your comments to: ibm corporation, international technical support organization dept. Hytd mail station p099 2455 south road poughkeepsie, ny 12601-5400 stay connected to ibm redbooks find us on facebook: http://www.Facebook.Com/ibmredbooks follow us on twitter: ht...

  • Page 29: Introducing Ibm Z13S Servers

    © copyright ibm corp. 2016. All rights reserved. 1 chapter 1. Introducing ibm z13s servers digital business has been driving the transformation of it to be more efficient, secure, adaptive, and intelligent. Today’s technology foundation must be capable of increasing the value of data, drawing on the...

  • Page 30

    2 ibm z13s technical guide 1.1 overview of ibm z13s servers the ibm z13s server, like its predecessors, is designed from the chip level up for intense data serving and transaction processing, the core of business. Ibm z13s servers are designed to provide unmatched support for data processing by prov...

  • Page 31

    Chapter 1. Introducing ibm z13s servers 3 open systems adapters (osas) across z/vm systems, z/vm 6.3 delivers enhanced availability and reduced cost of ownership in network environments. Ibm z13s brings a new approach for enterprise-grade linux with offerings and capabilities for availability, virtu...

  • Page 32

    4 ibm z13s technical guide also includes innovative smt capability, and provides 139 simd vector instruction subset for better performance. Depending on the model, z13s servers can support from 64 gb to a maximum of 4 tb of usable memory, with up to 2 tb of usable memory per cpc drawer. In addition,...

  • Page 33

    Chapter 1. Introducing ibm z13s servers 5 workload runs on a ziip processor, thus reducing the complexity of the ibm z/architecture®. Zaaps are no longer supported beginning with the z13 and z13s servers. Workload variability consult the lspr when considering performance on z13s servers. Individual ...

  • Page 34

    6 ibm z13s technical guide for i/o constraint relief, the ibm z13s server has improved scalability with support for three logical channel subsystems (lcsss), three subchannel sets (to support more devices per logical channel subsystem), and up to 32k devices per ibm ficon® channel up from 24k channe...

  • Page 35

    Chapter 1. Introducing ibm z13s servers 7 processing for z13s and z13 ga2 servers. It is intended to provide performance improvements and better availability for critical business workloads that cannot afford any impact to service levels. Flash express is easy to configure, and provides rapid time t...

  • Page 36

    8 ibm z13s technical guide that use tcp/ip communications can benefit immediately without requiring any application software or ip topology changes. With its lightweight design, smc-d is designed to improve throughput, latency, and cpu consumption over other alternatives without sacrificing quality ...

  • Page 37

    Chapter 1. Introducing ibm z13s servers 9 capacity that is consumed by an individual lpar or by a group of lpars. This limit is still valid when a pu defined as a cp or an ifl is shared across a set of lpars. This feature is designed to provide and enforce a physical capacity limit as an absolute (v...

  • Page 38

    10 ibm z13s technical guide network setup, and appliance configuration. The design of zaci allows a simpler product (function) deployment model. Several exploiters are planned for delivery through zaci. For instance, z/vse network appliance provides network access for tcp/ip socket applications runn...

  • Page 39

    Chapter 1. Introducing ibm z13s servers 11 beginning with z13s servers, ibm zaware runs in a zaci mode logical partition. Either cps or ifls can be configured to the ibm zaware partition. This special partition is defined for the exclusive use of the ibm z systems advanced workload analysis reporter...

  • Page 40

    12 ibm z13s technical guide special-purpose features: – cryptography – flash express – zedc express reliability, availability, and serviceability 1.3.1 models z13s servers have a machine type of 2965. Two models are offered: n10 and n20. The model name indicates the maximum number of pus available f...

  • Page 41

    Chapter 1. Introducing ibm z13s servers 13 the z13s cpc drawer system design provides the capability to increase the capacity of the system. You can add capacity by activating more cps, ifls, icfs, or ziips on an existing cpc drawer. The second cpc drawer can be used to provide more memory, or one o...

  • Page 42

    14 ibm z13s technical guide 1.3.4 cpc drawer up to two cpc drawers (minimum one) can be installed in the z13s frame. Each cpc drawer houses the scms, memory, and fanouts. The cpc drawer supports up to 8 pcie fanouts and 4 ifb fanouts for i/o and coupling connectivity. In the n20 two cpc drawer model...

  • Page 43

    Chapter 1. Introducing ibm z13s servers 15 in the unlikely case of a permanent core failure, each core can be individually replaced by one of the available spares. Core sparing is transparent to the operating system and applications. The n20 has two spares, but the n10 does not offer dedicated spare...

  • Page 44

    16 ibm z13s technical guide one memory port for each pu chip. Increased bandwidth between memory and i/o asymmetrical memory size and dram technology across cpc drawers large memory pages (1 mb and 2 gb) key storage storage protection key array that is kept in physical memory storage protection (mem...

  • Page 45

    Chapter 1. Introducing ibm z13s servers 17 that of zec12. This infrastructure is designed to reduce processor usage and latency, and provide increased throughput. Z13s servers offer two i/o infrastructure elements for driving the i/o features: pcie i/o drawers for pcie features, and up to one i/o dr...

  • Page 46

    18 ibm z13s technical guide also, when carried forward on an upgrade, the z13s servers support one i/o drawer on which the ficon express8 sx and lx (10 km) feature can be installed. In addition, infiniband coupling links hca3-o and hca3-o lr, which attach directly to the cpc drawers, are supported. ...

  • Page 47

    Chapter 1. Introducing ibm z13s servers 19 osm and osx channel path identifier types the z13s provides osa-express5s, osa-express4s, and channel-path identifier (chpid) types osa-express for unified resource manager (osm) and osa-express for zbx (osx) connections: osa-express for unified resource ma...

  • Page 48

    20 ibm z13s technical guide for communications between lpars in the same z13s server, hipersockets eliminates the need to use i/o subsystem features and to traverse an external network. Connection to hipersockets offers significant value in server consolidation by connecting many virtual servers. It...

  • Page 49

    Chapter 1. Introducing ibm z13s servers 21 connectivity in support of parallel sysplex environments is provided on z13s servers by the following features: integrated coupling adapter short reach - ica sr (pcie coupling links): the ica sr pcie fanout has two 8 gbps ports that use short reach optics (...

  • Page 50

    22 ibm z13s technical guide to support an upgrade from one cfcc level to the next, different levels of cfcc can coexist in the same sysplex while the coupling facility lpars are running on different servers. Cf lpars that run on the same server share the cfcc level. A cf running on a z13s server (cf...

  • Page 51

    Chapter 1. Introducing ibm z13s servers 23 enhanced console assisted recovery console assisted recovery (car) support is designed to help a backup time server (bts) determine whether the primary time server is still up and running if coupling traffic ceases. The car process is initiated by the bts i...

  • Page 52

    24 ibm z13s technical guide configurable crypto express5s feature crypto express5s represents the newest generation of cryptographic features. It is designed to complement the cryptographic capabilities of the cpacf. It is an optional feature of the z13 and z13s server generation. The crypto express...

  • Page 53

    Chapter 1. Introducing ibm z13s servers 25 operational and master keys. Tke also provides secure management of host cryptographic module and host capabilities. Support for an optional smart card reader that is attached to the tke workstation allows the use of smart cards that contain an embedded mic...

  • Page 54

    26 ibm z13s technical guide suse) can use flash express for temporary storage. Recent enhancements of flash express include 2 gb page support and dynamic reconfiguration for flash express. For more information, see appendix h, “flash express” on page 529. Zedc express zedc express , an optional feat...

  • Page 55

    Chapter 1. Introducing ibm z13s servers 27 ficon (better recovery on fiber) n+1 sch and power supplies n+1 ses ashrae a3 support for more information, see chapter 9, “reliability, availability, and serviceability” on page 355. 1.4 hardware management consoles and support elements the hmcs and ses ar...

  • Page 56

    28 ibm z13s technical guide up to 112 11 blades. Inmn tor switches. On the zbx model 004, the new local zbx support elements directly connect to the inmn within the zbx for management purposes. Because zbx model 004 is an independent node, there is no inmn connection to any z systems cpc. Iedn tor s...

  • Page 57

    Chapter 1. Introducing ibm z13s servers 29 when the datapower xi50z is installed in the zbx, the unified resource manager provides integrated management for the appliance. This configuration simplifies control and operations, including change management, energy monitoring, problem detection, problem...

  • Page 58

    30 ibm z13s technical guide dpm provides simplified, consumable, and enhanced partition lifecycle and dynamic i/o management capabilities by using the hardware management console and is designed to perform these tasks: create and provision an environment: – create new partitions – assignment of proc...

  • Page 59

    Chapter 1. Introducing ibm z13s servers 31 linux on z systems distributions: – suse linux enterprise server (sles): sles 12 and sles 11. – red hat enterprise linux (rhel): rhel 7 and rhel 6. – customers should monitor for new distribution releases supported for recommended and new linux on z systems...

  • Page 60

    32 ibm z13s technical guide z/vm support z/vm support for the z13 servers includes but is not limited to these enhancements: z/vm 6.2 and 6.3 provide these enhancements: – guest support for crypto express5s, and support for 85 crypto express domains. – absolute capping of an lpar group, enabling eac...

  • Page 61

    Chapter 1. Introducing ibm z13s servers 33 also available for z/os v1.13 and z/os v2.1 with ptfs, can help you identify configuration errors that can result in data integrity exposures. Z/os v2.2, and z/os v2.1 with ptfs support the new lpar absolute group capping function. Z/os v2.2 communications ...

  • Page 62

    34 ibm z13s technical guide to obtain the best performance, you must specify an architecture level of 11 by using the –qarch=arch11 option for the xl c/c++ for linux on z systems compiler or the arch(11) option for the other compilers. This option grants the compiler permission to use machine instru...

  • Page 63: Central Processor Complex

    © copyright ibm corp. 2016. All rights reserved. 35 chapter 2. Central processor complex hardware components this chapter introduces the ibm z13s hardware components, significant features and functions, and their characteristics and options. The main objective of this chapter is to explain the z13s ...

  • Page 64: 2.1  Frame and Drawers

    36 ibm z13s technical guide 2.1 frame and drawers the z systems frames are enclosures that are built to electronic industries alliance (eia) standards. The z13s central processor complex (cpc) has one 42u eia frame, which is shown in figure 2-1. The frame has positions for one or two cpc drawers, an...

  • Page 65

    Chapter 2. Central processor complex hardware components 37 two bulk power assemblies (bpas) that house the power components, with one in the front and the other in the rear of the system. Each bpa is equipped with up to three bulk power regulators (bprs), a controller, and two distributor cards. Th...

  • Page 66

    38 ibm z13s technical guide in figure 2-2, the various i/o drawer configurations are displayed when one cpc drawer is installed for both model n10 and n20. The view is from the rear of the a frame. Pcie drawers are installed from the top down an i/o drawer (legacy) is only present during a mes carry...

  • Page 67

    Chapter 2. Central processor complex hardware components 39 pcie i/o infrastructure the pcie i/o infrastructure uses the pcie fanout to connect to the pcie i/o drawer, which can contain the following features: ficon express16s, a two port card (long wavelength (lx) or short wavelength (sx)) and two ...

  • Page 68: Node 0

    40 ibm z13s technical guide z13s servers can have up to two cpc drawers installed (the minimum is one cpc drawer). The contents of the cpc drawer and its components are model dependent. The model n10 cpc drawer that is shown in figure 2-4 has a single node, half the structure of the model n20 cpc dr...

  • Page 69: Node 0

    Chapter 2. Central processor complex hardware components 41 figure 2-5 model n20 components (top view) figure 2-6 shows the front view of the cpc drawer with fanout slots and fsp slots for both a model n20 and a model n10. Figure 2-6 cpc drawer (front view) 20 memory dimm slots infiniband fanout slo...

  • Page 70

    42 ibm z13s technical guide figure 2-7 shows the memory dimm to pu scm relationship and the various buses on the nodes. Memory is connected to the scms through memory control units (mcus). There is one mcu per pu scm that provide the interface to controller on memory dimm. A memory controller drives...

  • Page 71

    Chapter 2. Central processor complex hardware components 43 2.2.1 cpc drawer interconnect topology figure 2-8 shows the point-to-point topology for cpc drawers and node communication through the smp cables. Each cpc drawer communicates directly to all other processor drawers in the cpc through two w...

  • Page 72

    44 ibm z13s technical guide if stp uses an ntp server without pps, a time accuracy of 100 ms to the ets is maintained. Although not part of the cpc drawer design, the oscs cards are located beside the drawers, and are connected to the same backplane to which the drawers are connected. Both drawers c...

  • Page 73

    Chapter 2. Central processor complex hardware components 45 figure 2-10 depicts a logical diagram of the system control design. Figure 2-10 conceptual overview of system control elements a typical fsp operation is to control a power supply. An se sends a command to the fsp to start the power supply....

  • Page 74: 2.3  Single Chip Modules

    46 ibm z13s technical guide 2.2.4 cpc drawer power each cpc drawer gets its power from two distributed converter assemblies (dcas) in the cpc drawer (see figure 2-11). The dcas provide the power for the cpc drawer. Loss of one dca leaves enough power to satisfy cpc drawer power requirements. The dca...

  • Page 75

    Chapter 2. Central processor complex hardware components 47 2.3.1 processor units and system control chips the two types of scms (pu and sc) are shown in figure 2-12. Figure 2-12 single chip modules (pu scm and sc scm) n20 cpc drawer both pu and sc chips use cmos 14s0 process state-of-the-art semico...

  • Page 76

    48 ibm z13s technical guide by design, each pu chip has eight cores. Core frequency is 4.3 ghz with a cycle time of 0.233 ns. When installed in a z13s server, the pu chips have either six or seven active cores. This limit means that a model n10 has 13 active cores, and the model n20 has 26 active co...

  • Page 77

    Chapter 2. Central processor complex hardware components 49 2.3.3 processor unit (core) each processor unit, or core, is a superscalar and out-of-order processor that has 10 execution units and two load/store units, which are divided into two symmetric pipelines as follows: four fixed-point units (f...

  • Page 78

    50 ibm z13s technical guide recovery unit (ru): the ru keeps a copy of the complete state of the system that includes all registers, collects hardware fault signals, and manages the hardware recovery actions. Dedicated coprocessor (cop): the dedicated coprocessor is responsible for data compression ...

  • Page 79

    Chapter 2. Central processor complex hardware components 51 2.3.5 system control chip the sc chip uses the cmos 14s0 22 nm soi technology, with 15 layers of metal. It measures 28.4 x 23.9 mm, has 7.1 billion transistors, and has 2.1 billion cells of edram. Each node of the cpc drawer has one sc chip...

  • Page 80

    52 ibm z13s technical guide 2.3.6 cache level structure z13s implements a four level cache structure, as shown in figure 2-16. Figure 2-16 cache level structure each core has its own 224-kb level 1 (l1) cache, split into 96 kb for instructions (i-cache) and 128 kb for data (d-cache). The l1 cache is...

  • Page 81: 2.4  Memory

    Chapter 2. Central processor complex hardware components 53 2.4 memory the maximum physical memory size is directly related to the number of cpc drawers in the system. The model n10 cpc drawer can contain up to 1,280 gb of physical memory. The model n20 single drawer system can contain up to 2,560 g...

  • Page 82

    54 ibm z13s technical guide 2.4.1 memory subsystem topology the z13s memory subsystem uses high speed, differential-ended communications memory channels to link a host memory to the main memory storage devices. Figure 2-17 shows an overview of the cpc drawer memory topology of a z13s server. Figure ...

  • Page 83

    Chapter 2. Central processor complex hardware components 55 2.4.2 redundant array of independent memory z13s servers use the raim technology. The raim design detects and recovers from failures of drams, sockets, memory channels, or dimms. The raim design requires the addition of one memory channel t...

  • Page 84

    56 ibm z13s technical guide 2.4.3 memory configurations physically, memory is organized in the following manner: within a drawer, all slots must be populated with the same size dimm. Each cpc drawer in a two drawer model n20 can contain different amounts of memory. A cpc drawer can have available un...

  • Page 85

    Chapter 2. Central processor complex hardware components 57 table 2-6 shows the standard memory plug summary by node for new build systems. Table 2-6 model n10 physically installed memory model n20 single drawer memory configurations the memory in model n20 with a single drawer can be configured in ...

  • Page 86

    58 ibm z13s technical guide figure 2-20 shows physical view of the n20 one cpc drawer memory dimm plug locations figure 2-20 model n20 one drawer memory plug locations table 2-7 shows the standard memory plug summary by node for new build systems. Table 2-7 model n20 single cpc drawer - physically i...

  • Page 87

    Chapter 2. Central processor complex hardware components 59 model n20 two drawer memory configurations the memory in model n20 with two drawers can be configured in the following manner: ten or twenty dimm slots per n20 drawer supported by two or four memory controllers with five slots each. The rem...

  • Page 88

    60 ibm z13s technical guide figure 2-21 shows the physical view of the n20 two cpc drawer memory dimm plug locations. Figure 2-21 model n20 two drawer memory plug locations table 2-8 shows the standard memory plug summary by node for new build systems. Table 2-8 model n20 with two cpc drawers - phys...

  • Page 89

    Chapter 2. Central processor complex hardware components 61 120 480 32 16 16 16 16 16 16 216 152 16 16 16 16 16 16 184 16 16 16 16 16 16 216 16 16 16 16 16 16 248 640 32 32 32 32 16 16 472 280 32 32 32 32 16 16 312 32 32 32 32 16 16 344 32 32 32 32 16 16 408 64 32 32 32 32 16 16 472 32 32 32 32 16 1...

  • Page 90

    62 ibm z13s technical guide figure 2-22 illustrates how the physical installed memory is allocated on a z13s server, showing hsa memory, raim, customer memory, and the remaining available unused memory that can be enabled by using lic when required. Figure 2-22 memory allocation diagram as an exampl...

  • Page 91

    Chapter 2. Central processor complex hardware components 63 cpc drawer 1 has 40 gb of hsa memory and up to 2008 gb for customer memory. Because the customer ordered 1496 gb, provided the granularity rules are met, 512 gb are available for future nondisruptive upgrades by activating lic. Memory upgra...

  • Page 92

    64 ibm z13s technical guide memory is activated for actual use. For the exact terms and conditions, contact your ibm representative. Table 2-9 feature codes for plan-ahead memory you install preplanned memory by ordering fc 1993 / 1996. The ordered amount of plan-ahead memory is charged with a reduc...

  • Page 93: 2.6  Connectivity

    Chapter 2. Central processor complex hardware components 65 downstream data signals can be spared by using two spare wires on both the upstream and downstream paths. One of these signals can be used to spare a clock signal line (one upstream and one downstream). The following improvements were also ...

  • Page 94

    66 ibm z13s technical guide figure 2-23 shows the location of the fanouts for a model n10. Model n10 has four pcie fanout slots and two ifb fanout slots. The cpc drawer has two fsps for system control. Lgxx is the location code. Up to 4 pcie fanouts (lg11 - lg14) and two ifb fanouts (lg09 - lg10) ca...

  • Page 95

    Chapter 2. Central processor complex hardware components 67 host channel adapter (hca3-o lr (1xifb)): this optical long reach fanout provides 1x infiniband coupling link connectivity up to a 10 km (6.2 miles) unrepeated, or 100 km (62 miles) when extended by using z systems qualified dwdm equipment)...

  • Page 96

    68 ibm z13s technical guide a conceptual view of how redundant i/o interconnect is accomplished is shown in figure 2-25. Figure 2-25 redundant i/o interconnect for i/o drawer normally, the hca2-c fanout in a cpc drawer connects to the ifb-mp (a) card and services domain a in an i/o drawer. In the sa...

  • Page 97: 2.7  Model Configurations

    Chapter 2. Central processor complex hardware components 69 pcie i/o connection the pcie i/o drawer supports up to 32 pcie features. They are organized in four hardware domains per drawer, as shown in figure 2-26. Figure 2-26 redundant i/o interconnect for pcie i/o drawer each domain is driven throu...

  • Page 98

    70 ibm z13s technical guide capacity marked cp: a processor that is purchased for future use as a cp is marked as available capacity. It is offline and not available for use until an upgrade for the cp is installed. It does not affect software licenses or maintenance charges. Ifl: the integrated fac...

  • Page 99

    Chapter 2. Central processor complex hardware components 71 2.7.1 upgrades concurrent cp, ifl, icf, ziip, or sap upgrades are done within a z13s server. Concurrent upgrades require available pus, and that extra pus be installed previously, but not activated. Spare pus are used to replace defective p...

  • Page 100

    72 ibm z13s technical guide 2.7.2 concurrent pu conversions assigned cps, assigned ifls, and unassigned ifls, icfs, ziips, and saps can be converted to other assigned or unassigned feature codes. Most pu conversions are nondisruptive. In exceptional cases, the conversion can be disruptive, such as w...

  • Page 101

    Chapter 2. Central processor complex hardware components 73 2.7.4 model capacity identifier and msu values all model capacity identifiers have a related msu value that is used to determine the software license charge for mlc software, as shown in table 2-12. Table 2-12 model capacity identifier and ...

  • Page 102

    74 ibm z13s technical guide 2.7.5 capacity backup the capacity backup (cbu) feature delivers temporary backup capacity in addition to what an installation might have already installed in numbers of assigned cps, ifls, icfs, ziips, and optional saps. The following cbu types are available: cbu for cp ...

  • Page 103

    Chapter 2. Central processor complex hardware components 75 the real activation of cbu lasts up to 90 days with a grace period of two days to prevent sudden deactivation when the 90-day period expires. The contract duration can be set from one to five years. The cbu record describes the following pr...

  • Page 104

    76 ibm z13s technical guide cbu for cp rules consider the following guidelines when planning for cbu for cp capacity: the total cbu cp capacity features are equal to the number of added cps plus the number of permanent cps changing capacity level. For example, if two cbu cps are added to the current...

  • Page 105: 2.8  Power and Cooling

    Chapter 2. Central processor complex hardware components 77 a cross-over does not necessarily mean that the cp count for the additional temporary capacity increases. The same d03 can temporarily be upgraded to a server with model capacity identifier f03. In this case, the number of cps does not incr...

  • Page 106

    78 ibm z13s technical guide non-raised floor environment, the top exit power and top exit i/o cabling features are mandatory. 2.8.2 high-voltage dc power in data centers today, many businesses pay increasingly expensive electric bills, and are running out of power. The ibm z systems cpc high voltage...

  • Page 107

    Chapter 2. Central processor complex hardware components 79 2. Click planning → tools → power estimation tools. 3. Specify the quantity for the features that are installed in your system. This tool estimates the power consumption for the specified configuration. The tool does not verify that the spe...

  • Page 108

    80 ibm z13s technical guide clock frequency 4.3 ghz 4.3 ghz 4.3 ghz maximum number of pcie gen3 fanouts 4 8 16 maximum number of infiniband fanouts 2 4 8 iinterconnect bus - infiniband cable 6 gbps 6 gbps 6 gbps iinterconnect bus - pcie cable 16 gbps 16 gbps 16 gbps number of support elements 2 2 2 ...

  • Page 109: Central Processor Complex

    © copyright ibm corp. 2016. All rights reserved. 81 chapter 3. Central processor complex system design this chapter explains some of the design considerations for the ibm z13s processor unit. This information can be used to understand the functions that make the z13s server a system that accommodate...

  • Page 110: 3.1  Overview

    82 ibm z13s technical guide 3.1 overview the z13s symmetric multiprocessor (smp) system is the next step in an evolutionary trajectory that began with the introduction of the ibm system/360 in 1964. Over time, the design was adapted to the changing requirements that were dictated by the shift toward...

  • Page 111

    Chapter 3. Central processor complex system design 83 the z13s system design has the following main objectives: offer a flexible infrastructure to concurrently accommodate a wide range of operating systems and applications, which range from the traditional systems (for example, z/os and z/vm) to the...

  • Page 112: 3.3  Cpc Drawer Design

    84 ibm z13s technical guide 3.3 cpc drawer design a z13s system can have up to two cpc drawers in a full configuration, up to 20 pus can be characterized, and up to 4 tb of customer usable memory capacity can be ordered. Each cpc drawer is physically divided into two nodes to improve the processor a...

  • Page 113

    Chapter 3. Central processor complex system design 85 the 4-level cache structure is implemented within the pu and storage control (sc) scms. Each node of the cpc drawer has one l4 cache. The first three levels (l1, l2, and l3) are on each pu chip (pu scm), and l4 is on the sc scms: l1 and l2 caches...

  • Page 114

    86 ibm z13s technical guide although large caches mean increased access latency, the new technology of cmos 14s0 (22 nm chip lithography) and the lower cycle time allow z13s servers to increase the size of cache levels (l1, l2, and l3) within the pu chip by using denser packaging. This design reduce...

  • Page 115

    Chapter 3. Central processor complex system design 87 compared to zbc12, the z13s cache design has much larger cache level sizes. Z13s servers have more affinity between the memory of a partition, the l4 cache in the sc scm, and the cores in the pu scms of a node. The access time of the private cach...

  • Page 116: 3.4  Processor Unit Design

    88 ibm z13s technical guide figure 3-5 shows a simplified topology of a z13s two cpc drawer system. Figure 3-5 point-to-point topology z13s two cpc drawers communication inter-cpc drawer communication takes place at the l4 cache level, which is implemented on sc scms in each node. The sc function re...

  • Page 117

    Chapter 3. Central processor complex system design 89 performance per watt improvements when compared to the zbc12 system enhanced instruction dispatch and grouping efficiency enhanced branch prediction structure and sequential instruction fetching millicode improvements decimal floating-point (dfp)...

  • Page 118

    90 ibm z13s technical guide figure 3-6 shows two threads (a and b) running on the same processor core on different pipeline stages, sharing the core resources. Figure 3-6 two threads running simultaneously on the same processor core the use of smt provides more efficient use of the processors’ resou...

  • Page 119: Simd Registers

    Chapter 3. Central processor complex system design 91 the 32 new vector registers have 128 bits. The 139 new instructions include string operations, vector integer, and vector floating point operations. Each register contains multiple data elements of a fixed size. The instructions code specifies wh...

  • Page 120

    92 ibm z13s technical guide here are some examples of simd instructions: integer byte to quadword add, sub, and compare integer byte to doubleword min, max, and average integer byte to word multiply string find 8-bits, 16-bits, and 32-bits string range compare string find any equal string load to bl...

  • Page 121

    Chapter 3. Central processor complex system design 93 program results the ooo execution does not change any program results. Execution can occur out of (program) order, but all program dependencies are honored, ending up with the same results of the in-order (program) execution. This implementation ...

  • Page 122

    94 ibm z13s technical guide figure 3-10 shows how ooo core execution can reduce the run time of a program. Figure 3-10 z13s in-order and out-of-order core execution improvements the left side of figure 3-10 shows an in-order core execution. Instruction 2 has a large delay because of an l1 cache miss...

  • Page 123

    Chapter 3. Central processor complex system design 95 3.4.4 superscalar processor a scalar processor is a processor that is based on a single-issue architecture, which means that only a single instruction is run at a time. A superscalar processor allows concurrent (parallel) execution of instruction...

  • Page 124

    96 ibm z13s technical guide figure 3-11 shows the location of the coprocessor on the chip. Figure 3-11 compression and cryptography accelerators on a core in the chip cp assist for cryptographic function cpacf accelerates the encrypting and decrypting of ssl/tls transactions, virtual private network...

  • Page 125

    Chapter 3. Central processor complex system design 97 benefits of the dfp accelerator the dfp accelerator offers the following benefits: avoids rounding issues, such as those that happen with binary-to-decimal conversions. Controls existing binary-coded decimal (bcd) operations better. Follows the d...

  • Page 126

    98 ibm z13s technical guide under hardware control is possible because the r-unit has the full designed state in its buffer. The principle is shown in figure 3-12. Figure 3-12 pu error detection and recovery 3.4.9 branch prediction because of the ultra-high frequency of the pus, the penalty for a wr...

  • Page 127

    Chapter 3. Central processor complex system design 99 3.4.11 translation lookaside buffer the tlb in the instruction and data l1 caches use a secondary tlb to enhance performance. In addition, a translator unit is added to translate misses in the secondary tlb. The size of the tlb is kept as small a...

  • Page 128

    100 ibm z13s technical guide 3.4.14 instruction set extensions the processor supports many instructions to support functions: hexadecimal floating point instructions for various unnormalized multiply and multiply-add instructions. Immediate instructions, including various add, compare, or, exclusive...

  • Page 129

    Chapter 3. Central processor complex system design 101 the function that is assigned to a pu is set by the licensed internal code (lic). The lic is loaded when the system is initialized (at power-on reset (por)) and the pus are characterized . Only characterized pus have a designated function. Non-c...

  • Page 130

    102 ibm z13s technical guide an ifl pool of two ifls a ziip pool of five ziips pus are placed in the pools in the following circumstances: when the system undergoes por at the time of a concurrent upgrade as a result of the addition of pus during a cbu following a capacity on-demand upgrade through ...

  • Page 131

    Chapter 3. Central processor complex system design 103 for more information about capacity settings, see 2.7, “model configurations” on page 69. 3.5.3 integrated facility for linux an integrated facility for linux (ifl) is a pu that can be used to run linux, linux guests on z/vm operating systems, k...

  • Page 132

    104 ibm z13s technical guide coupling thin interrupts with the introduction of driver 15f (zec12 and ibm zenterprise bc12 (zbc12)), the z systems architecture provides a new thin interrupt class called coupling thin interrupts . The capabilities that are provided by hardware, firmware, and software ...

  • Page 133

    Chapter 3. Central processor complex system design 105 in figure 3-13, the cpc on the left has two environments that are defined (production and test), each of which has one z/os and one coupling facility image. The coupling facility images share an icf. Figure 3-13 icf options: shared icfs the lpar...

  • Page 134

    106 ibm z13s technical guide and supported by the z/os operating system. Smt enables continued scaling of per-processor capacity. How ziips work ziips are designed to support designated z/os workloads. One of the workloads is java code execution. When java code must be run (for example, under contro...

  • Page 135

    Chapter 3. Central processor complex system design 107 a ziip runs only ibm authorized code. This ibm authorized code includes the z/os jvm in association with parts of system code, such as the z/os dispatcher and supervisor services. A ziip cannot process i/o or clock comparator interruptions, and ...

  • Page 136

    108 ibm z13s technical guide data in a central fact table and is surrounded by more dimension tables that hold information about each perspective of the data. A star schema query, for example, joins various dimensions of a star schema data set. Db2 utilities that are used for index maintenance, such...

  • Page 137

    Chapter 3. Central processor complex system design 109 ziips are orderable by using fc 1927. Up to two ziip can be ordered for each cp or marked cp configured in the system. Pus characterized as ziips within a configuration are grouped into the ziip pool. This configuration allows ziips to have thei...

  • Page 138

    110 ibm z13s technical guide 3.5.7 reserved processors reserved processors are defined by the pr/sm to allow for a nondisruptive capacity upgrade. Reserved processors are like spare logical processors, and can be shared or dedicated. Reserved cps can be defined to an lpar dynamically to allow for no...

  • Page 139

    Chapter 3. Central processor complex system design 111 the assignment rules follow this order: spare: on cpc drawer 1, no spare is assigned on the high pu scm. In the model n10, no spares are assigned. Ifp: if only cpc drawer 1 is present (model n10, one cpc drawer machine, or model n20, one cpc dra...

  • Page 140: 3.6  Memory Design

    112 ibm z13s technical guide application preservation if no spare pu is available, application preservation (z/os only) is started. The state of the failing processor is passed to another active processor that is used by the operating system. Through operating system recovery services, the task is r...

  • Page 141

    Chapter 3. Central processor complex system design 113 pr/sm has control over all physical memory, and can so make physical memory available to the configuration when a cpc drawer is added. In prior z systems processors, memory allocation was striped across the available cpc drawers because there wa...

  • Page 142

    114 ibm z13s technical guide memory distribution across the cpc drawers will be different, as will be the total amount of available memory. Z13s cpc drawer memory information on a z13s model n10 machine, the maximum amount of memory is limited to 984 gb of customer addressable memory. If more memory...

  • Page 143

    Chapter 3. Central processor complex system design 115 resolve all the memory it needs. Therefore, a long-running process can benefit even without frequent memory access. Weigh the benefits of whether something in this category must use large pages as a result of the system-level costs of tying up r...

  • Page 144: 3.7  Logical Partitioning

    116 ibm z13s technical guide z13s servers can run in lpar or dynamic partition manager (dpm) mode. For dpm mode, see appendix e, “ibm dynamic partition manager” on page 501. When running in lpar mode, storage is placed into a single storage pool that is called the lpar single storage pool . This poo...

  • Page 145

    Chapter 3. Central processor complex system design 117 3.7.1 overview logical partitioning is a function that is implemented by the pr/sm on z13s servers. The z13s server runs either in lpar-, or dynamic partition manager mode. Therefore, all system aspects are controlled by pr/sm functions. Pr/sm i...

  • Page 146

    118 ibm z13s technical guide hiperdispatch pr/sm and z/os work in tandem to use processor resources more efficiently. Hiperdispatch is a function that combines the dispatcher actions and the knowledge that pr/sm has about the topology of the system. Performance can be optimized by redispatching unit...

  • Page 147

    Chapter 3. Central processor complex system design 119 the weight and number of online logical processors of an lpar can be dynamically managed by the lpar cpu management function of the intelligent resource director (ird). These processors can then be used to achieve the defined goals of this speci...

  • Page 148

    120 ibm z13s technical guide channels: channels can be shared between lpars by including the partition name in the partition list of a channel-path identifier (chpid). I/o configurations are defined by the iocp or the hcd with the chpid mapping tool (cmt). The cmt is an optional tool that is used to...

  • Page 149

    Chapter 3. Central processor complex system design 121 logically partitioned mode if the z13s server runs in lpar mode, each of the 40 lpars can be defined to operate in one of the following image modes: esa/390 mode to run the following systems: – a z/architecture operating system, on dedicated or ...

  • Page 150

    122 ibm z13s technical guide table 3-5 shows all lpar modes, required characterized pus, operating systems, and the pu characterizations that can be configured to an lpar image. The available combinations of dedicated (ded) and shared (shr) processors are also shown. For all combinations, an lpar al...

  • Page 151

    Chapter 3. Central processor complex system design 123 the dynamic addition of these resources. In z/os, this support is available since version 1 release 10 (z/os v1.10). Adding a crypto feature to a logical partition you can plan the addition of crypto express5s features to an lpar on the crypto p...

  • Page 152

    124 ibm z13s technical guide provides an enforceable upper limit on the amount of the specified processor type that can be used in a partition. Absolute capping is ideal for processor types and operating systems that the z/os wlm cannot control. Absolute capping is not intended as a replacement for ...

  • Page 153

    Chapter 3. Central processor complex system design 125 before you activate an lpar, main storage (and optionally, expanded storage) must be defined to the lpar. All installed storage can be configured as main storage. Each z/os individual lpar can be defined with a maximum of 4 tb of main storage. Z...

  • Page 154

    126 ibm z13s technical guide certain z/architecture operating systems, such as z/os, always change the 31-bit addressing mode and operate in 64-bit mode. Other z/architecture operating systems, such as z/vm, can be configured to change to 64-bit mode or to stay in 31-bit mode and operate in the esa/...

  • Page 155

    Chapter 3. Central processor complex system design 127 linux only mode in linux only mode, storage addressing can be 31 bit or 64 bit, depending on the operating system architecture and the operating system configuration, in the same way as in esa/390 mode. Only linux and z/vm operating systems can ...

  • Page 156

    128 ibm z13s technical guide without the reserved storage definition, an lpar storage upgrade is a disruptive process that requires the following actions: 1. Partition deactivation 2. An initial storage size definition change 3. Partition activation the additional storage capacity for an lpar upgrad...

  • Page 157

    Chapter 3. Central processor complex system design 129 3.7.5 lpar dynamic storage reconfiguration dynamic storage reconfiguration on z13s servers allows an operating system that is running on an lpar to add (nondisruptively) its reserved storage amount to its configuration. This process can occur on...

  • Page 158

    130 ibm z13s technical guide ird has the following characteristics: ird processor management: wlm dynamically adjusts the number of logical processors within an lpar and the processor weight based on the wlm policy. The ability to move the processor weights across an lpar cluster provides processing...

  • Page 159: 3.9  Clustering Technology

    Chapter 3. Central processor complex system design 131 3.9 clustering technology parallel sysplex is the clustering technology that is used with z13 and z13s servers. Figure 3-16 illustrates the components of a parallel sysplex as implemented within the z systems architecture. The figure shows one o...

  • Page 160

    132 ibm z13s technical guide systems management: the architecture provides the infrastructure to satisfy client requirements for continuous availability, and provides techniques for achieving simplified systems management consistent with this requirement. Resource sharing: a number of base (z/os) co...

  • Page 161

    Chapter 3. Central processor complex system design 133 the cfcc is implemented by using the active wait technique. This technique means that the cfcc is always running (processing or searching for service) and never enters a wait state. It also means that the cf control code uses all the processor c...

  • Page 162

    134 ibm z13s technical guide 3. If there is no work, cf sleeps again for a longer period. 4. If there is new work, cf enters the normal active wait until the work is completed. After all work is complete, the process starts again. With the introduction of the coupling thin interrupt support, which i...

  • Page 163

    Chapter 3. Central processor complex system design 135 3.9.4 cfcc and flash express use from cfcc level 19 and later, flash express can be used. It improves resilience while providing cost-effective standby capacity to help manage the potential overflow of ibm websphere mq shared queues. Structures ...

  • Page 164

    136 ibm z13s technical guide.

  • Page 165: System Structure

    © copyright ibm corp. 2016. All rights reserved. 137 chapter 4. Central processor complex i/o system structure this chapter describes the i/o system structure and connectivity options that are available on z13s servers. This chapter includes the following sections: introduction to the infiniband and...

  • Page 166: Infrastructure

    138 ibm z13s technical guide 4.1 introduction to the infiniband and pcie for i/o infrastructure z13s servers support two types of internal i/o infrastructure: infiniband infrastructure for i/o drawers peripheral component interconnect express (pcie)-based infrastructure for pcie i/o drawers 4.1.1 in...

  • Page 167

    Chapter 4. Central processor complex i/o system structure 139 4.1.3 infiniband specifications the infiniband specification defines a point-to-point, bidirectional serial communications between two nodes. It is intended for the connection of processors with high-speed peripheral devices, such as disk...

  • Page 168: 4.2  I/o System Overview

    140 ibm z13s technical guide a pcie gen3 x16 link has these data transmission rates: data transmission rate per lane: 8 gb/s * 128/130 bit (encoding) = 7.87 gb/s=984.6 mb/s data transmission rate per link: 984.6 mb/s * 16 (lanes) = 15.75 gb/s considering the pcie link is full-duplex mode, the data t...

  • Page 169

    Chapter 4. Central processor complex i/o system structure 141 concurrent i/o upgrade – you can concurrently add i/o features to z13s servers if unused i/o slot positions are available. Concurrent pcie i/o drawer upgrade – extra pcie i/o drawers can be installed concurrently if free frame slots for p...

  • Page 170: 4.3  I/o Drawer

    142 ibm z13s technical guide 4.3 i/o drawer the i/o drawer is five eia units high, and supports up to eight i/o feature cards. Each i/o drawer supports two i/o domains (a and b) for a total of eight i/o card slots. Each i/o domain uses an infiniband multiplexer (ifb-mp) card in the i/o drawer and a ...

  • Page 171: 4.4  Pcie I/o Drawer

    Chapter 4. Central processor complex i/o system structure 143 the i/o drawer domains and their related i/o slots are shown in figure 4-2. The ifb-mp cards are installed at slot 09 at the rear side of the i/o drawer. The i/o features are installed from the front and rear side of the i/o drawer. Two i...

  • Page 172: Rear

    144 ibm z13s technical guide the locations of the dcas, amds, pcie switch cards, and i/o feature cards in the pcie i/o drawer are shown in figure 4-3. Figure 4-3 pcie i/o drawer 7u 7u ~311mm rear front domain 3 domain 1 domain 0 domain 2 amd 16 gbps pcie switch card 560mm (max) dca.

  • Page 173

    Chapter 4. Central processor complex i/o system structure 145 the i/o structure in a z13s cpc is shown in figure 4-4. The pcie switch card provides the fanout from the high-speed x16 pcie host bus to eight individual card slots. The pcie switch card is connected to the drawers through a single x16 p...

  • Page 174

    146 ibm z13s technical guide the pcie i/o drawer domains and their related i/o slots are shown in figure 4-5. Figure 4-5 pcie i/o drawer with 32 pcie slots and four i/o domains each i/o domain supports up to eight features (ficon, osa, and crypto), and each drawer supports up to two flash express fe...

  • Page 175: 4.6  Fanouts

    Chapter 4. Central processor complex i/o system structure 147 features determines how many pcie i/o drawers are required. During upgrade to a z13s server, the correct mix of i/o drawers and pcie i/o drawers is determined based on the type and number of i/o features (new and carry-forward) that are r...

  • Page 176

    148 ibm z13s technical guide five types of fanout cards are supported by z13s servers. Each slot can hold one of the following five fanouts: pcie gen3 fanout card: this copper fanout provides connectivity to the pcie switch card in the pcie i/o drawer. Ica sr: this adapter provides 8x pcie optical c...

  • Page 177

    Chapter 4. Central processor complex i/o system structure 149 4.6.1 pcie generation 3 fanout (fc 0173) the pcie generation 3 fanout card provides connectivity to an pcie i/o drawer by using a copper cable. One port on the fanout card is dedicated for pcie i/o. The bandwidth of this pcie fanout card ...

  • Page 178

    150 ibm z13s technical guide the ica sr can be used only for coupling connectivity between z13 and z13s servers. It does not support connectivity to zec12, zbc12, z196, or z114 servers, and it cannot be connected to hca3-o or hca3-o lr coupling fanouts. The ica sr fanout requires new cabling that is...

  • Page 179

    Chapter 4. Central processor complex i/o system structure 151 data set (iocds). The coupling links can be defined as shared between images within a css. They also can be spanned across multiple csss in a cpc. Each hca3-o (12x ifb) fanout has an assigned adapter id (aid) number. This number must be u...

  • Page 180

    152 ibm z13s technical guide each hca3-o lr (1xifb) fanout has an assigned aid number. This number must be used for definitions in iocds to create a relationship between the physical fanout location and the chpid number. For more information about aid numbering, see “adapter id number assignment” on...

  • Page 181

    Chapter 4. Central processor complex i/o system structure 153 table 4-4 illustrates the aid assignment for each fanout slot relative to the drawer location on a new build system. Table 4-4 aid number assignment fanout slots the fanout slots are numbered lg03 - lg14 left to right, as shown in table 4...

  • Page 182: 4.7  I/o Features (Cards)

    154 ibm z13s technical guide fanout features that are supported by the z13s server are shown in table 4-5. The table provides the feature type, feature code, and information about the link that is supported by the fanout feature. Table 4-5 fanout summary 4.7 i/o features (cards) i/o features (cards)...

  • Page 183

    Chapter 4. Central processor complex i/o system structure 155 4.7.1 i/o feature card ordering information table 4-6 lists the i/o features that are supported by z13s servers and the ordering information for them. Table 4-6 i/o features and ordering information channel feature feature code new build ...

  • Page 184

    156 ibm z13s technical guide 4.7.2 physical channel report a pchid reflects the physical location of a channel-type interface. A pchid number is based on these factors: the i/o drawer, and pcie i/o drawer location the channel feature slot number the port number of the channel feature a chpid does no...

  • Page 185: 4.8  Connectivity

    Chapter 4. Central processor complex i/o system structure 157 the following list explains the content of this sample pchid report: feature code 0170 (hca3-o lr (1xifb)) is installed in cpc drawer 1 (location a21a, slot lg07), and has aid 04 assigned. Feature code 0172 (integrated coupling adapter - ...

  • Page 186

    158 ibm z13s technical guide 4.8.1 i/o feature support and configuration rules table 4-7 lists the i/o features that are supported. The table shows the number of ports per card, port increments, the maximum number of feature cards, and the maximum number of channels for each feature type. Also, the ...

  • Page 187

    Chapter 4. Central processor complex i/o system structure 159 coupling links that are defined as cs5, cib, or icp hipersockets that are defined as iqd 5 the following features are exclusively plugged into a pcie i/o drawer and do not require the definition of a chpid and chpid type: each crypto expr...

  • Page 188

    160 ibm z13s technical guide 4.8.2 ficon channels the ficon express16s, ficon express8s, and ficon express8 features conform to the following architectures: fibre connection (ficon) high performance ficon on z systems (zhpf) fibre channel protocol (fcp) they provide connectivity between any combinat...

  • Page 189

    Chapter 4. Central processor complex i/o system structure 161 each ficon express8 feature occupies one i/o slot in the i/o drawer. Each feature has four ports, each supporting an lc duplex connector, with one pchid and one chpid associated with each port. All ficon express16s, ficon express8s, and f...

  • Page 190

    162 ibm z13s technical guide ficon express8s the ficon express8s feature installs exclusively in the pcie i/o drawer. Each of the two independent ports is capable of 2 gbps, 4 gbps, or 8 gbps. The link speed depends on the capability of the attached switch or device. The link speed is auto-negotiate...

  • Page 191

    Chapter 4. Central processor complex i/o system structure 163 forward error correction forward error correction (fec) is a technique used for reducing data errors when transmitting over unreliable or noisy communication channels (improving signal to noise ratio). By adding redundancy error-correctio...

  • Page 192

    164 ibm z13s technical guide improve utilization of ficon director and isl easier to manage with a predicable and repeatable i/o performance. Ficon dynamic routing can be enabled by definition of dynamic routing capable switches and control units in hcd. Also, z/os has implemented a health check fun...

  • Page 193

    Chapter 4. Central processor complex i/o system structure 165 ficon support for multiple-hop cascaded san configurations before the introduction of the z13 and z13s servers, z systems ficon san configurations only supported a single isl (or single hop) in a cascaded ficon san environment. Ibm z13s s...

  • Page 194

    166 ibm z13s technical guide 4.8.3 osa-express5s the osa-express5s feature is exclusively in the pcie i/o drawer. The following osa-express5s features can be installed on z13s servers: osa-express5s 10 gigabit ethernet lr, fc 0415 osa-express5s 10 gigabit ethernet sr, fc 0416 osa-express5s gigabit e...

  • Page 195

    Chapter 4. Central processor complex i/o system structure 167 defined as chpid type osx, the 10 gbe port provides connectivity and access control to the intraensemble data network (iedn) from z13s servers to z bladecenter extension (zbx). The 10 gbe feature is designed to support attachment to a sin...

  • Page 196

    168 ibm z13s technical guide osa-express5s gigabit ethernet sx (fc 0414) the osa-express5s gigabit ethernet (gbe) short wavelength (sx) feature has one pcie adapter and two ports. The two ports share a channel path identifier (chpid type osd exclusively). The ports support attachment to a 1 gbps eth...

  • Page 197

    Chapter 4. Central processor complex i/o system structure 169 osa-express4s gigabit ethernet lx, fc 0404 osa-express4s gigabit ethernet sx, fc 0405 osa-express4s 1000base-t ethernet, fc 0408 table 4-11 lists the characteristics of the osa-express4s features. Table 4-11 osa-express4s features osa-exp...

  • Page 198

    170 ibm z13s technical guide spanned channel, and can be shared among lpars within and across logical channel subsystems. The osa-express4s 10 gbe sr feature supports the use of an industry standard small form factor lc duplex connector. Ensure that the attaching or downstream device has an sr trans...

  • Page 199

    Chapter 4. Central processor complex i/o system structure 171 the osa-express4s 1000base-t ethernet feature supports auto-negotiation when attached to an ethernet router or switch. If you allow the lan speed and duplex mode to default to auto-negotiation, the osa-express port and the attached router...

  • Page 200

    172 ibm z13s technical guide intranode management network the inmn is a private and physically isolated 1000base-t ethernet internal management network that operates at 1 gbps. It connects all resources (z13s and zbx model 004 components) of an ensemble node for management purposes. It is pre-wired,...

  • Page 201

    Chapter 4. Central processor complex i/o system structure 173 traffic can be ipv4 or ipv6, or non-ip, such as appletalk, decnet, ipx, netbios, or sna. Hipersockets devices are protocol-independent and layer 3 independent. Each hipersockets device (layer 2 and layer 3 mode) has its own mac address. T...

  • Page 202

    174 ibm z13s technical guide shared memory communication - direct memory access with the z13 and z13s servers, ibm introduces shared memory communication - direct memory access (smc-d). Smc-d maintains the socket-api transparency aspect of smc-r so that applications using tcp/ip communications can b...

  • Page 203

    Chapter 4. Central processor complex i/o system structure 175 chpids per port and seven subchannels (devices) per chpid. Ica sr supports transmission of stp messages. Parallel sysplex using ifb connects to z13, z13s, zec12, zbc12, z196, or z114 servers. 12x infiniband coupling links are fiber optic ...

  • Page 204

    176 ibm z13s technical guide the maximum for ifb links is 32, for ica sr is 16, and for ic is 32. The maximum number of combined external coupling links (active ica sr links and ifb lr) is 80 per z13s server. And z13s servers support up to 256 coupling chpids per cpc, which provides enhanced connect...

  • Page 205

    Chapter 4. Central processor complex i/o system structure 177 to eliminate any single points of failure in a parallel sysplex configuration, have at least the following components: two coupling links between the z/os and coupling facility images. Two coupling facility images not running on the same ...

  • Page 206

    178 ibm z13s technical guide provide the fastest connectivity, which is faster than any external link alternatives. Result in better coupling efficiency than with external links, effectively reducing the server cost that is associated with parallel sysplex technology. Can be used in test or producti...

  • Page 207

    Chapter 4. Central processor complex i/o system structure 179 up to eight pcie and four ifb fanout cards can be installed in each cpc drawer, as shown in figure 4-9. Figure 4-9 cpc drawer front view: coupling links previous generation z systems machines, the z114 and zbc12, use a different design of...

  • Page 208

    180 ibm z13s technical guide in this case, a z13s-n20 is needed to fulfill all ifb connectivity, as shown in figure 4-10. Figure 4-10 hca3-o fanouts: z13s versus z114 / zbc12 servers it is beyond the scope of this book to describe all possible migration scenarios. Always involve subject matter exper...

  • Page 209

    Chapter 4. Central processor complex i/o system structure 181 by eliminating ficon express8 (and thus the need to carry forward the i/o drawers), you can preserve infiniband fanout slots for coupling over infiniband fanouts. Replace ficon express8 with ficon express8s or (depending on your san fabri...

  • Page 210

    182 ibm z13s technical guide coupling links and server time protocol all external coupling links can be used to pass time synchronization signals by using stp. Stp is a message-based protocol in which timing messages are passed over data links between servers. The same coupling links can be used to ...

  • Page 211

    Chapter 4. Central processor complex i/o system structure 183 for more information about stp configuration, see these books: server time protocol planning guide, sg24-7280 server time protocol implementation guide, sg24-7281 server time protocol recovery guide, sg24-7380 4.9.3 pulse per second input...

  • Page 212: 4.12  Flash Express

    184 ibm z13s technical guide 10gbe roce express all native pcie features should be ordered in pairs for redundancy. According to their physical location in the pcie i/o drawer, the features are assigned to one of the two resource groups that are running on the ifp, providing management functions and...

  • Page 213: 4.13  10Gbe Roce Express

    Chapter 4. Central processor complex i/o system structure 185 figure 4-11 shows a pcie i/o drawer that is fully populated with flash express cards. Figure 4-11 pcie i/o drawer that is fully populated with flash express cards 4.12.1 ibm flash express read/write cache on ibm z13s and z13 servers, driv...

  • Page 214: 4.14  Zedc Express

    186 ibm z13s technical guide the maximum supported unrepeated distance, point-to-point, is 300 meters (984 ft.). A client-supplied cable is required. Three types of cables can be used for connecting the port to the selected 10 gbe switch or to a 10gbe roce express feature on another server: om3 50 m...

  • Page 215: Central Processor Complex

    © copyright ibm corp. 2016. All rights reserved. 187 chapter 5. Central processor complex channel subsystem this chapter addresses the concepts of the ibm z13s channel subsystem, including multiple channel subsystems and multiple subchannel sets. It also describes the technology, terminology, and im...

  • Page 216: 5.1  Channel Subsystem

    188 ibm z13s technical guide 5.1 channel subsystem channel subsystem (css) is a group of facilities that z systems uses to control i/o operations. The channel subsystem directs the flow of information between i/o devices and main storage. It allows data processing to occur at the same time as i/o pr...

  • Page 217

    Chapter 5. Central processor complex channel subsystem 189 purchased memory. This fixed size memory for hsa eliminates the requirement of additional planning for initial i/o configuration and pre-planning for future i/o expansions. The objects are always reserved in the z13s hsa during por, no matte...

  • Page 218

    190 ibm z13s technical guide connections, a channel path can be a physical connection to a control unit with i/o devices, like ficon, or an internal logical one, like hipersockets. Each channel path in a channel subsystem has a unique 2-digit heximal identifier that is known as a channel-path identi...

  • Page 219

    Chapter 5. Central processor complex channel subsystem 191 subchannel number the subchannel number is a four-digit heximal number, ranging from 0x0000 to 0xffff, assigned to a subchannel within a subchannel set of a channel subsystem. Subchannels in each subchannel set are always assigned subchannel...

  • Page 220

    192 ibm z13s technical guide device in subchannel set 0 and additional subchannel sets an lcss will always has the first subchannel set (ssid 0), which can have up to 63.75k subchannels, with 256 subchannels reserved by the channel subsystem. That user can always define their i/o devices in this sub...

  • Page 221

    Chapter 5. Central processor complex channel subsystem 193 the display ios,config command the z/os display ios,config(all) command that is shown in figure 5-2 includes information about the multiple subchannel sets. Figure 5-2 output for display ios,config(all) command 5.1.3 channel path spanning wi...

  • Page 222

    194 ibm z13s technical guide a sample of channel paths that are defined as dedicated, shared, and spanned is shown in figure 5-3. Figure 5-3 z systems css: channel subsystems with channel spanning in the sample, three different definitions of a channel path are shown: chpid ff, assigned to pchid 20a...

  • Page 223

    Chapter 5. Central processor complex channel subsystem 195 an example that summarizes the definition of these lpar-related identifications is shown in figure 5-4. Figure 5-4 css, lpar, and identifier example lpar name the lpar name is defined as partition name parameter in the resource statement of ...

  • Page 224

    196 ibm z13s technical guide 5.2 i/o configuration management the following tools are available to help maintain and optimize the i/o configuration: ibm configurator for e-business (econfig): the econfig tool is available from your ibm representative. It is used to create configurations or upgrades ...

  • Page 225

    Chapter 5. Central processor complex channel subsystem 197 maximum number of subchannels per css 191.74 k ss0: 65280 ss1 - ss2: 65535 maximum number of chpids per css 256.

  • Page 226

    198 ibm z13s technical guide.

  • Page 227: Cryptography

    © copyright ibm corp. 2016. All rights reserved. 199 chapter 6. Cryptography this chapter describes the hardware cryptographic functions that are available on ibm z13s servers. The cp assist for cryptographic function (cpacf) with the peripheral component interconnect express (pcie) cryptographic co...

  • Page 228

    200 ibm z13s technical guide 6.1 cryptography in ibm z13 and z13s servers ibm z13 and z13s servers introduce the new pci crypto express5s feature, together with a redesigned cpacf coprocessor, managed by a new trusted key entry (tke) workstation. Also, the ibm common cryptographic architecture (cca)...

  • Page 229

    Chapter 6. Cryptography 201 that the intended party can de-scramble the data but an interloper cannot. This idea is also referred to as confidentiality. Authentication: this is the process of determining whether the partners in communication are who they claim to be, which can be done by using certi...

  • Page 230

    202 ibm z13s technical guide when using a secret encryption method, it is possible that a back door has been designed into it. If an algorithm is public, many experts can form an opinion about it, and the method can be more thoroughly investigated for potential weaknesses and vulnerabilities. 6.2.3 ...

  • Page 231

    Chapter 6. Cryptography 203 because a secure key must be handled in a special hardware device, the use of secret keys is usually far slower than using clear keys as illustrated in figure 6-1. Figure 6-1 three levels of protection with three levels of speed. 6.2.4 algorithms the algorithms of modern ...

  • Page 232

    204 ibm z13s technical guide one-way algorithms are not strictly speaking cryptographic functions at all. They do not use keys, and can only scramble data. They cannot de-scramble it. These algorithms are used extensively within cryptographic procedures for digital signing and they tend to be develo...

  • Page 233

    Chapter 6. Cryptography 205 finally, for entering keys in a secure way into the crypto express5s hsm, a tke is required, usually also equipped with smart card readers. Section 6.6, “tke workstation” on page 222 provides additional information. Table 6-1 lists the feature codes and describes the purp...

  • Page 234

    206 ibm z13s technical guide to access and use the cryptographic hardware devices that are provided by z13s servers, the application must use an application programming interface (api) provided by the operating system. In z/os, the integrated cryptographic service facility (icsf) provides the apis a...

  • Page 235

    Chapter 6. Cryptography 207 6.4 cp assist for cryptographic functions as already mentioned, attached to every pu on an scm in a cpc of a z13s server are two independent engines, one for compression and one for cryptographic purposes, as shown in figure 6-4. This cryptographic coprocessor, called the...

  • Page 236

    208 ibm z13s technical guide these functions are provided as problem-state z/architecture instructions that are directly available to application programs. These instructions are known as message-security assist (msa). When enabled, the cpacf runs at processor speed for every cp, ifl, and ziip. For ...

  • Page 237

    Chapter 6. Cryptography 209 and protected keys can be used. Protected keys require a crypto express5s card running in cca mode. For more information, see 6.5.2, “crypto express5s as a cca coprocessor” on page 214. The hashing algorithm sha-1, and sha-2 support for sha-224, sha-256, sha-384, and sha-...

  • Page 238

    210 ibm z13s technical guide wrapping keys are generated during the clear reset each time an lpar is activated or reset. No customizable option is available at the support element (se) or hardware management console (hmc) that permits or avoids the wrapping key generation. Figure 6-5 shows this func...

  • Page 239: 6.5  Crypto Express5S

    Chapter 6. Cryptography 211 6.5 crypto express5s the crypto express5s feature (fc 0890) is an optional feature that is exclusive to z13 and z13s servers. Each feature has one pcie cryptographic adapter. The crypto express5s feature occupies one i/o slot in a z13 or z13s pcie i/o drawer. This feature...

  • Page 240

    212 ibm z13s technical guide each z13s server supports up to 16 crypto express5s features. Table 6-2 shows configuration information for crypto express5s. Table 6-2 crypto express5s features the concept of dedicated processor does not apply to the pcie cryptographic adapter. Whether configured as a ...

  • Page 241

    Chapter 6. Cryptography 213 6.5.1 cryptographic asynchronous functions the optional pcie cryptographic coprocessor crypto express5s provides asynchronous cryptographic functions to z13s servers. Over 300 cryptographic algorithms and modes are supported, including the following: des/tdes w des/tdes m...

  • Page 242

    214 ibm z13s technical guide several of these algorithms require a secure key and must run on an hsm, some of them can also run with a clear key on the cpacf. Many standards are only supported when the crypto express5s card is running in cca mode, many also when the card is running in ep11 mode. The...

  • Page 243

    Chapter 6. Cryptography 215 on z13s servers, the crypto express5s card is delivered with cca level 5.2 firmware. A new set of cryptographic functions and callable services are provided by the ibm cca lic to enhance the functions that secure financial transactions and keys: greater than 16 domains su...

  • Page 244

    216 ibm z13s technical guide vfpe allows customers to add encryption to their applications in such a way that the encrypted data can flow through their systems without requiring a massive redesign of their application. In the example, if the credit card number is vfpe-encrypted at the point of entry...

  • Page 245

    Chapter 6. Cryptography 217 pka translate udx function into cca udx is custom code that allows the client to add unique operations and extensions to the cca firmware. Certain udx functions are integrated into the base cca code over time to accomplish the following tasks: removes headaches and pain p...

  • Page 246

    218 ibm z13s technical guide software requirements: – z/os v2.2 – z/os v2.1or z/os v1.13 with the cryptographic support for z/os v1r13-z/os v2r1 web deliverable (fmid hcr77b1) with ptfs – z/vm 5.4, 6.2, and 6.3 with ptfs for guest exploitation 6.5.3 crypto express5s as an ep11 coprocessor a crypto e...

  • Page 247

    Chapter 6. Cryptography 219 6.5.5 management of crypto express5s with zec12 and older servers, each cryptographic coprocessor has 16 physical sets of registers or queue registers. With z13, this number is raised to 85. This amount corresponds to the maximum number of lpars running on a z13, which is...

  • Page 248

    220 ibm z13s technical guide this task is accomplished by using the customize/delete activation profile task, which is in the operational customization group, either from the hmc or from the se. Modify the cryptographic initial definition from the crypto option in the image profile, as shown in figu...

  • Page 249

    Chapter 6. Cryptography 221 cryptographic candidate list identifies the cryptographic coprocessor numbers that are eligible to be accessed by this logical partition. From the list, select the coprocessor numbers, in the range 0 - 15, that identify the pci-x adapters to be accessed by this partition....

  • Page 250: 6.6  Tke Workstation

    222 ibm z13s technical guide 6.6 tke workstation the tke workstation is an optional feature that offers key management functions. A tke tower workstation feature (fc 0847) or a tke rack mounted workstation feature (fc 0097) is required for z13s servers to manage the crypto express5s feature. The tke...

  • Page 251

    Chapter 6. Cryptography 223 6.6.3 tke workstation with licensed internal code 8.0 to control the crypto express5s card in a z13s server, a tke workstation (fc 0847 or 0097) with lic 8.0 (fc 0877) or lic 8.1 (fc 0878) is required. Lic 8.0 does not provide the new functions of lic 8.1. Tke lic 8.1 is ...

  • Page 252

    224 ibm z13s technical guide 6.6.5 tke hardware support and migration information the new tke 8.1 lic (fc 0878) is shipped with z13 servers after ga2, and with z13s servers. If a new tke 8.1 is purchased, two versions are available: tke 8.1 tower workstation (fc 0847) tke 8.1 rack-mounted workstatio...

  • Page 253

    Chapter 6. Cryptography 225 6.7 cryptographic functions comparison table 6-4 lists the functions or attributes on z13s servers for the two cryptographic hardware features. In the table, “x” indicates that the function or attribute is supported. Table 6-4 cryptographic functions on z13s servers serve...

  • Page 254

    226 ibm z13s technical guide provides the highest asymmetric (encrypted key) encryption performance - x x - disruptive process to enable - note a note a note a requires iocds definition - - - - uses chpid numbers - - - - uses pchids (one pchid) - x x x requires cpacf enablement (fc 3863) x b x b x b...

  • Page 255

    Chapter 6. Cryptography 227 6.8 cryptographic software support the software support levels for cryptographic functions are listed in chapter 7.4, “cryptographic support” on page 285. Iso 16609 cbc mode triple des message authentication code (mac) support - x - - aes gmac, aes gcm, aes xts mode, cmac...

  • Page 256

    228 ibm z13s technical guide.

  • Page 257: Software Support

    © copyright ibm corp. 2016. All rights reserved. 229 chapter 7. Software support this chapter lists the minimum operating system requirements and support considerations for the ibm z13s (z13s) and its features. It addresses z/os, z/vm, z/vse, z/tpf, linux on z systems, and kvm for ibm z systems. Bec...

  • Page 258

    230 ibm z13s technical guide 7.1 operating systems summary table 7-1 lists the minimum operating system levels that are required on the z13s. For similar information about the ibm zbx model 004, see 7.15, “ibm z bladecenter extension (zbx) model 004 software support” on page 304. Table 7-1 z13s mini...

  • Page 259

    Chapter 7. Software support 231 functions that are available on previous servers but no longer supported by z13s have been removed. For a list of supported functions and the z/os and z/vm minimum required support levels, see table 7-3 on page 234for z/vse, z/tpf, and linux on z systems, see table 7-...

  • Page 260

    232 ibm z13s technical guide 7.2.3 z/vse support is provided by z/vse v5r1 and later. Note the following considerations: z/vse runs in z/architecture mode only. Z/vse supports 64-bit real and virtual addressing. Z/vse v5 has an architecture level set (als) that requires ibm system z9 or later. Z/vse...

  • Page 261

    Chapter 7. Software support 233 for the latest information about supported linux distributions on z systems, see this website: http://www.Ibm.Com/systems/z/os/linux/resources/testedplatforms.Html ibm is working with its linux distribution partners to provide further use of selected z13s functions in...

  • Page 262

    234 ibm z13s technical guide table 7-3 shows the minimum support levels for z/os and z/vm. Table 7-3 z13s function minimum support requirements summary (part 1 of 2) function z/os v2 r2 z/os v2 r1 z/os v1r13 z/os v1r12 z/vm v6r3 z/vm v6r2 z13s a y y y y y y maximum processor unit (pus) per system im...

  • Page 263

    Chapter 7. Software support 235 crypto express5s y y y y y y elliptic curve cryptography (ecc) y y y y y y hipersockets i 32 hipersockets y y y y y y hipersockets completion queue y y y n y y hipersockets integration with intraensemble data network (iedn) y y y n n n hipersockets virtual switch brid...

  • Page 264

    236 ibm z13s technical guide t10-dif support by the ficon express8s and ficon express8 features when defined as chpid type fcp n n n n y y global resource serialization (grs) ficon ctc toleration y y y y n n ficon express8 chpid 10km long wave (lx) and short wave (sx) type fc y y y y y y ficon expre...

  • Page 265

    Chapter 7. Software support 237 osa-express5s 1000base-t ethernet chpid type osd (one port per chpid) y y y y y y osa-express5s 1000base-t ethernet chpid type ose y y y y y y osa-express5s 1000base-t ethernet chpid type osm y y y y n k n k osa-express5s 1000base-t ethernet chpid type osn y y y y y y...

  • Page 266

    238 ibm z13s technical guide parallel sysplex and other server time protocol (stp) y y y y - - coupling over infiniband chpid type cib y y y y n n infiniband coupling links 12x at a distance of 150 m (492 ft.) y y y y n n infiniband coupling links 1x at an unrepeated distance of 10 km (6.2 miles) y ...

  • Page 267

    Chapter 7. Software support 239 table 7-4 shows the minimum support levels for z/vse, z/tpf, and linux on z systems. Table 7-4 z13 functions minimum support requirements summary (part 2 of 2) function z/vse v6r1 z/vse v5r2 z/vse v5r1 z/tpf v1r1 linux on z systems z13s y y a y a y y maximum pus per s...

  • Page 268

    240 ibm z13s technical guide hipersockets completion queue y y y g n y hipersockets integration with iedn n n n n n hipersockets virtual switch bridge - - - - y j hipersockets network traffic analyzer n n n n y k hipersockets multiple write facility n n n n n hipersockets support of ipv6 y y y n y h...

  • Page 269

    Chapter 7. Software support 241 ficon express8s, ficon express8, ficon express16s 10km lx, and ficon express4 sx support of scsi disks chpid type fcp y y y n y ficon express8s chpid type fc y y y y y ficon express8 chpid type fc y y y y n y n ficon express 16s (chpid type fc) when using ficon or ctc...

  • Page 270

    242 ibm z13s technical guide osa-express5s 1000base-t ethernet chpid type osd (two ports per chpid) y y y y o y r osa-express5s 1000base-t ethernet chpid type osd (one port per chpid) y y y y o y osa-express5s 1000base-t ethernet chpid type ose y y y n n osa-express5s 1000base-t ethernet chpid type ...

  • Page 271: 7.3  Support By Function

    Chapter 7. Software support 243 7.3 support by function this section addresses operating system support by function. Only the currently in-support releases are covered. Tables in this section use the following convention: n/a not applicable na not available 7.3.1 single system image a single system ...

  • Page 272

    244 ibm z13s technical guide maximum number of pus per system image table 7-5 lists the maximum number of pus supported by each os image and by special purpose lpars. On the z13s, the image size is restricted by the number of pus available: maximum of 6 cps or 6 cps with 12 ziips maximum of 20 ifls ...

  • Page 273

    Chapter 7. Software support 245 (operlog). It looks for unusual messages, unusual message patterns that typical monitoring systems miss, and unique messages that might indicate system health issues. The ibm zaware virtual appliance requires the monitored clients to run z/os v1r13 with ptfs or later....

  • Page 274

    246 ibm z13s technical guide cics/ts v2r3 and later. Db2 udb for z/os version 8 and later. Ims version 8 and later. Ziip assisted hipersockets for large messages. The functioning of a ziip is transparent to application programs. In z13s, the ziip processor is designed to run in smt mode, with up to ...

  • Page 275

    Chapter 7. Software support 247 7.3.4 maximum main storage size table 7-6 lists the maximum amount of main storage that is supported by current operating systems. A maximum of 4 tb of main storage can be defined for an lpar on a z13s. If an i/o drawer is present (as carry-forward), the lpar maximum ...

  • Page 276

    248 ibm z13s technical guide partition’s flash express storage is isolated like the main storage, and sees only its own space in the flash storage space. Flash express provides 1.4 tb of storage per feature pair. Up to four pairs can be installed, for a total of 5.6 tb. All paging data can easily be...

  • Page 277

    Chapter 7. Software support 249 7.3.6 z enterprise data compression express the growth of data that must be captured, transferred, and stored for a long time is unrelenting. Software-implemented compression algorithms are costly in terms of processor resources, and storage costs are not negligible e...

  • Page 278

    250 ibm z13s technical guide communications over remote direct memory access (smc-r) in z/os v2r1 or later. It is transparent to applications and can be used for lpar-to-lpar communication on a single z13s or for server-to-server communication in a multiple cpc environment. Z/os v2r1 or later with p...

  • Page 279

    Chapter 7. Software support 251 table 7-11 lists the support requirements for 2-gb large pages. Table 7-11 minimum support requirements for 2-gb large pages 7.3.9 hardware decimal floating point industry support for decimal floating point is growing, with ibm leading the open standard definition. Ex...

  • Page 280

    252 ibm z13s technical guide 7.3.11 separate lpar management of pus the z13s uses separate pu pools for each optional pu type. The separate management of pu types enhances and simplifies capacity planning and management of the configured lpars and their associated processor resources. Table 7-14 lis...

  • Page 281

    Chapter 7. Software support 253 table 7-15 lists the minimum operating system level that is required on z13s. Table 7-15 minimum support requirements for lpar physical capacity limit enforcement 7.3.14 capacity provisioning manager the provisioning architecture enables clients to better control the ...

  • Page 282

    254 ibm z13s technical guide the new resources are immediately made available to the operating system and, in the case of z/vm, to its guests. The dynamic pu add function is not supported for zaware-mode lpars. 7.3.16 hiperdispatch the hiperdispatch=yes/no parameter in the ieaoptxx member of sys1.Pa...

  • Page 283

    Chapter 7. Software support 255 the pr/sm in the system z9 ec to zec12 servers attempts to assign all logical processors to one book, packed into pu chips of that book in cooperation with operating system hiperdispatch optimize shared cache usage. The pr/sm in the z13s seeks to assign all logical pr...

  • Page 284

    256 ibm z13s technical guide (64 x 1024) -1=65535 table 7-19 lists the minimum operating system level that is required on the z13s. Table 7-19 minimum support requirements for 63.75-k subchannels 7.3.18 multiple subchannel sets multiple subchannel sets (mss), first introduced in z9 ec, provide a mec...

  • Page 285

    Chapter 7. Software support 257 table 7-21 lists the minimum operating systems level that is required on the z13s. Table 7-21 minimum software requirement for four subchannel sets 7.3.20 ipl from an alternative subchannel set z13s supports ipl from subchannel set 1 (ss1), or subchannel set 2 (ss2) i...

  • Page 286

    258 ibm z13s technical guide 7.3.23 hipersockets integration with the intraensemble data network the hipersockets integration with the intraensemble data network (iedn) is available on z13, z13s, zec12, zbc12, z196, and z114 servers. Hipersockets integration with the iedn combines the hipersockets n...

  • Page 287

    Chapter 7. Software support 259 7.3.25 hipersockets multiple write facility the hipersockets multiple write facility allows the streaming of bulk data over a hipersockets link between two lpars. Multiple output buffers are supported on a single signal adapter (siga) write instruction. The key advant...

  • Page 288

    260 ibm z13s technical guide table 7-28 lists the minimum support requirements for hipersockets layer 2. Table 7-28 minimum support requirements for hipersockets layer 2 7.3.28 hipersockets network traffic analyzer for linux on z systems hipersockets network traffic analyzer (hs nta), introduced wit...

  • Page 289

    Chapter 7. Software support 261 7.3.30 ficon express8s the ficon express8s feature is exclusively installed in the peripheral component interconnect express (pcie) i/o drawer. It provides a link rate of 8 gbps, with auto negotiation to 4 or 2 gbps for compatibility with previous devices and investme...

  • Page 290

    262 ibm z13s technical guide 7.3.31 ficon express8 the ficon express8 features provide a link rate of 8 gbps, with auto-negotiation to 4 gbps or 2 gbps for compatibility with previous devices and investment protection. Both 10 km (6.2 miles) lx and sx connections are offered (in a feature, all conne...

  • Page 291

    Chapter 7. Software support 263 7.3.32 z/os discovery and auto-configuration z/os discovery and auto-configuration (zdac) is designed to automatically run a number of i/o configuration definition tasks for new and changed disk and tape controllers that are connected to a switch or director, when att...

  • Page 292

    264 ibm z13s technical guide zdac applies to all ficon features that are supported on z13s when configured as chpid type fc. Table 7-32 lists the minimum support requirements for zdac. Table 7-32 minimum support requirements for zdac 7.3.33 high-performance ficon zhpf, first provided on system z10, ...

  • Page 293

    Chapter 7. Software support 265 support is also added for the d ios,zhpf system command to indicate whether zhpf is enabled, disabled, or not supported on the server. Similar to the existing ficon channel architecture, the application or access method provides the channel program (ccws). The way tha...

  • Page 294

    266 ibm z13s technical guide 7.3.35 32 k subchannels for the ficon express16s to help facilitate growth and continue to enable server consolidation, the z13s supports up to 32 k subchannels per ficon express16s channel (chpid). More devices can be defined per ficon channel, which includes primary, s...

  • Page 295

    Chapter 7. Software support 267 port type (n_port_id) fc-4 types that are supported classes of service that are supported by the channel the platform and name server registration service are defined in the fc-gs-4 standard. 7.3.38 ficon link incident reporting ficon link incident reporting allows an...

  • Page 296

    268 ibm z13s technical guide 7.3.41 osa-express5s 10-gigabit ethernet lr and sr the osa-express5s 10-gigabit ethernet feature, introduced with the zec12 and zbc12, is installed exclusively in the pcie i/o drawer. Each feature has one port, which is defined as either chpid type osd or osx. Chpid type...

  • Page 297

    Chapter 7. Software support 269 operating system support is required to recognize and use the second port on the osa-express5s gigabit ethernet feature. Table 7-39 lists the minimum support requirements for osa-express5s gigabit ethernet lx and sx. Table 7-39 minimum support requirements for osa-exp...

  • Page 298

    270 ibm z13s technical guide table 7-40 lists the minimum support requirements for osa-express5s 1000base-t. Table 7-40 minimum support requirements for osa-express5s 1000base-t ethernet 7.3.44 osa-express4s 10-gigabit ethernet lr and sr the osa-express4s 10-gigabit ethernet feature, introduced with...

  • Page 299

    Chapter 7. Software support 271 table 7-41 lists the minimum support requirements for osa-express4s 10-gigabit ethernet lr and sr features. Table 7-41 minimum support requirements for osa-express4s 10-gigabit ethernet lr and sr 7.3.45 osa-express4s gigabit ethernet lx and sx the osa-express4s gigabi...

  • Page 300

    272 ibm z13s technical guide 7.3.46 osa-express4s 1000base-t ethernet the osa-express4s 1000base-t ethernet feature is installed exclusively in the pcie i/o drawer. Each feature has one pcie adapter and two ports. The two ports share a chpid, which is defined as osc, osd, ose, osm, or osn. The ports...

  • Page 301

    Chapter 7. Software support 273 7.3.47 open systems adapter for ibm zaware the ibm zaware server requires connections to the graphical user interface (gui) browser and z/os monitored clients. An osa channel is the most logical choice for allowing gui browser connections to the server. By using this ...

  • Page 302

    274 ibm z13s technical guide 7.3.49 intranode management network the inmn is one of the ensemble’s two private and secure internal networks. The inmn is used by the unified resource manager functions. The inmn is a private and physically isolated 1000base-t ethernet internal platform management netw...

  • Page 303

    Chapter 7. Software support 275 for ccl planning information, see ibm communication controller for linux on system z v1.2.1 implementation guide, sg24-7223. For the most recent ccl information, see this website: http://www.Ibm.Com/software/network/ccl/ cdlc, when used with ccl, emulates selected fun...

  • Page 304

    276 ibm z13s technical guide 7.3.53 vlan management enhancements table 7-45 lists the minimum support requirements for vlan management enhancements for the osa-express5s and osa-express4s features (chpid type osd). Table 7-45 minimum support requirements for vlan management enhancements 7.3.54 garp ...

  • Page 305

    Chapter 7. Software support 277 a primary objective of iwq is to provide improved performance for business-critical interactive workloads by reducing contention that is created by other types of workloads. The following types of z/os workloads are identified and assigned to unique input queues: z/os...

  • Page 306

    278 ibm z13s technical guide 7.3.58 link aggregation support for z/vm link aggregation (ieee 802.3ad) that is controlled by the z/vm virtual switch (vswitch) allows the dedication of an osa-express5s or osa-express4s port to the z/vm operating system. The port must be part of an aggregated group tha...

  • Page 307

    Chapter 7. Software support 279 7.3.62 qdio optimized latency mode qdio optimized latency mode (olm) can help improve performance for applications that have a critical requirement to minimize response times for inbound and outbound data. Olm optimizes the interrupt processing in the following manner...

  • Page 308

    280 ibm z13s technical guide 7.3.64 osa-express5s and osa-express4s checksum offload osa-express5s and osa-express4s features, when configured as chpid type osd, provide checksum offload for several types of traffic, as indicated in table 7-51. Table 7-51 minimum support requirements for osa-express...

  • Page 309

    Chapter 7. Software support 281 in extending the use of adapter interruptions to osd (qdio) channels, the processor utilization to handle a traditional i/o interruption is reduced. This benefits osa-express tcp/ip support in z/vm, z/vse, and linux on z systems. Adapter interruptions apply to all of ...

  • Page 310

    282 ibm z13s technical guide this data can then be forwarded to z/os tools for easier problem determination and resolution. The network traffic analyzer is supported by the osa-express5s and osa-express4s features on z13s when in qdio mode (chpid type osd). It is used by z/os v1r12 and later. 7.3.71...

  • Page 311

    Chapter 7. Software support 283 infiniband coupling links support for hca3-o 3 (12xifb) fanout that supports infiniband coupling links 12x at a maximum distance of 150 meters (492 feet) is listed in table 7-54. Table 7-54 minimum support requirements for coupling links over infiniband infiniband cou...

  • Page 312

    284 ibm z13s technical guide to a thread on a ziip (for eligible workloads in z/os) or an ifl (for z/vm) core in single-thread or smt mode. For more information, see 7.9, “simultaneous multithreading” on page 295. Table 7-57 lists the minimum support requirements for smt. Table 7-57 minimum support ...

  • Page 313: 7.4  Cryptographic Support

    Chapter 7. Software support 285 table 7-59 lists the minimum support requirements for smc-d. Table 7-59 minimum support requirements for smc-d. For more information, see appendix d, “shared memory communications” on page 475. 7.4 cryptographic support z13s provides two major groups of cryptographic ...

  • Page 314

    286 ibm z13s technical guide 7.4.2 crypto express5s support of crypto express5s functions varies by operating system and release. Table 7-61 lists the minimum software requirements for the crypto express5s features when configured as a coprocessor or an accelerator. For more information, see 6.5, “c...

  • Page 315

    Chapter 7. Software support 287 table 7-62 lists the icsf fmids and web-deliverable codes for z/os v1r12 through v2r2. Later fmids include the functions of previous ones. Table 7-62 z/os icsf fmids icsf fmid z/os web deliverable name supported function hcr7770 v1r12 v1r11 v1r10 cryptographic support...

  • Page 316

    288 ibm z13s technical guide hcr77a1 v2r1 v1r13 v1r12 cryptographic support for z/os v1r13-v2r1 ap configuration simplification kds key utilization statistics dynamic ssm udx reduction and simplification europay-mastercard-visa (emv) enhancements key wrapping and other security enhancements owh/rng ...

  • Page 317: 7.5  Gdps Virtual Appliance

    Chapter 7. Software support 289 7.4.5 icsf migration considerations consider the following if you have installed the cryptographic support for z/os v1r13 – z/os v2r1 web deliverable (fmid hcr77a1) to accommodate the change in the way master keys are processed to determine which coprocessors become a...

  • Page 318

    290 ibm z13s technical guide 7.6.1 general guidelines the ibm z13s introduces the latest z systems technology. Although support is provided by z/os starting with z/os v1r12, use of z13s depends on the z/os release. Z/os.E is not supported on z13s. In general, consider the following guidelines: do no...

  • Page 319

    Chapter 7. Software support 291 using the capacity provisioning function requires these prerequisites: tcp/ip connectivity to observed systems rmf distributed data server must be active cim server must be active security and cim customization capacity provisioning manager customization in addition, ...

  • Page 320

    292 ibm z13s technical guide easily. Ibm zaware provides a simple gui for easy identification of message anomalies, which can facilitate faster problem resolution. Ibm zaware is enhanced as part of z13s general availability to process messages without message ids. This feature allows you to analyze ...

  • Page 321

    Chapter 7. Software support 293 7.8.1 cfcc level 21 cfcc level 21 is delivered on the z13s with driver level 27. Cfcc level 21 introduces the following enhancements: usability enhancement: – enable systems management applications to collect valid coupling facility (cf) lpar information by using z/os...

  • Page 322

    294 ibm z13s technical guide to support an upgrade from one cfcc level to the next, different levels of cfcc can be run concurrently while the coupling facility lpars are running on different servers. Cf lpars that run on the same server share the cfcc level. The cfcc level for z13s servers is cfcc ...

  • Page 323

    Chapter 7. Software support 295 structure size requirements for real memory get larger at initial allocation time to accommodate more control objects that are needed to use flash memory. The cfsizer structure recommendations consider these additional requirements, both for sizing the structure’s fla...

  • Page 324

    296 ibm z13s technical guide when procview core or core,cpu_ok are specified in z/os running in z13s, hiperdispatch is forced to run as enabled. You cannot disable it.The procview statement cannot be changed dynamically, so you must run an ipl after changing it to make the new setting effective. The...

  • Page 325

    Chapter 7. Software support 297 the default in z/vm is multithreading disabled by default. There is no dynamic switching of the smtmode. It can be enabled or disabled only by setting the multithreading statement in the configuration file. After the statement is set, you must run an ipl of the partit...

  • Page 326: 7.11  The Midaw Facility

    298 ibm z13s technical guide z/os v2r1 has support for simd through an spe. The z/vm support for simd support will be delivered for z13/z13s with ptf for apar wm65733 for guest exploitation. Os support includes the following items: enablement of vector registers. Use of vector registers using xl c/c...

  • Page 327

    Chapter 7. Software support 299 the midaw facility provides a more efficient ccw/idaw structure for certain categories of data-chaining i/o operations: midaw can improve ficon performance for extended format data sets. Non-extended data sets can also benefit from midaw. Midaw can improve channel uti...

  • Page 328

    300 ibm z13s technical guide the data address in the initial idaw for example, three idaws are required when these events occur: the orb specifies format-2 idaws with 4 kb blocks. The ccw count field specifies 8 kb. The first idaw designates a location in the middle of a 4 kb block. Ccws with data c...

  • Page 329

    Chapter 7. Software support 301 when the ccw count goes to zero or the last midaw (with the last flag) ends. The combination of the address and count in a midaw cannot cross a page boundary. This configuration means that the largest possible count is 4 k. The maximum data count of all the midaws in ...

  • Page 330: 7.12  Iocp

    302 ibm z13s technical guide users of the executing fixed channel programs in real storage (excpvr) instruction can construct channel programs that contain midaws. However, doing so requires that they construct an iobe with the iobemida bit set. Users of the excp instruction cannot construct channel...

  • Page 331: 7.14  Ickdsf

    Chapter 7. Software support 303 the iocs file contains detailed information for each channel and path assignment, each control unit, and each device in the configuration. 7.13 worldwide port name tool part of the installation of your z13s system is the pre-planning of the san environment. Ibm has a ...

  • Page 332: Support

    304 ibm z13s technical guide 7.15 ibm z bladecenter extension (zbx) model 004 software support zbx model 004 houses two types of blades: power platform specific ibm blades for linux and windows operating systems for information about zbx upgrades, see 8.3.4, “mes upgrades for the zbx” on page 327. 7...

  • Page 333: 7.16  Software Licensing

    Chapter 7. Software support 305 table 7-65 lists the minimum support requirements for datapower sysplex distributor support. Table 7-65 minimum support requirements for datapower sysplex distributor support 7.16 software licensing this section describes the software licensing options that are availa...

  • Page 334

    306 ibm z13s technical guide the ibm international passport advantage agreement can be downloaded from the “learn about software licensing” website: ftp://ftp.Software.Ibm.Com/software/passportadvantage/pa_agreements/pa_agreemen t_international_english.Pdf the remainder of this section describes the...

  • Page 335

    Chapter 7. Software support 307 even by using the soft capping option, the use of the partition can reach its maximum share based on the number of logical processors and weights in the image profile. Only the 4-hour rolling average utilization is tracked, allowing utilization peaks above the defined...

  • Page 336

    308 ibm z13s technical guide 7.16.4 advanced entry workload license charge aewlc was introduced with the z114. They use the measuring and reporting mechanisms, and the existing msu tiers, from the entry workload license charges (ewlc) pricing metric and the midrange workload license charges (mwlc) p...

  • Page 337

    Chapter 7. Software support 309 7.16.7 parallel sysplex license charges pslcs apply to a large range of mainframe servers. The list can be obtained from this website: http://www.Ibm.Com/systems/z/resources/swprice/reference/exhibits/hardware.Html although it can be applied to stand-alone cpcs, the m...

  • Page 338: 7.17  References

    310 ibm z13s technical guide 7.16.9 zbx licensed software the software licensing for the zbx select system x and power7 blades and datapower xi50z follows the same rules as licensing for blades that are installed outside of zbx. Powervm enterprise edition must be licensed for power7 blades at the ti...

  • Page 339: System Upgrades

    © copyright ibm corp. 2016. All rights reserved. 311 chapter 8. System upgrades this chapter provides an overview of ibm z systems z13s upgrade capabilities and procedures, with an emphasis on capacity on demand (cod) offerings. The upgrade offerings to the z13s central processor complex (cpc) have ...

  • Page 340: 8.1  Upgrade Types

    312 ibm z13s technical guide 8.1 upgrade types this section summarizes the types of upgrades available for ibm z13s servers. 8.1.1 overview of upgrade types upgrades can be categorized as described in the following scenario. Permanent and temporary upgrades in various situations, separate types of u...

  • Page 341

    Chapter 8. System upgrades 313 disruptive an upgrade is disruptive when the resources that are added to an operating system (os) image require that the os is recycled to configure the newly added resources. Nondisruptive nondisruptive upgrades do not require that you restart the software that is run...

  • Page 342

    314 ibm z13s technical guide capacity provisioning manager (cpm) as a component of z/os capacity provisioning, cpm monitors business-critical workloads that are running on z/os systems on z13s cpcs. Customer profile this information is on ibm resource link, and contains customer and machine informat...

  • Page 343

    Chapter 8. System upgrades 315 8.1.3 permanent upgrades permanent upgrades can be ordered through an ibm marketing representative, or started by the customer with the ciu on ibm resource link. Permanent upgrades ordered through an ibm representative through a permanent upgrade, you can perform these...

  • Page 344

    316 ibm z13s technical guide activate channels. Activate cryptographic engines. Change specialty engine (recharacterization). Add zbx features. Blades can only be added if blade entitlement is already present. Permanent upgrades initiated through ciu on ibm resource link ordering a permanent upgrade...

  • Page 345: 8.2  Concurrent Upgrades

    Chapter 8. System upgrades 317 8.2 concurrent upgrades concurrent upgrades on z13s servers can provide extra capacity with no server outage. In most cases, with prior planning and os support, a concurrent upgrade is nondisruptive to the os. Given today’s business environment, the benefits of the con...

  • Page 346

    318 ibm z13s technical guide note the following model upgrade information: liccc upgrade: – does not change the server model 2965 n10 because an additional node (3 scms and memory) or cpc drawer is not added. – can change the mci, the capacity setting, or both. Hardware installation upgrade: – can c...

  • Page 347

    Chapter 8. System upgrades 319 lpars might also have to free the pus to be converted, and the oss must have support to configure processors offline or online for the pu conversion to be done non-disruptively. 8.2.2 customer initiated upgrade facility the ciu facility is an ibm online system through ...

  • Page 348

    320 ibm z13s technical guide as part of the setup, the customer provides one resource link id for configuring and placing ciu orders and, if required, a second id as an approver. The ids are then set up for access to the ciu support. The ciu facility is beneficial by enabling upgrades to be ordered ...

  • Page 349

    Chapter 8. System upgrades 321 temporary upgrades are represented in the z13s server by a record . All temporary upgrade records, downloaded from the rsf or installed from portable media, are resident on the se hard disk drive (hdd). At the time of activation, the customer can control everything loc...

  • Page 350

    322 ibm z13s technical guide the software charges can vary according to the license agreement for the individual products. See your ibm software group (swg) representative for exact details. On/off cod can concurrently add processors (cps, icfs, ziips, ifls, and saps), increase the mci, or both, up ...

  • Page 351

    Chapter 8. System upgrades 323 8.3 miscellaneous equipment specification upgrades mes upgrades enable concurrent and permanent capacity growth. Mes upgrades enable the concurrent addition of processors (cps, icfs, ziips, ifls, and saps), memory capacity, i/o ports, hardware, and entitlements to the ...

  • Page 352

    324 ibm z13s technical guide the store system information (stsi) instruction gives more useful and detailed information about the base configuration, and about temporary upgrades. Stsi enables you to more easily resolve billing situations where independent software vendor (isv) products are in use. ...

  • Page 353

    Chapter 8. System upgrades 325 an lpar can dynamically take advantage of a memory upgrade if reserved storage was defined to that lpar. The reserved storage is defined to the lpar as part of the image profile. Reserved memory can be configured online to the lpar by using the lpar dynamic storage rec...

  • Page 354

    326 ibm z13s technical guide figure 8-4 shows the memory configurations and upgrades for a z13s model n20 single cpc drawer. Figure 8-4 memory sizes and upgrades for the single drawer n20 figure 8-5 shows the memory configurations and upgrades for a z13s model n20 with two cpc drawers. Figure 8-5 me...

  • Page 355

    Chapter 8. System upgrades 327 the accurate planning and definition of the target configuration are vital to maximize the value of these features. 8.3.4 mes upgrades for the zbx the mes upgrades for zbx can concurrently add only the following components: blades only if there are any slots available ...

  • Page 356

    328 ibm z13s technical guide and control the zbx model 004. With the zbx model 004, the concept of owning the cpc does not exist. Because the system upgrade is always disruptive, the zbx upgrade will also be a disruptive task. If installing a new build z13s server and planning to take over an existi...

  • Page 357

    Chapter 8. System upgrades 329 maintenance charges are automatically adjusted as a result of a permanent upgrade. Software charges, based on the total capacity of the server on which the software is installed, are adjusted to the new capacity in place after the permanent upgrade is installed. Softwa...

  • Page 358

    330 ibm z13s technical guide figure 8-8 illustrates the process for a permanent upgrade. When the liccc is passed to the remote support facility, you are notified through email that the upgrade is ready to be downloaded. Figure 8-8 ciu-eligible order activation example the major components in the pr...

  • Page 359

    Chapter 8. System upgrades 331 figure 8-9 shows the initial view of the machine profile on resource link. Figure 8-9 machine profile the number of cps, icfs, ziips, ifls, and saps, the memory size, cbu features, unassigned cps, and unassigned ifls on the current configuration are displayed on the le...

  • Page 360

    332 ibm z13s technical guide when the order is available for download, you receive an email that contains an activation number. You can then retrieve the order by using the perform model conversion task from the se, or through soo to the se from an hmc. To retrieve the order, follow these steps: 1. ...

  • Page 361

    Chapter 8. System upgrades 333 8.5.1 overview the capacity for cps is expressed in msus. The capacity for speciality engines is expressed in the number of speciality engines. Capacity tokens are used to limit the resource consumption for all types of processor capacity. Capacity tokens are introduce...

  • Page 362

    334 ibm z13s technical guide 8.5.2 ordering concurrently installing temporary capacity by ordering on/off cod is possible in the following quantities: cp features equal to the msu capacity of installed cps ifl features up to the number of installed ifls icf features up to the number of installed icf...

  • Page 363

    Chapter 8. System upgrades 335 as an example, for a z13s server with capacity identifier d02, you can used the following methods to deliver a capacity upgrade through on/off cod: one option is to add cps of the same capacity setting. With this option, the mci can be changed to a d03, which adds one ...

  • Page 364

    336 ibm z13s technical guide staged orders can have separate pricing. When the order is downloaded and activated, the daily costs are based on the pricing at the time of the order. The staged orders do not have to be installed in order sequence. If a staged order is installed out of sequence, and la...

  • Page 365

    Chapter 8. System upgrades 337 8.5.3 on/off cod testing each on/off cod-enabled z13s server is entitled to one no-charge 24-hour test. The test has no ibm charges that are associated with temporary hardware capacity, ibm software, or ibm maintenance. The test can be used to validate the processes to...

  • Page 366

    338 ibm z13s technical guide 8.5.4 activation and deactivation when a previously ordered on/off cod is retrieved from resource link, it is downloaded and stored on the se hard disk. You can activate the order when the capacity is needed, either manually or through automation. If the on/off cod offer...

  • Page 367

    Chapter 8. System upgrades 339 maintenance the maintenance price is adjusted as a result of an on/off cod activation. Software software parallel sysplex license charge (pslc) customers are billed at the msu level that is represented by the combined permanent and temporary capacity. All pslc products...

  • Page 368

    340 ibm z13s technical guide sysplex-wide data aggregation and propagation occur in the rmf distributed data server (dds). The rmf common information model (cim) providers and associated cim models publish the rmf monitor iii data. The cpm, a function inside z/os, retrieves critical metrics from one...

  • Page 369

    Chapter 8. System upgrades 341 the control over the provisioning infrastructure is run by the cpm through the capacity provisioning domain (cpd), which is controlled by the capacity provisioning policy (cpp). An example of a cpd is shown in figure 8-15. Figure 8-15 a capacity provisioning domain the...

  • Page 370

    342 ibm z13s technical guide confirmation mode: in this mode, cpm processes capacity-provisioning policies and interrogates the installed temporary offering records. Every action that is proposed by the cpm needs to be confirmed by the operator. Autonomic mode: this mode is similar to the confirmati...

  • Page 371

    Chapter 8. System upgrades 343 one be stopped before the inactive one can be activated. This operation decreases the current capacity during the change. The provisioning management routines can interrogate the installed offerings, their content, and the status of the content of the offering. To avoi...

  • Page 372

    344 ibm z13s technical guide fc 0119: 1 cpe capacity unit-ifl fc 0120: 100 cpe capacity unit-ifl fc 0121: 1 cpe capacity unit-icf fc 0122: 100 cpe capacity unit-icf fc 0123: 1 cpe capacity unit-zaap fc 0124: 100 cpe capacity unit-zaap fc 0125: 1 cpe capacity unit-ziip fc 0126: 100 cpe capacity unit-...

  • Page 373: 8.7  Capacity Backup

    Chapter 8. System upgrades 345 8.7 capacity backup cbu provides reserved emergency backup processor capacity for unplanned situations in which capacity is lost in another part of your enterprise, and you want to recover by adding the reserved capacity on a designated z13s server. Cbu is the quick, t...

  • Page 374

    346 ibm z13s technical guide the processors that can be activated by cbu come from the available unassigned pus on any cpc drawer. The maximum number of cbu features that can be ordered is 20. The number of features that can be activated is limited by the number of unused pus on the server. However,...

  • Page 375

    Chapter 8. System upgrades 347 failure to deactivate the cbu feature before the expiration date can cause the system to degrade gracefully back to its original configuration. The system does not deactivate dedicated engines, or the last of in-use shared engines. For detailed instructions, see the z ...

  • Page 376

    348 ibm z13s technical guide the test cbu has a 10-day limit, and must be deactivated in the same way as the real cbu, using the same facility through the se. Failure to deactivate the cbu feature before the expiration date can cause the system to degrade gracefully back to its original configuratio...

  • Page 377: 8.8  Nondisruptive Upgrades

    Chapter 8. System upgrades 349 in addition, two cp cbu features can be used to change the capacity level of permanent cps, and the third cp cbu feature can be used to add a cp, which enables the activation of d03, e03, and f03 through z03. In this example, you are offered 49 possible configurations ...

  • Page 378

    350 ibm z13s technical guide 8.8.1 processors cps, icfs, ziips, ifls, and saps can be concurrently added to a z13s server if unassigned pus are available in any installed cpc drawer. The number of ziips cannot exceed twice the number of cps, plus unassigned cps. If necessary, more lpars can be creat...

  • Page 379

    Chapter 8. System upgrades 351 processor identification the following instructions are used to obtain processor information: the stsi instruction: stsi reports the processor model and model capacity identifier for the base configuration and for any additional configuration changes through temporary ...

  • Page 380

    352 ibm z13s technical guide the mci contains the base capacity, the on/off cod, and the cbu. The mpci and the model permanent capacity rating (mpcr) contain the base capacity of the system, and the mtci and model temporary capacity rating (mtcr) contain the base capacity and the on/off cod. Store c...

  • Page 381

    Chapter 8. System upgrades 353 planning is done in advance. Planning ahead is a key factor for enabling nondisruptive upgrades. The following list describes the major reasons for disruptive upgrades. However, by carefully planning, and by reviewing “suggestions to avoid disruptive upgrades”, you can...

  • Page 382

    354 ibm z13s technical guide one major customer requirement is to eliminate the necessity for a customer authorization connection to the ibm resource link system at the time of activation of any offering. This requirement was first met by the zbc12 and is also available with z13s servers. After the ...

  • Page 383: Serviceability

    © copyright ibm corp. 2016. All rights reserved. 355 chapter 9. Reliability, availability, and serviceability this chapter describes the reliability, availability, and serviceability (ras) features of z13s servers. The z13s design is focused on providing higher availability by reducing planned and u...

  • Page 384: 9.1  The Ras Strategy

    356 ibm z13s technical guide 9.1 the ras strategy the ras strategy is to manage change by learning from previous generations and investing in new ras function to eliminate or minimize all sources of outages. Enhancements to z systems ras designs are implemented on the z13s system through the introdu...

  • Page 385

    Chapter 9. Reliability, availability, and serviceability 357 diagnostics capability that is intended to help you quickly pinpoint problems. This process in turn can help you to more rapidly address service disruptions. Ibm zaware uses analytics to examine z/os and linux on z systems messages to find...

  • Page 386: 9.3  Ras Functions

    358 ibm z13s technical guide 9.3 ras functions unscheduled, scheduled, and planned outages have been addressed for the mainframe family of servers for many years: unscheduled this outage occurs because of an unrecoverable malfunction in a hardware component of the server. Scheduled this outage is ca...

  • Page 387

    Chapter 9. Reliability, availability, and serviceability 359 memory subsystem improvements z13s servers use raim, which is a concept that is known in the disk industry as raid. Raim design detects and recovers from dram, socket, memory channel, or dimm failures. The raim design includes the addition...

  • Page 388

    360 ibm z13s technical guide redundant humidity sensors redundant altimeter sensors corrosion sensors 1 unified support for the zbx the zbx is supported the same as any other feature on z13s servers. Single processor core checkstop and sparing this indicator implies that a processor core has malfunc...

  • Page 389

    Chapter 9. Reliability, availability, and serviceability 361 previous firmware updates, which require an initial machine load (iml) of the z13s server to be activated, can block the ability to run a concurrent driver upgrade. An icon on the support element (se) allows you or your ibm service support...

  • Page 390

    362 ibm z13s technical guide 9.5 ras capability for the hmc and se the hardware management console (hmc) and the se have the following ras capabilities: back up from hmc and se on a scheduled basis, the hmc hard disk drive (hdd) is backed up to the usb flash memory drive, a customer provided ftp ser...

  • Page 391

    Chapter 9. Reliability, availability, and serviceability 363 the manage alternate hmc task allows you to reload internal code onto the alternate hmc to minimize hmc downtime during an upgrade to a new driver level. After the new driver is installed on the alternate hmc, it can be made active by runn...

  • Page 392

    364 ibm z13s technical guide zbx firmware the testing, delivery, installation, and management of the zbx firmware is handled in the same way as z13s servers. The same processes and controls are used. All fixes to the zbx are downloaded to the zbx model 004 internal ses and applied to the zbx. The mc...

  • Page 393

    Chapter 9. Reliability, availability, and serviceability 365 the zbx model 004 is based on the bladecenter and blade hardware offerings that contain ibm certified components. Zbx model 004 bladecenter and blade ras features are extended considerably for ibm z systems servers: hardware redundancy at ...

  • Page 394

    366 ibm z13s technical guide 9.7 considerations for powerha in zbx environment an application that runs on aix can be provided with high availability by using the powerha systemmirror for aix (formerly known as ibm hacmp™). Powerha is easy to configure because it is menu-driven, and provides high av...

  • Page 395

    Chapter 9. Reliability, availability, and serviceability 367 application planning (start/stop/monitoring scripts and operating system, processor, and memory resources) powerha software installation and cluster configuration application integration (integrating storage, networking, and application sc...

  • Page 396

    368 ibm z13s technical guide ibm zaware improves the overall ras capability of z13s servers by providing these advantages: identify when and where to look for a problem drill down to identify the cause of the problem improve problem determination in near real time reduce problem determination effort...

  • Page 397: Environmental Requirements

    © copyright ibm corp. 2016. All rights reserved. 369 chapter 10. Environmental requirements this chapter describes the environmental requirements for ibm z systems servers. It lists the dimensions, weights, power, and cooling requirements as an overview of what is needed to plan for the installation...

  • Page 398

    370 ibm z13s technical guide 10.1 ibm z13s power and cooling z13s servers have a number of options for installation on a raised floor, or on a non-raised floor. Furthermore, the cabling can come into the bottom of the machine, or into the top of the machine. 10.1.1 power and i/o cabling the z13s ser...

  • Page 399

    Chapter 10. Environmental requirements 371 figure 10-2 shows the frame extensions, also called chimneys , that are installed to each corner on the left side of the a frame when the top exit i/o cabling feature (fc 7920) is ordered. The bottom of the chimney is closed with welded sheet metal. Figure ...

  • Page 400

    372 ibm z13s technical guide the power requirements depend on the number of central processor complex (cpc) drawers, and the number of i/o drawers, that are installed. Table 10-1 lists the maximum power requirements for z13s servers. These numbers assume the maximum memory configurations, all drawer...

  • Page 401

    Chapter 10. Environmental requirements 373 10.1.4 balanced power plan ahead a balanced power plan ahead feature is available for future growth, also assuring adequate and balanced power for all possible configurations. With this feature, downtimes for upgrading a system are eliminated by including t...

  • Page 402

    374 ibm z13s technical guide 10.1.7 new environmental class for ibm z13s servers: ashrea class a3 ashrae 2 (american society of heating, refrigerating, and air-conditioning engineers) is an organization devoted to the advancement of indoor-environment-control technology in the heating, ventilation, ...

  • Page 403

    Chapter 10. Environmental requirements 375 10.2.1 weights and dimensions installation can be on a raised floor or a non-raised floor. You can find the most up-to-date details about the installation requirements for z13s servers in z13s installation manual for physical planning, gc28-6953. Table 10-3...

  • Page 404

    376 ibm z13s technical guide 10.3.1 ibm zbx configurations the zbx can have from one to four racks. The racks are shipped separately, and are bolted together at installation time. Each rack can contain up to two bladecenter chassis, and each chassis can contain up to 14 single-wide blades. The numbe...

  • Page 405

    Chapter 10. Environmental requirements 377 the following power cord options are available for the zbx: fc 0531: 4.3 m (14.1 ft.), 60a/208 v, us power cord, single phase. Fc 0532: 4.3 m (14.1 ft.), 63a/230 v, non-us power cord, single phase. Fc 0533: 4.3 m (14.1 ft.), 32a/380 v-415 v, non-us power co...

  • Page 406

    378 ibm z13s technical guide heat released by configurations table 10-6 shows the typical heat that is released by the various zbx solution configurations. Table 10-6 ibm zbx power consumption and heat output optional rear door heat exchanger (fc 0540) for data centers that have limited cooling capa...

  • Page 407

    Chapter 10. Environmental requirements 379 figure 10-4 shows the ibm rear door heat exchanger details. Figure 10-4 rear door heat exchanger (left) and functional diagram the ibm rear door heat exchanger also offers a convenient way to handle hazardous “hot spots”, which might help you lower the tota...

  • Page 408: 10.4  Energy Management

    380 ibm z13s technical guide figure 10-5 top exit support for the zbx height reduction fc 0570 this feature is required if it is necessary to reduce the shipping height for the zbx. Select this feature when it is deemed necessary for delivery clearance purposes. Order it if you have doorways with op...

  • Page 409

    Chapter 10. Environmental requirements 381 through a simple network management protocol (snmp) application programming interface (api), energy information is available to, for example, active energy manager, which is a plug-in of ibm systems director. See 10.4.4, “ibm systems director active energy ...

  • Page 410

    382 ibm z13s technical guide this function enables operations personnel with no z systems knowledge to query the maximum power draw of the system, as shown in figure 10-6. The implementation helps to avoid capping enforcement through dynamic capacity reduction. The customer controls are implemented ...

  • Page 411

    Chapter 10. Environmental requirements 383 the monitors dashboard of the hmc enables you to display power and other environmental data. It also enables you to start a dashboard histogram display, where you can trend a particular value of interest, such as the power consumption of a blade or the ambi...

  • Page 412

    384 ibm z13s technical guide ibm systems director active energy manager provides customers with the intelligence necessary to effectively manage power consumption in the data center. Active energy manager, which is an extension to ibm director systems management software, enables you to meter actual...

  • Page 413

    Chapter 10. Environmental requirements 385 when capping is enabled for a z13s server, this capping level is used as a threshold for a warning message that informs you that the z13s server went above the set cap level. Being under the limit of the cap level is equal to the maximum potential power val...

  • Page 414

    386 ibm z13s technical guide.

  • Page 415: Hardware Management Console

    © copyright ibm corp. 2016. All rights reserved. 387 chapter 11. Hardware management console and support elements the hardware management console (hmc) supports many functions and tasks to extend the management capabilities of z13s servers. When tasks are performed on the hmc, the commands are sent ...

  • Page 416

    388 ibm z13s technical guide 11.1 introduction to the hmc and se the hmc is a stand-alone computer that runs a set of management applications. The hmc is a closed system, which means that no other applications can be installed on it. The hmc is used to set up, manage, monitor, and operate one or mor...

  • Page 417

    Chapter 11. Hardware management console and support elements 389 11.2.1 driver level 27 hmc and se enhancements and changes the hmc and se with driver 27 has several enhancements and changes for z13s servers: rack-mounted hmc a new rack-mounted hmc (fc 0094) is available. For more information, see 1...

  • Page 418

    390 ibm z13s technical guide se and hmc against unauthorized booting from removable media. An ibm service support representative (ibm ssr) might perform the following tasks, which require booting from removable media: – engineering change (ec) upgrade – save or restore of save/restore data – hard di...

  • Page 419

    Chapter 11. Hardware management console and support elements 391 enabling diagnostic sampling for a specific lpar is shown in figure 11-2. Figure 11-2 change lpar security zbx firmware management. As a result of a mes upgrade from a zbx model 002 or zbx model 003, the zbx model 004 becomes a stand-a...

  • Page 420

    392 ibm z13s technical guide after you get to the ibm knowledge center, click z systems, and then click z13s. 11.2.2 rack-mounted hmc feature codes (fc) 0094 and fc 0096 provide a rack-mounted hmc. They cannot be ordered as a feature code for zec12, zbc12, or previous z systems servers. The hmc is a...

  • Page 421

    Chapter 11. Hardware management console and support elements 393 11.2.3 new support elements the ses are no longer two notebooks in one z13s server. They are now two servers that are installed at the top of the cpc frame. They are managed by the keyboard, pointing device, and display that are mounte...

  • Page 422

    394 ibm z13s technical guide figure 11-5 shows the information that is required to configure your backup ftp server. Figure 11-5 configure backup ftp server backup of hmcs a backup of the hmc can be performed to the following media a ufd an ftp server a ufd and ftp server figure 11-6 shows the desti...

  • Page 423

    Chapter 11. Hardware management console and support elements 395 it is no longer possible to do the primary se backup to a ufd of a z13s or zbx model 004 se. Table 11-1 shows the se backup options for external media. Table 11-1 se backup options figure 11-7 shows examples of the different destinatio...

  • Page 424

    396 ibm z13s technical guide scheduled operations for the backup of hmcs and ses the scheduled operation task with the new backup options for hmc is changed, as shown in figure 11-8. Figure 11-8 scheduled operation for hmc backup the scheduled operation task with the new backup options for the ses i...

  • Page 425

    Chapter 11. Hardware management console and support elements 397 table 11-2 shows a summary of the se drivers and versions that are supported by the new hmc version 2.13.1 (driver 27). Table 11-2 z13s hmc at driver 27: z systems support summary 11.2.6 hmc feature codes hmcs older than fc 0091 are no...

  • Page 426

    398 ibm z13s technical guide for more information, see 11.2.2, “rack-mounted hmc” on page 392. 11.2.7 tree style user interface and classic style user interface two user interface styles are provided with an hmc. The tree style user interface (default) uses a hierarchical model that is common in new...

  • Page 427

    Chapter 11. Hardware management console and support elements 399 j04 port on the ses. Other z systems servers and hmcs also can be connected to the switch. To provide redundancy, install two ethernet switches. Only the switch (and not the hmc directly) can be connected to the ses. Figure 11-10 shows...

  • Page 428

    400 ibm z13s technical guide 11.3.1 network planning for the hmc and se plan the hmc and se network connectivity carefully to allow for current and future use. Many of the z systems capabilities benefit from the network connectivity options that are available. For example, these functions, which dep...

  • Page 429

    Chapter 11. Hardware management console and support elements 401 these examples are shown in figure 11-12. Figure 11-12 hmc connectivity examples for more information, see the following resources: the hmc and se (version 2.13.1) console help system, or go to the ibm knowledge center at the following...

  • Page 430

    402 ibm z13s technical guide 11.3.3 rsf is broadband-only rsf through a modem is not supported on the z13s hmc. Broadband is needed for hardware problem reporting and service. For more information, see 11.4, “remote support facility” on page 403. 11.3.4 tcp/ip version 6 on the hmc and se the hmc and...

  • Page 431

    Chapter 11. Hardware management console and support elements 403 ipv6 addresses are easily identified. A fully qualified ipv6 address has 16 bytes. It is written as eight 16-bit hexadecimal blocks that are separated by colons, as shown in the following example: 2001:0db8:0000:0000:0202:b3ff:fe1e:832...

  • Page 432

    404 ibm z13s technical guide 11.4.2 rsf connections to ibm and enhanced ibm service support system if the hmc and se are at driver 27, the driver uses a new remote infrastructure at ibm when the hmc connects through rsf for certain tasks. Check your network infrastructure settings to ensure that thi...

  • Page 433

    Chapter 11. Hardware management console and support elements 405 existing customer-installed firewalls between the remote hmc and its managed objects must permit communications between the hmc and the se. For service and support, the remote hmc also requires connectivity to ibm, or to another hmc wi...

  • Page 434

    406 ibm z13s technical guide the hardware messages task displays hardware-related messages at the cpc level, lpar level, or se level. It also displays hardware messages that relate to the hmc itself. 11.5.2 logical partition management use the hmc to define lpar properties, such as the number of pro...

  • Page 435

    Chapter 11. Hardware management console and support elements 407 a group name, processor capping value, and partition membership are specified at hw console: set an absolute capacity cap by cpu type on a group of lpars. Allows each of the partitions to consume capacity up to their individual limits ...

  • Page 436

    408 ibm z13s technical guide updates to x3270 support the configure 3270 emulators task on the hmc and tke consoles was enhanced with driver 15 to verify the authenticity of the certificate that is returned by the 3270 server when a secure and encrypted ssl connection is established to an ibm host. ...

  • Page 437

    Chapter 11. Hardware management console and support elements 409 11.5.4 hmc and se microcode the microcode for the hmc, se, cpc, and zbx is included in the driver/version. The hmc provides the management of the driver upgrade through enhanced driver maintenance (edm). Edm also provides the installat...

  • Page 438

    410 ibm z13s technical guide figure 11-17 shows how the driver, bundle, ec stream, mcl, and mcfs interact with each other. Figure 11-17 microcode terms and interaction.

  • Page 439

    Chapter 11. Hardware management console and support elements 411 microcode installation by mcl bundle target a bundle is a set of mcls that are grouped during testing and released as a group on the same date. You can install an mcl to a specific target bundle level. The system information window is ...

  • Page 440

    412 ibm z13s technical guide monitor task group the monitor task group on the hmc and se includes monitoring-related tasks for the z13s server, as shown in figure 11-19. Figure 11-19 hmc monitor task group the monitors dashboard task the monitors dashboard task supersedes the system activity display...

  • Page 441

    Chapter 11. Hardware management console and support elements 413 figure 11-20 shows an example of the monitors dashboard task. Figure 11-20 monitors dashboard task starting with driver 27, you can display the activity for an lpar by processor type, as shown in figure 11-21. Figure 11-21 display the ...

  • Page 442

    414 ibm z13s technical guide the crypto utilization percentage is displayed on the monitors dashboard according to the physical channel id (pchid) number. The associated crypto number (adjunct processor number) for this pchid is also shown in the table. It provides information about the usage rate o...

  • Page 443

    Chapter 11. Hardware management console and support elements 415 environmental efficiency statistics task the environmental efficiency statistics task (figure 11-25) is part of the monitor task group. It provides historical power consumption and thermal information for the zenterprise cpc, and is av...

  • Page 444

    416 ibm z13s technical guide hmc/se version 2.13.1 provides the following cod information: – millions of service units (msu) and processor tokens – last activation time – pending resources that are shown by processor type instead of just a total count – option to show the details of installed and st...

  • Page 445

    Chapter 11. Hardware management console and support elements 417 adjust time by up to plus or minus 60 seconds. Schedule changes to the offsets listed. Stp can automatically schedule daylight saving time, based on the selected time zone. Monitor the status of the ctn. Monitor the status of the coupl...

  • Page 446

    418 ibm z13s technical guide as shown in figure 11-26, the ntp option is the recommended option, if an ntp server is available. If an ntp server is not available for this hmc, any defined cpc se can be selected after you select selected cpcs. Figure 11-26 customize console date and time.

  • Page 447

    Chapter 11. Hardware management console and support elements 419 the timing network window now includes the next scheduled daylight saving time change and the next leap second adjustment, as shown in figure 11-27. The schedules that are shown are the ones for the next dls change (either given per au...

  • Page 448

    420 ibm z13s technical guide requirements ecar is only available on z13 driver 27 and z13s servers. In a mixed environment with previous generation machines, you should define a z13 or z13s server as the pts and cts. For more planning and setup information, see the following publications: server tim...

  • Page 449

    Chapter 11. Hardware management console and support elements 421 ntp symmetric key and autokey authentication with symmetric key and autokey authentication, the highest level of ntp security is available. Hmc level 2.12.0 and later provide windows that accept and generate key information to be confi...

  • Page 450

    422 ibm z13s technical guide time coordination for zbx components ntp clients that run on blades in the zbx can synchronize their time to the se battery operated clock (boc). The se boc is synchronized to the z13s time-of-day (tod) clock every hour. This process allows the se clock to maintain a tim...

  • Page 451

    Chapter 11. Hardware management console and support elements 423 full log offload and delta log offload (since the last offload request) are provided. Offloading to removable media and to remote locations by ftp is available. The offloading can be manually started by the new audit and log management...

  • Page 452

    424 ibm z13s technical guide 11.5.11 system input/output configuration analyzer on the se and hmc the system input/output configuration analyzer task supports the system i/o configuration function. The information that is needed to manage a system’s i/o configuration must be obtained from many separ...

  • Page 453

    Chapter 11. Hardware management console and support elements 425 11.5.13 cryptographic support this section lists the cryptographic management and control functions that are available in the hmc and the se. Cryptographic hardware z13s servers include both standard cryptographic hardware and optional...

  • Page 454

    426 ibm z13s technical guide figure 11-30 shows an example of the cryptographic configuration window. Figure 11-30 cryptographic configuration window the usage domain zeroize task is provided to clear the appropriate partition crypto keys for a usage domain when you remove a crypto card from a parti...

  • Page 455

    Chapter 11. Hardware management console and support elements 427 setting up is a disruptive action. The selection of dpm mode of operation is done by using a function called enable dynamic partition manager under the se cpc configuration menu as shown in figure 11-31. Figure 11-31 enabling dynamic p...

  • Page 456: 11.6  Hmc In An Ensemble

    428 ibm z13s technical guide after the cpc is restarted and you log on to the hmc that has this cpc defined, the hmc shows another welcome window as shown in figure 11-32. Figure 11-32 hmc welcome window (cpc in dpm mode) new lpars can be added by selecting get started. More information can be found...

  • Page 457

    Chapter 11. Hardware management console and support elements 429 figure 11-33 shows the unified resource manager functions and suites. Figure 11-33 unified resource manager functions and suites overview unified resource manager provides the following functions: hypervisor management provides tasks f...

  • Page 458

    430 ibm z13s technical guide performance management provides a global performance view of all the virtual servers that support workloads that are deployed in an ensemble. The virtual server workload performance goal is like a simplified z/os workload manager (wlm) policy: – you can define, monitor, ...

  • Page 459

    Chapter 11. Hardware management console and support elements 431 automate/advanced management suite advanced management functions for ibm system x blades delivers a workload definition and performance policy monitoring and reporting. The automate function adds goal-oriented resource monitoring manag...

  • Page 460

    432 ibm z13s technical guide 11.6.2 ensemble definition and management the ensemble starts with a pair of hmcs that are designated as the primary and alternate hmcs, and are assigned an ensemble identity. The zenterprise cpcs and zbxs are then added to the ensemble through an explicit action at the ...

  • Page 461

    Chapter 11. Hardware management console and support elements 433 the following tasks show ensemble-related data on another hmc: scheduled operations: displays ensemble-introduced scheduled operations, but you can only view these scheduled operations. User role: shows ensemble tasks. You can modify a...

  • Page 462

    434 ibm z13s technical guide 11.6.3 hmc availability the hmc is attached to the same lan as the server’s and zbx model 004 ses. This lan is referred to as the customer managed management network . The hmc communicates with each cpc and with each zbx model 004 se. If the z13s node is defined as a mem...

  • Page 463

    Chapter 11. Hardware management console and support elements 435 figure 11-34 depicts an ensemble with a zec12, a z13s, and a stand-alone zbx servers that are managed by the unified resource manager in the primary and alternate hmcs. Figure 11-34 ensemble example with primary and alternate hmcs for ...

  • Page 464

    436 ibm z13s technical guide.

  • Page 465: Performance

    © copyright ibm corp. 2016. All rights reserved. 437 chapter 12. Performance this chapter describes the performance considerations for ibm z13s. This chapter includes the following sections: performance information lspr workload suite fundamental components of workload capacity performance relative ...

  • Page 466

    438 ibm z13s technical guide 12.1 performance information the ibm z13s model z06 is designed to offer approximately 51% more capacity and 8 times the amount of memory than the ibm zenterprise bc12 (zbc12) model z06 system. Uniprocessor performance also has increased. A z13s model z01 offers, on aver...

  • Page 467: 12.2  Lspr Workload Suite

    Chapter 12. Performance 439 12.2 lspr workload suite historically, lspr capacity tables, including pure workloads and mixes, have been identified with application names or a software characteristic such as: cics information management system (ims) traditional online transaction processing workload (...

  • Page 468

    440 ibm z13s technical guide however, the path length that is associated with the operating system or subsystem might vary based on a number of factors: competition with other tasks in the system for shared resources. As the total number of tasks grows, more instructions are needed to manage the res...

  • Page 469

    Chapter 12. Performance 441 figure 12-2 shows a memory nest in a z13s single cpc drawer system with two nodes. Figure 12-2 memory hierarchy on the z13s one cpc drawer system (two nodes) workload capacity performance is sensitive to how deep into the memory hierarchy the processor must go to retrieve...

  • Page 470

    442 ibm z13s technical guide rni reflects the distribution and latency of sourcing data from shared caches and memory, as shown in figure 12-3. Figure 12-3 relative nest intensity many factors influence the performance of a workload. However, usually what these factors are influencing is the rni of ...

  • Page 471

    Chapter 12. Performance 443 application-owning regions (aors) or batch initiators) that are needed to support a workload. This factor always has existed, but its sensitivity is higher with the current high frequency microprocessors. Spreading the same workload over more address spaces than necessary...

  • Page 472

    444 ibm z13s technical guide 12.6 relating production workloads to lspr workloads historically, a number of techniques were used to match production workloads to lspr workloads: application name (a client running cics can use the cics lspr workload) application type (create a mix of the lspr online ...

  • Page 473

    Chapter 12. Performance 445 processor hardware. These past techniques approximated the hardware characteristics that were not available through software performance reporting tools. Beginning with the z10 processor, the hardware characteristics can now be measured by using cpu mf (smf 113) counters ...

  • Page 474

    446 ibm z13s technical guide remember both the workload characteristics and lpar configuration. For these reasons, when you plan capacity, zpcr and involving ibm technical support are recommended. 12.7.1 main performance improvement drivers with z13s the z13s is designed to deliver new levels of per...

  • Page 475

    Chapter 12. Performance 447 clock frequency at 4.3 ghz ibm cmos 14s0 22 nm soi technology with ibm edram technology the z13s design has the following enhancements as compared with the zbc12: increased total number of pus that are available on the system, from 18 to 26, and number of characterizable ...

  • Page 476

    448 ibm z13s technical guide.

  • Page 477: Ibm Z Appliance Container

    © copyright ibm corp. 2016. All rights reserved. 449 appendix a. Ibm z appliance container infrastructure this appendix introduces the ibm z appliance container infrastructure (zaci) framework available on z13s and z13 driver level 27 servers. It briefly describes the reason why ibm has created the ...

  • Page 478: A.1  What Is Zaci?

    450 ibm z13s technical guide a.1 what is zaci? An appliance is an integration of operating system, middleware, and software components that work autonomously and provide core services and infrastructure that focus on consumability and security. Appliances can be implemented as firmware or software, ...

  • Page 479

    Appendix a. Ibm z appliance container infrastructure 451 figure a-1 shows the basic outline of the zaci framework. Figure a-1 zaci framework basic outline 12.7.2 example: deploying ibm zaware the new type of lpar called zaci will be used to deploy the ibm zaware server application. The ibm zaware lo...

  • Page 480

    452 ibm z13s technical guide figure a-2 shows the new lpar type. Figure a-2 ibm zaware image profile based on zaci existing ibm zaware lpars will be automatically converted during enhanced driver maintenance from driver 22 to driver 27. No reconfiguration of ibm zaware is required. A new icon will i...

  • Page 481: Ibm Z Systems Advanced

    © copyright ibm corp. 2016. All rights reserved. 453 appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) this appendix introduces ibm z systems advanced workload analysis reporter (ibm zaware), which was first available with ibm zenterprise zec12. This feature is designed to o...

  • Page 482

    454 ibm z13s technical guide b.1 troubleshooting in complex it environments in a 24x7 operating environment, a system problem or incident can drive up operations costs and disrupt service to clients for hours or even days. Current it environments cannot afford recurring problems or outages that take...

  • Page 483: B.2  Introducing Ibm Zaware

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 455 b.2 introducing ibm zaware ibm zaware version 2 is an integrated expert solution that contains sophisticated analytics, ibm insight into the problem domain, and web-browser-based visualization. It is an adaptive analytics...

  • Page 484

    456 ibm z13s technical guide ibm zaware and tivoli® service management can be integrated by using the ibm zaware api to provide the following capabilities: provide visibility into ibm zaware anomalies by using event management improve mean time to repair (mttr) through integration with existing prob...

  • Page 485

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 457 ibm zaware delivers sophisticated detection and diagnostic capabilities that identify when and where to look for a problem. The cause of the anomalies can be difficult to determine. High-speed analytics on large quantitie...

  • Page 486

    458 ibm z13s technical guide you can use ibm zaware along with problem diagnosis solutions that are included in z/os with any large and complex z/os installation with mission-critical applications and middleware. B.3 understanding ibm zaware technology ibm zaware creates a base model of system behav...

  • Page 487

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 459 figure b-3 depicts the basic components of an ibm zaware environment. Figure b-3 basic components of the ibm zaware environment ibm zaware runs in an independent lpar as firmware. Ibm zaware has the following characterist...

  • Page 488

    460 ibm z13s technical guide figure b-4 shows ibm zaware image profile on the hardware management console (hmc). Ibm zaware is displayed on the general tab of the image profile. Figure b-4 hmc image profile for an ibm zaware lpar figure b-5 shows the tab of the image profile on the hmc. The setup in...

  • Page 489

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 461 the following settings are configured in the ibm zaware image profile: host name: a unique host name for the lpar master user id and password: used after the lpar is initialized by the workstation through the intranet to ...

  • Page 490

    462 ibm z13s technical guide figure b-7 shows a sample bar score view. Figure b-7 ibm zaware bar score with intervals the analytics create a statistical model of the normal message traffic that is generated by each monitored client (system or group of systems, such as z/os or linux on z systems). Th...

  • Page 491

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 463 ibm zaware detects conditions that typical monitoring systems miss because of these challenges: message suppression (message too common): common messages are useful for long-term health issues. Uniqueness (message not com...

  • Page 492

    464 ibm z13s technical guide the first iteration of this work requires you to mark each message to be ignored on a per system basis. That is, for each message that you want to ignore, you must mark that particular message on each system for which ibm zaware is to ignore the message. You can choose f...

  • Page 493

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 465 ibm zaware feature-related definitions are listed in table b-2. Table b-2 ibm zaware feature code definitions 12.7.3 feature on demand (fod) feature on demand is a centralized way to flexibly entitle features and function...

  • Page 494

    466 ibm z13s technical guide figure b-8 shows the window for ibm zaware feature on demand status and value shown under the perform model conversion, feature on demand manage function. Figure b-8 feature on demand window for ibm zaware feature only one fod liccc record can be installed or staged at a...

  • Page 495

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 467 monitored clients. Fc 0101 and fc 0102 are mutually exclusive. Therefore, if you have one, you cannot have the other. In addition, in most cases, the number of fc 0102 features on dr must match the number of fc 0101 featu...

  • Page 496

    468 ibm z13s technical guide ibm zaware monitored client requirements ibm zaware monitored clients can be in the same cpc as the ibm zaware host system or in different cpcs. They can be in the same site or multiple sites. Distance between the ibm zaware host systems and monitored clients can be up t...

  • Page 497

    Appendix b. Ibm z systems advanced workload analysis reporter (ibm zaware) 469 d. Optional: authorize users or groups to access the ibm zaware gui. E. Optional: modify the configuration values that control the ibm zaware analytics operation. F. Configure a network connection for each z/os and linux ...

  • Page 498

    470 ibm z13s technical guide.

  • Page 499: Channel Options

    © copyright ibm corp. 2016. All rights reserved. 471 appendix c. Channel options this appendix describes all channel attributes, the required cable types, the maximum unrepeated distance, and the bit rate for z13s servers. This appendix includes the following sections: c.1, “channel options supporte...

  • Page 500

    472 ibm z13s technical guide c.1 channel options supported on z13s servers for all optical links, the connector type is lc duplex, except for the 12xinfiniband (ifb) and the integrated coupling adapter (ica sr) connection, which are established with multifiber push-on (mpo) connectors. The mpo conne...

  • Page 501

    Appendix c. Channel options 473 c.2 maximum unrepeated distance for ficon sx features table c-2 shows the maximum unrepeated distances for ficon sx features. Table c-2 maximum unrepeated distance for ficon sx features 10gbe remote direct memory access (rdma) over converged ethernet (roce) express 04...

  • Page 502

    474 ibm z13s technical guide.

  • Page 503: Shared Memory

    © copyright ibm corp. 2016. All rights reserved. 475 appendix d. Shared memory communications this appendix briefly describes the optional shared memory communications (smc) functionality implemented on ibm z systems servers as shared memory communications - remote direct memory access (smc-r) and t...

  • Page 504

    476 ibm z13s technical guide d.1 shared memory communications overview the volume of data being generated and transmitted by technologies driven by cloud, mobile, analytics, and social computing applications is growing. This increases the pressure on business it organizations to be able to provide f...

  • Page 505

    Appendix d. Shared memory communications 477 a reliable “lossless” ethernet network fabric (lan for layer 2 data center network distance) an rdma network interface card (rnic) figure d-1 rdma technology overview rdma technology is now available on ethernet. Roce uses an existing ethernet fabric (swi...

  • Page 506

    478 ibm z13s technical guide figure d-2 dynamic transition from tcp to smc-r the hybrid model of smc-r uses these key existing attributes: it follows the standard tcp/ip connection setup. The hybrid model switches to rdma (smc-r) dynamically. The tcp connection remains active (idle) and is used to c...

  • Page 507

    Appendix d. Shared memory communications 479 figure d-3 shows the concept of the shared roce mode. Figure d-3 shared roce mode concepts the physical function (pf) driver communicates to the physical function in the pcie adapter. The pf driver has a relatively limited function: manages resource alloc...

  • Page 508

    480 ibm z13s technical guide table d-1 shows the differences in the number of ports and shared support for different systems. Table d-1 roce number of enabled ports and shared/dedicated environment the 10gbe roce express feature shown in figure d-4 is used exclusively in the pcie i/o drawer. Each fe...

  • Page 509

    Appendix d. Shared memory communications 481 if the ibm 10gbe roce express features are connected to 10 gbe switches, the switches must support the following requirements: global pause function frame (as described in the ieee 802.3x standard) should be enabled priority flow control (pfc) disabled no...

  • Page 510

    482 ibm z13s technical guide instead of 10 gbe. Osd connections can flow through the same 10 gbe switches or different switches. An osa-express feature, defined as channel-path identifier (chpid) type osd, is required to establish smc-r. Figure d-6 shows the interaction of osd and the rnic. The open...

  • Page 511

    Appendix d. Shared memory communications 483 d.2.7 hardware configuration definitions the following hardware configuration definitions (hcds) are important. Function id the roce feature is identified by a hexadecimal function identifier (fid). It has a dedicated limit in the range 00 - ff. The share...

  • Page 512

    484 ibm z13s technical guide figure d-7 shows the three physically separate networks defined. Figure d-7 physical network id example sample iocp function statement example d-1 shows one sample iocp function configuration to define an roce express adapter shared between lpars. Example d-1 iocp functi...

  • Page 513

    Appendix d. Shared memory communications 485 the following advantages are gained as shown in figure d-8: no middleware or application changes (transparent) ease of deployment (no ip topology changes) lpar-to-lpar communications on a single central processing complex (cpc) server-to-server communicat...

  • Page 514

    486 ibm z13s technical guide smc-r requires the following software: z/os v2r1 (with ptfs) or higher are the only supported operating systems for the smc-r protocol. You cannot roll back to previous z/os releases. Z/os guests under z/vm 6.3 are supported to use 10gbe roce features ibm is working with...

  • Page 515

    Appendix d. Shared memory communications 487 cics to cics connectivity through internet protocol interconnectivity (ipic) optimized sysplex distributor intra-sysplex load balancing dynamic virtual ip address (vipa) and sysplex distributor support are often deployed for high availability (ha) and sca...

  • Page 516

    488 ibm z13s technical guide sysplex distributor after roce figure d-10 shows a roce sysplex distributor: the initial connection request goes through the sysplex distributor stack. The session then flows directly between the client and the target over the roce cards. Figure d-10 sysplex distributor ...

  • Page 517

    Appendix d. Shared memory communications 489 d.3 shared memory communications - direct memory access this section describes the new shared memory communication - direct memory access (smc-d) functions implemented in ibm z13 and z13s driver level 27 servers. Concepts co-location of multiple tiers of ...

  • Page 518

    490 ibm z13s technical guide d.3.1 internal shared memory technology overview ism is a new function supported by the z13 and z13s machines. It is the firmware that provides the connectivity for shared memory access between multiple operating systems within the same cpc. It provides the same function...

  • Page 519

    Appendix d. Shared memory communications 491 smc-d is a “hybrid” solution as shown in figure d-13: it uses a tcp connection to establish the smc-d connection. The tcp connection can be either through the osa adapter or iqd hipersockets a tcp option (smcd) controls switching from tcp to “out of band”...

  • Page 520

    492 ibm z13s technical guide – preservation of the existing ip security model, such as ip filters, policies, vlans, and ssl – minimal network administration and management changes host application software is not required to change, so all host application workloads can benefit immediately. D.3.3 in...

  • Page 521

    Appendix d. Shared memory communications 493 figure d-14 shows the basic concept of vpci adapters: figure d-14 concept of vpci adapter implementation note: basic z/vm support is available: generic zpci pass-through support starting from z/vm 6.3 the use of the zpci architecture remains basically unc...

  • Page 522

    494 ibm z13s technical guide figure d-15 shows an smc-d configuration in which ethernet provides the connectivity. Figure d-15 smc-d configuration that uses ethernet to provide connectivity ip subnet ‘a’ z/os a (lp 1) z/os b (lp 2) ibm z13 or z13s ism network (pnet x) osa osa qdio qdio pr/sm etherne...

  • Page 523

    Appendix d. Shared memory communications 495 figure d-16 shows an smc-d configuration in which hipersockets provide the connectivity. Figure d-16 smc-d configuration that uses hipersockets to provide connectivity d.3.5 planning considerations in the z/os smc-d implementation, z/os uses a single virt...

  • Page 524

    496 ibm z13s technical guide 4. Enable smc-d in at least two z/os instances (single parameter in tcp/ip global configuration). Both z/os instances must execute on the same cpc. 5. Review and adjust as needed available real memory and fixed memory usage limits (z/os and cs). Smc requires fixed memory...

  • Page 525

    Appendix d. Shared memory communications 497 example d-3 shows a sample iocp function configuration that defines ism adapters shared between lpsrs and multiple vlans on the same cpc as shown in figure d-18. Figure d-18 multiple lpars connected through multiple vlans workloads can be logically isolat...

  • Page 526

    498 ibm z13s technical guide configuration considerations the iocds (hcd) definitions for ism pci vfs are not directly related to the software (smc-d) exploitation of ism (that is, the z/os tcp/ip and smc-d implementation and usage are not directly related to the i/o definition). The user defines a ...

  • Page 527

    Appendix d. Shared memory communications 499 table d-2 shows a list of required apars per z/os subsystem. Table d-2 table with prerequisite apars for smc-d enablement d.3.10 enabling smc-d support in z/os communications server the new parameter smcd (see figure d-19) will be available on the globalc...

  • Page 528

    500 ibm z13s technical guide d.3.11 smc-d support overview smc-d requires ibm z13 and ibm z13s servers to be at driver level 27 or later for support of ism. Smc-d requires the following software: z/os v2r2 with ptfs (see table d-2 on page 499) or later is the only supported operating systems for the...

  • Page 529

    © copyright ibm corp. 2016. All rights reserved. 501 appendix e. Ibm dynamic partition manager this appendix contains an introduction to ibm dynamic partition manager (dpm) on z systems. It provides a description about how dynamic partition manager environment can be initialized and managed. This ap...

  • Page 530: E.2  Why Use Dpm?

    502 ibm z13s technical guide e.1 what is ibm dynamic partition manager? Dpm is a z systems mode of operation that provides a simplified approach to creating and managing virtualized environments, reducing the barriers to its adoption for new and existing customers. The implementation provides built-...

  • Page 531

    Appendix e. Ibm dynamic partition manager 503 e.3 ibm z systems servers and dpm traditional ibm z systems servers are highly virtualized with the goal of maximizing the utilization of compute and i/o (storage and network) resources, and simultaneously lowering the total amount of resources needed fo...

  • Page 532

    504 ibm z13s technical guide the firmware partition master control services (mcs), along with the support element (se), provides services to create and manage the linux native partitions, or partitions that run kvm. The connectivity from the se to the mcs is provided through the internal management ...

  • Page 533

    Appendix e. Ibm dynamic partition manager 505 after this option is selected, a new window is displayed where the user enters the two osa express5s 1000base-t ports selected and cabled to the system control hubs (schs) during the z systems installation. This window is shown in the figure e-3. Figure ...

  • Page 534

    506 ibm z13s technical guide figure e-4 shows the dpm mode welcome window. The three options at the bottom (getting started, guides, and learn more have mouse-over functions that either briefly describe their meaning or provide more functions. Figure e-4 dpm mode welcome window the hmc can monitor a...

  • Page 535

    Appendix e. Ibm dynamic partition manager 507 figure e-5 shows the welcome window from a traditional hmc when none of the defined cpc objects are running in dpm mode. Figure e-5 traditional hmc welcome window when no cpcs are running in dpm mode e.4.1 defining partitions in dpm mode after the cpc is...

  • Page 536

    508 ibm z13s technical guide the first option on the left of the window shown on figure e-6 on page 507 is getting started. Thisoption initiates the dpm wizard application on the hmc, which allows users to define their partitions, and then associate processor and memory resources, network and storag...

  • Page 537

    Appendix e. Ibm dynamic partition manager 509 figure e-7 also shows an option on the lower left called advanced. The advanced option allows users to open a window that contains all definitions made for the partition. This window provides extra settings for some of the definitions, such as defining a...

  • Page 538

    510 ibm z13s technical guide e.4.2 summary dpm provides simplified z systems hardware and virtual infrastructure management that includes integrated dynamic i/o management for users who intend to run linux on z systems and kvm for ibm z as hypervisor, running in a partition. The new mode, dpm, provi...

  • Page 539: Kvm For Ibm Z Systems

    © copyright ibm corp. 2016. All rights reserved. 511 appendix f. Kvm for ibm z systems this appendix contains an introduction to open fertilization with kvm for ibm z systems and a description of how the environment can be managed. This appendix includes the following sections: why kvm for ibm z sys...

  • Page 540

    512 ibm z13s technical guide f.1 why kvm for ibm z systems organizations are challenged to drive new business opportunities while reducing budgets, managing increased it complexity, and improving staff productivity. Although cost considerations are important, businesses are looking at other benefits...

  • Page 541

    Appendix f. Kvm for ibm z systems 513 platform mobility easier. Its live relocation capabilities enable you to move virtual machines and workloads without incurring downtime. Table f-1 shows the features that are supported by kvm for ibm z systems. Table f-1 supported features in kvm for ibm z syste...

  • Page 542

    514 ibm z13s technical guide work on physical cpus. Lpus can be dynamically added to and removed from an lpar. Lpars can be added, modified, activated, or deactivated in z systems platforms by using the hardware management console (hmc). Kvm for ibm z systems also uses pr/sm to access storage device...

  • Page 543

    Appendix f. Kvm for ibm z systems 515 signaling protocol to directly exchange data between the osa microprocessor and network software. Kvm for ibm z systems can use the osa features by virtualizing them for linux on ibm z systems to use. For more information about storage and network connectivity f...

  • Page 544

    516 ibm z13s technical guide the kvm for ibm z systems kernel provides the core virtualized infrastructure. It can schedule virtual machines on real cpus and manage their access to real memory. Qemu runs in a user space and implements virtual machines using kvm module functions. Qemu virtualizes rea...

  • Page 545

    Appendix f. Kvm for ibm z systems 517 libvirt is open source software that is on kvm and many other hypervisors to provide low-level virtualization capabilities that interface with kvm through a cli called virsh. A list of virsh commands can be found in the table f-2 on page 517 table f-2 basic virs...

  • Page 546

    518 ibm z13s technical guide configure the nagios monitoring, see getting started with kvm for ibm z systems, sg24-8332. F.3 managing the kvm for ibm z systems environment kvm for ibm z systems integrates with standard openstack virtualization managements, enabling enterprises to easily integrate li...

  • Page 547

    Appendix f. Kvm for ibm z systems 519 libvirt provides different methods of access, from a command line called virsh to a low-level api for many programming languages (see figure f-4). Figure f-4 kvm management by using the libvirt api layers the main component of the libvirt software is the libvirt...

  • Page 548

    520 ibm z13s technical guide f.3.1 hypervisor performance manager hpm monitors and manages workload performance of the virtual machines under kvm by performing the following operations: detect when a virtual machine is not achieving its goals when it is a member of a workload resource group. Determi...

  • Page 549: Native Peripheral Component

    © copyright ibm corp. 2016. All rights reserved. 521 appendix g. Native peripheral component interconnect express this appendix introduces the design of native peripheral component interconnect express (pcie) features management on z13s and includes the concepts of the integrated firmware processor ...

  • Page 550

    522 ibm z13s technical guide g.1 design of native pcie i/o adapter management the native pcie adapter is a new category of features introduced since zec12. These features are 10gbe remote direct memory access (rdma) over converged ethernet (roce) express, zenterprise data compression (zedc) express,...

  • Page 551

    Appendix g. Native peripheral component interconnect express 523 g.1.3 resource groups the ifp allocates two rgs for running the management functions of native pcie feature adapters. A native pcie feature adapter is managed by one of the resource groups according to which i/o domain this adapter is ...

  • Page 552

    524 ibm z13s technical guide g.1.4 management tasks the ifp and resource groups perform all management tasks on the native pcie adapters: firmware update of adapters and resource groups error recovery and failure data collection diagnostic and maintenance tasks firmware update of adapters and resour...

  • Page 553

    Appendix g. Native peripheral component interconnect express 525 considering availability, install adapters of the same type in slots of different i/o domains, drawers, fanouts, and resource groups (for roce express and zedc express). The next sections describe more information about achieving a hig...

  • Page 554

    526 ibm z13s technical guide figure g-2 on page 525 lists the following information for each adapter: pchid and ports resource group the adapter is attached to (comment column) physical location (drawer, slot) the report shows a balanced configuration where i/o domains and resource groups are distri...

  • Page 555

    Appendix g. Native peripheral component interconnect express 527 figure g-3 example of iocp statements for zedc express and 10gbe roce express g.3.1 function identifier the function identifier (fid) is a hexadecimal number between 000 and fff that you use to assign a pchid to the function to identif...

  • Page 556

    528 ibm z13s technical guide notes: for more information about the function statement, see z systems input/output configuration program user’s guide for icp iocp, sb10-7163. Definition of10 gbe roce express feature is required to pair up with an osd chpid definition, by the parameter of pnetid. The ...

  • Page 557: Flash Express

    © copyright ibm corp. 2016. All rights reserved. 529 appendix h. Flash express this appendix covers the ibm flash express feature introduced on the zec12 server. Flash memory is a non-volatile computer storage technology. It was introduced on the market decades ago. Flash memory is commonly used tod...

  • Page 558: H.1  Flash Express Overview

    530 ibm z13s technical guide h.1 flash express overview flash express, which is implemented by using flash ssds mounted in peripheral component interconnect express (pcie) flash express feature cards, is available as carry forward or as a new order for z13s servers. Flash express is an innovative so...

  • Page 559

    Appendix h. Flash express 531 cards, redundant pcie paths to flash express cards are provided by redundant i/o interconnect. Unlike other pcie i/o cards, they can be accessed from the host only by a unique protocol. A flash express pcie adapter integrates four ssd cards of 400 gb each for a total of...

  • Page 560: H.2  Using Flash Express

    532 ibm z13s technical guide for higher resiliency and high availability, flash express cards are always installed in pairs. A maximum of four pairs are supported in a z13s system, providing a maximum of 5.6 tb of storage. In each flash express card, data is stored in a raid configuration. If an ssd...

  • Page 561

    Appendix h. Flash express 533 table h-1 gives the minimum support requirements for flash express. Table h-1 minimum support requirements for flash express you can use the flash express allocation windows on the se or hardware management console (hmc) to define the initial and maximum amount of flash...

  • Page 562

    534 ibm z13s technical guide at ipl, z/os detects whether flash is assigned to the partition. Z/os automatically uses flash express for paging unless otherwise specified by using the new z/os pagescm=none parameter. All paging data can be on flash express memory. The function is easy to use, and the...

  • Page 563

    Appendix h. Flash express 535 flash express is used by the auxiliary storage manager (asm) with paging data sets to satisfy page-out and page-in requests received from the real storage manager (rsm). It supports 4 kb and 1 mb page sizes. Asm determines where to write a page based on space availabili...

  • Page 564

    536 ibm z13s technical guide exportable. The applet also creates two advanced encryption standard (aes) symmetric keys. One of these aes keys is known as the key-encrypting key (kek), which is retained on the smart card. The kek can also be exported. The other aes key becomes the flash encryption ke...

  • Page 565

    Appendix h. Flash express 537 during the alternate se initialization, application programming interfaces (apis) are called to initialize the alternate smart card in it with the applet code and create the rsa public/private key pair. The api returns the public key of the smart card that is associated...

  • Page 566

    538 ibm z13s technical guide figure h-7 key serving topology h.3.3 error recovery scenarios possible error scenarios are described in this section. Primary support element failure when the primary se fails, a switch is made to the alternate se, which then becomes the new primary. When the former pri...

  • Page 567

    Appendix h. Flash express 539 primary and alternate support elements fail if the primary and the alternate se both fail, the key cannot be served. If the devices are still up, the key is still valid. If either or both ses are recovered, the files holding the flash encryption key/authentication key c...

  • Page 568

    540 ibm z13s technical guide.

  • Page 569: Gdps Virtual Appliance

    © copyright ibm corp. 2016. All rights reserved. 541 appendix i. Gdps virtual appliance this appendix discusses the geographically dispersed parallel sysplex (gdps) virtual appliance. This appendix includes the following sections: gdps overview overview of gdps virtual appliance gdps virtual applian...

  • Page 570: I.1  Gdps Overview

    542 ibm z13s technical guide i.1 gdps overview gdps is a collection of offerings, each addressing a different set of it resiliency goals that can be tailored to meet the recovery point objective (rpo) and recovery time objective (rto) for your business. Each offering uses a combination of server and...

  • Page 571

    Appendix i. Gdps virtual appliance 543 an ha/dr implementation has various levels: high availability (ha): the attribute of a system to provide service during defined periods at agreed upon levels by masking unplanned outages from users. Ha employs component duplication (hardware and software), auto...

  • Page 572

    544 ibm z13s technical guide i.2 overview of gdps virtual appliance to reduce it costs and complexity, many enterprises are consolidating independent servers into linux images (guests) running on z systems servers. Linux on z systems can be implemented either as guests running under z/vm or native l...

  • Page 573

    Appendix i. Gdps virtual appliance 545 the gdps virtual appliance solution implements the gdps/pprc multiplatform resilience for z systems (xdr), which coordinates near-continuous availability and dr solution by using these techniques: disk error detection heartbeat for smoke tests initial program l...

  • Page 574

    546 ibm z13s technical guide figure i-3 shows an overview of gdps virtual appliance implementation. Figure i-3 gdps virtual appliance architecture overview keep in mind the following considerations for gdps virtual appliance architecture: pprc ensures that the remote copy is identical to the primary...

  • Page 575

    Appendix i. Gdps virtual appliance 547 the gdps virtual appliance implements the following functions: awareness of a failure in a linux on z systems node or cluster by monitoring (heartbeats) all nodes and cluster master nodes. If a node or cluster fails, it can be set up to automatically ipl the no...

  • Page 576

    548 ibm z13s technical guide figure i-4 shows the operation principle of a disk failover operation that uses hyperswap. Figure i-4 gdps storage failover without hyperswap, the procedure to change the primary disk to the secondary can take up to 2 hours, or even more, depending on the installation si...

  • Page 577

    Appendix i. Gdps virtual appliance 549 i.3.3 disaster recovery in a site disaster, gdps appliance will immediately issue a freeze for all applicable primary devices. This action is taken to protect the integrity of the secondary data. The gdps cluster then resets site 1 and site 2 systems and update...

  • Page 578

    550 ibm z13s technical guide.

  • Page 579: Ibm Zenterprise Data

    © copyright ibm corp. 2016. All rights reserved. 551 appendix j. Ibm zenterprise data compression express this appendix describes the optional ibm zenterprise data compression (zedc) express feature of the z13 and z13s, zec12, and ibm zbc12 servers. The appendix includes the following sections: over...

  • Page 580: J.1  Overview

    552 ibm z13s technical guide j.1 overview the growth of data that needs to be captured, transferred, and stored for large periods of time is unrelenting. In addition, software-implemented compression algorithms can be costly in terms of processor resources and storage costs. Zedc express, an optiona...

  • Page 581: J.3  Software Support

    Appendix j. Ibm zenterprise data compression express 553 figure j-1 illustrates the pcie i/o drawer structure and the relationships among card slots, domains, and resource groups. Figure j-1 relationships between the pcie i/o drawer card slots, i/o domains, and resource groups j.3 software support s...

  • Page 582

    554 ibm z13s technical guide j.3.1 z/vm v6r3 support with ptfs z/os guests that run under z/vm v6.3 with ptfs can now use the zedc express feature. Zedc for z/os v2.1 or later and the zedc express feature are designed to support a data compression function to help provide high-performance, low-laten...

  • Page 583: Related Publications

    © copyright ibm corp. 2016. All rights reserved. 555 related publications the publications listed in this section are considered particularly suitable for a more detailed discussion of the topics covered in this book. Ibm redbooks the following ibm redbooks publications provide additional informatio...

  • Page 584

    556 ibm z13s technical guide.

  • Page 585: Ibm Z13S Technical Guide

    Isbn 0 738 441 67 8 sg 24- 82 94- 00 (1.0” spine) 0. 875”1.49 8 ” 460 788 pages ibm z13s technical guide.

  • Page 588: Back Cover

    Ibm.Com /redbooks printed in u.S.A . Back cover isbn 0738441678 sg24-8294-00 ®.