Chuck Laing- Senior Technical Staff Member 23 October

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems http:

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems The Top Ten Things SA’s should know about Storage § Storage Overview - what's inside? 1. 2. 3. 4. What is the Physical makeup? What is the Virtual makeup (good throughput design tips) What is a Storage Pool - where do I place data? What should I be aware of/what should I avoid? (Tips & Pitfalls-Tuning) To Stripe or not Stripe, that is the question! § 5. Connectivity- Picking the right drivers – § 6. Documentation - why it matters – – – § 7. Topology Diagrams 8. Disk Mapping (view at a glance) 9. Easy Storage Inquiry Tools 10. How to Improve Performance – 3 Host Attachment Kits Bottlenecks © 2013 IBM Corporation 3

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Knowledge is POWER! As System’s Administrators – we don’t always KNOW what we don’t know about storage v Ask for storage leveraging what you know v Avoid bottlenecks v Use tools available v Speed problem isolation v Make more informed architectural decisions What we are NOT going to do today: v Try to turn you into storage administrators v Boil the ocean 4 © 2013 IBM Corporation 4

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems What is SVC? NO…Saying SAN Volume Controller doesn't count! § SVC provides flexibility across the entire storage infrastructure ! Manage the storage pool from a central point Make changes to the storage without disrupting host applications Volume SAN Apply copy services across the storage pool SAN Volume Controller Advanced Copy Services Storage Subsystems DS 8000 15 K rpm 6 6 © 2013 IBM Corporation HDS SATA DS 4000 RAID 5 EMC RAID 1 HP JBOD Combine the capacity from multiple arrays on frames into storage pools

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems What is V 7000? IBM builds virtualization into the storage infrastructure Store more with what’s on the floor Flexibility to auto or manually migrate full volumes to meet needs Make changes to the storage without disrupting host applications Virtual Disk Automated use of SSDs through Sub-LUN tiering with Easy Tier Virtual Disk SAN Volume Controller / V 7000 Advanced Copy Services Apply common copy services across the storage pool 7 Manage the storage pool from a central point Storage Pool HP DS 8000 © 2013 IBM Corporation HDS DS 3000 EMC Combine the capacity from multiple arrays into a single pool of storage 7

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems What does “Easy Tier” allow you to do? Breakthrough Performance with Smart Management of Data • IBM Easy Tier provides two basic features: – Easy Tier Automatic Mode Automatically migrates sub-volume data to the right tier in mixed SSD/HDD extent pools (so called hybrid pools) by moving hot extents to SSD and cold extents to HDD. – Easy Tier Manual Mode Manually merge extent pools or migrate full volumes nondisruptively between extent pools to easily optimize data placement across pools and different storage tiers as well as the restriping of volumes within an extent pool • • Dynamic Volume Relocation Dynamic Extent Pool Merge • IBM Storage Tier Advisor Tool (STAT) provides guidance for SSD capacity planning based on existing No Migration workloads. By showing the hot extents on each volume 1 hr Avg: 9. 13 ms it allows to easily evaluate the current SSD usage and to 1 hr SD: 1. 89 ms identify data which would benefit from SSD. Migration Active st st 2 nd+3 rd hr Avg: 7. 61 ms 2 nd+3 rd hr SD: 1. 31 ms DB 2 Brokerage Workload w/Easy Tier DS 8700 Increase of 241% in transactions! • Easy Tier operates on DS 8000, SVC, Storwize V 7000 8 © 2013 IBM Corporation 8

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Virtual Storage Infrastructure Store more with what’s on the floor Virtual Server Infrastructure 9 © 2013 IBM Corporation Storage Hypervisor Manage Virtual Storage Infrastructure Enterprise SAN Volume Controller Tivoli Storage Productivity Center Midrange Storwize V 7000 9

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems IBM XIV builds virtualization into the storage system Store more with what’s on the floor • Functionality – – – Thin Provisioning, snapshot, replication, migration, reporting Built-in Data Migration Facility All inclusive • Management – Ease of management, GUI, single tier, single configuration • Reliability – No downtime, scalability, 60 minute redundancy, self healing • Performance – Consistent performance, no tuning, no hot spots, performance increases with capacity • Cost – Low acquisition costs, low management costs, low operating costs “If we need to add a new application, we can provision the storage very easily without worrying about the technical side of things. That’s really the ‘wow IBM XIV factor’…” Uli Müller Storage System IT Director EURONICS Deutschland e. G 10 © 2013 IBM Corporation 10

IBM System Storage DS 8800 Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Store more with what’s on the floor DS 8800 Enhancements DS 8800 Benefits • Up to 40% better performance • Save floor space with nearly twice the drive density • Reduce costs with up to 36% less energy consumed 11 Host over 1, 000 drives on up to 40% less floor space than IBM DS 8700 with 3. 5 in. disks © 2013 IBM Corporation 2. 5 in. Drives Primary Power Supplies New 2. 5” small form factor drives 50% more drives per enclosure IBM POWER 6+ processors 8 Gb/sec host and device adapters Management Console POWER 6+ Controllers Batteries • • High Density Enclosures, I/O drawers Front view (cover removed) 11

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems A

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems DS 8000 Hardware Physical Makeup Is it important to know the physical makeup? ; Does it really matter? Summary - DS 8700 (2 -way/4 way base frame) 242 x model 941 13 © 2013 IBM Corporation Familiar Layout 99, 999% = ¼ day in 72 years MTBF

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Physical to Logical - peeling back the layers Even numbered extpools Primary IO Data flow ownership Odd numbered extpools Primary IO Data flow ownership Balance Just like an onion -virtualization has many layers A 14 © 2013 IBM Corporation B Arrays across Enclosures 14

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems How

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems DS 8000 Virtualization layers (Concepts & Architecture) How logical extents in ranks are formed from the DS 8000, 6+P+S type array format EXT 1 1 GB 16 EXT 2 1 GB © 2013 IBM Corporation EXT 3 3 EXT 3 1 GB EXT 4 1 GB EXT 5 1 GB

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems SVC - From Physical to Logical View Mapping to Hosts w/SDD or supported Multi. Path Driver Space-efficient Volume Storage Pool Stripe 16 MB – 2 GB Managed Disk LUN 17 vdisk 0 125 GB vdisk 1 10 GB vdisk 2 525 GB mdiskgrp 0 [EMC Group] 400 GB vdisk 3 1500 GB vdisk 4 275 GB vdisk 5 5 GB mdiskgrp 1 [IBM Group] 600 GB mdisk 0 100 GB mdisk 1 100 GB mdisk 2 100 GB mdisk 3 100 GB mdisk 4 200 GB mdisk 5 200 GB mdisk 6 200 GB EMC 100 GB IBM 200 GB © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Examples of correct Host to SVC Cluster zoning Preferred path for vdisk 1 is SVC N 1 P 2 & N 1 P 3 Preferred path for vdisk 2 is SVC N 2 P 2 & N 2 P 3 Non Preferred path for vdisk 1 is SVC N 2 P 2 &N 2 P 3 Non Preferred path for vdisk 2 is SVC N 1 P 2 &N 1 P 3 18 © 2013 IBM Corporation

IBM XIV Grid Storage Standard Features Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems XIV a Massively Parallel – Grid Architecture What is XIV? § Virtualized grid SAN storage: üData distribution across all drives üNo RAID groups to manage • Automatic load balancing ü Consistent performance Raid Groups Disk Tuning • Integrated software: ü ü ü 19 Thin Provisioning High Performance, flexible SNAPshots Remote Replication Consistency Groups Easy to use GUI based management Data migration © 2013 IBM Corporation Complex Mgmt.

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems IBM XIV Storage Distribution Algorithm § Each volume is spread across all drives § Data is “cut” into 1 MB “partitions” and stored on the disks § XIV algorithm automatically distributes partitions across all disks in the system XIV disks behave like connected pseudo-randomly vessels, as the distribution algorithm aims for constant disk equilibrium. Thus, IBM XIV’s Storage overall disk usage could approach 100% utilization when loaded Interface Switching Data Module 20 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems XIV Distribution Algorithm on System Changes § Data distribution only changes when the system changes – Equilibrium is kept when new hardware is added – Equilibrium is kept when old hardware is removed – Equilibrium is kept after a hardware failure Data Module 1 Node 2 Data Module 3 Node 3 21 © 2013 IBM Corporation Data Module 2

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems XIV Distribution Algorithm on System Changes § Data distribution only changes when the system changes – Equilibrium is kept when new hardware is added – Equilibrium is kept when old hardware is removed – Equilibrium is kept after a hardware failure Data Module 1 Data Module 2 Data Module 3 Data Module 4 Node 4 [ hardware upgrade ] 22 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems XIV Distribution Algorithm on System Changes § Data distribution only changes when the system changes – Equilibrium is kept when new hardware is added – Equilibrium is kept when old hardware is removed – Equilibrium is kept after a hardware failure The fact that distribution is full and automatic ensures that all spindles join the effort of data [re-distribution ]after hardware failure configuration change. Tremendous performance gains are seen in recovery/optimization times thanks to this fact. Data Module 1 Data Module 3 23 © 2013 IBM Corporation Data Module 2 Data Module 4

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems IBM Storwize V 7000 What is a V 7000? Modular Hardware Building Blocks Software inherited from prior Offerings plus Enhancements Software inherited from SVC and DS 8000 RAID • RAID 0, 1, 5, 6, 10 • Storage Virtualization (Internal and external disks) • Non-disruptive Data Migration • Global & Metro Mirror (Multi-site) New and enhanced Software functions • New GUI (Easy-to-use, web based, XIV like) • Easy Tier. TM SSD exploitation • RAID & enclosure RAS services and diagnostics • Additional scalability improvements • Flash. Copy Thin Provisioning • Integration with IBM Systems Director • Enhancements to TPC, FCM and TSM support 99, 999% = ¼ day in 72 years MTBF 24 24 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Tips – § Device Queue Depth – Queue Depth can help or hurt performance per LUN • • Be aware of Queue Depth when planning system layout, adjust only if necessary Queue Depth is central to the following fundamental performance formula: IO Rate = Number of Commands * Response Time per Command • • To calculate - best thing to do is go to each device “Information Center” URLs listed in • link slide For example: – IO Rate = 32 Commands per Second /. 01 Seconds (10 milliseconds) per Command = 3200 IOPs • What are the default Queue Depths? ___ Some real-world examples: • OS=Default Queue Depth= Expected IO Rate § HBA transfer rates • AIX – FC adapters Standalone = 16 per LUN = 1600 IOPs per LUN • AIX VIOS = 20 per LUN = 2000 IOPs per LUN § LVM striping vs spreading = 3 per LUN = 300 IOPs per LUN • AIX VIOC • Windows = 32 per Disk = 3200 IOPS per LUN § Data Placement • Content provided by Mark Chitti – Random versus sequential – Spreading versus Isolation What are the most common/Important OS I/O tuning parameters? 25 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Placement and Host Vdisk mapping § Spreading versus Isolation – Spreading the I/O across MDGs exploits the aggregate throughput offered by more physical resources working together – Spreading I/O across the hardware resources will also render more throughput than isolating the I/O to only a subset of hardware resource – You may reason that the more hardware resources you can spread across, the better the throughput • Don’t spread file systems across multiple frames Makes it more difficult to manage code upgrades, etc. § Should you ever isolate data to specific hardware resources? § Name a circumstance! • Isolation – In some cases more isolation on dedicated resources may produce better I/O throughput by eliminating I/O contention – Separate Flash. Copy – Source and Target LUNs – on isolated spindles 26 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Since the time to complete an I/O operation depends on: What cause disk latency? a. Seek time- time to position the read/write head Does Cache mitigate disk latency? b. Rotational delay- time waiting for disk to spin to proper starting point c. Transfer time LUN 1 Made up of strips from the outer section/edge of each physical Disk RAID-5 6+P LUN 1 LUN 3 You could deduce that: a) Logical-disk 3 would be a better place to store data that will be randomly accessed since the read/write heads would most likely have shorter seek times to the middle of the EXT 4 disks. EXT 11 EXT 3 3 EXT 5 EXT 2 LV LV 2 EXT 3 LUN 3 Made up of strips from the middle sections of each physical Disk RAID-5 6+P LV 3 EXT 4 LV 4 EXT 5 LV 5 b) Logical-disk 1 would provide greater sequential throughput since it is on the outer edge of the disks. 27 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Placement What causes THRASHING? Placing Applications on the same LUNs/Pools result in IO contention Most commonly when workloads peak at the same time or log files and data files share physical spindles LUN 1 made of strips on the outer edge of the DDMs (1 s) also could have App A Raid-5 7+P For existing applications, use storage and server performance monitoring tools to understand current application workload characteristics such as: • • 1 1 Copy services requirements (Point-in-Time Copy, Remote Mirroring) Host connection utilization and throughput (HBA Host connections) Remote mirroring link utilization and throughput Extent pool or 8 Ranks Strip 1 Strip 2 Strip 3 Strip 4 Strip 5 28 1 1 1 Read/Write ratio 2 2 2 2 5 5 5 5 4 4 4 4 Random/sequential ratio 4 3 3 3 3 Average transfer size (blocksize) Peak workload (I/Os per second for random access, and MB per second for sequential access) LUN 3 made of strips in the middle of the DDMs (3 s) also could have Peak workload periods (time of day, time of month) App B Raid-5 7+P © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Placement - Why Storage Tiering? – Key Drivers Storage is not used cost efficiently Top Customer Pain Points § 70+% in Tier 1 §<40% utilized Slide and content provided by Laura Richardson 29 © 2013 IBM Corporation 29

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Placement - Storage Pools and Striping § § Should you ever stripe with previrtualized volumes? We recommend not striping or spreading in SVC, V 7000 and XIV Storage Pools Avoid LVM spreading with any striped storage pool You can use file system striping with DS 8000 storage pools – Across storage pools with a finer granularity stripe – Within DS 8000 storage pools but on separate spindles when volumes are created sequentially Striped Pools Sequential Pools Host Stripe No Host Stripe S t r i p e Host Stripe - Raid-0 only 30 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Random IO Data layout Disk subsystem 1 2 3 4 5 What does random LV creation order, help prevent? 1 2 3 4 5 datavg # mklv lv 1 –e x hdisk 1 hdisk 2 … hdisk 5 # mklv lv 2 –e x hdisk 3 hdisk 1 …. hdisk 4 …. . Use a random order for the hdisks for each LV RAID array LUN or logical disk PV Slide Provided by Dan Braden 31 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Sequential IO Data layout § Does understanding the backend enable good front-end configuration? § Sequential IO (with no random IOs) best practice: – Create RAID arrays with data stripes a power of 2 • RAID 5 arrays of 5 or 9 disks • RAID 10 arrays of 2, 4, 8, or 16 disks – Create VGs with one LUN per array – Create LVs that are spread across all PVs in the VG using a PP or LV strip size >= a full stripe on the RAID array – Do application IOs equal to, or a multiple of, a full stripe on the RAID array – Avoid LV Striping • Reason: Can’t dynamically change the stripe width for LV striping – Use PP Striping • Reason: Can dynamically change the stripe width for PP striping Slide Provided by Dan Braden 32 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Layout - OS Spreading versus Striping § Is there is a difference? What’s the diff? – Do you know what are your volumes made of! File system spread 33 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Data Layout Summary § Does data layout affect IO performance more than any tunable IO parameter? § Good data layout avoids dealing with disk hot spots – An ongoing management issue and cost § Data layout must be planned in advance – Changes are generally painful § iostat might and filemon can show unbalanced IO § Best practice: evenly balance IOs across all physical disks unless TIERING § Random IO best practice: – Spread IOs evenly across all physical disks unless dedicated resources are needed to isolate specific performance sensitive data • For disk subsystems • Create RAID arrays of equal size and type • Create VGs with one LUN from every array • Spread all LVs across all PVs in the VG 34 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Supported Connectivity - Does zoning matter? – SVC example § Symmetry § Balance § Performance Throughput SVC to Host • TIP – These make good topology diagrams 35 © 2013 IBM Corporation Storage to SVC

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Connectivity – topology and diagrams facilitate resiliency Can you have too many paths? • If you need to clean up the pathing by reducing to the Preferred path for vdisk 2 is SVC N 2 P 2 & N 2 P 3 Preferred path for vdisk 1 is SVC N 1 P 2 & N 1 P 3 supported number for SVC then here is a useful script Non Preferred path for vdisk 2 is SVC N 1 P 2 &N 1 P 3 Non Preferred path for vdisk 1 is SVC N 2 P 2 &N 2 P 3 • How to remove missing paths on VIO server using a script ths Prefer red Pa * Non Pre fer red Pa ths # cat rmmissing_path lsdev -Cc disk |grep -i MPIO |awk '{print $1}' | while read LINE; do lspath -l $LINE -F "name path_id parent connection status" |grep -i Missing | while read ELINE; do Server/Host view of the datapaths echo $ELINE |awk '{print $3, $4}' |read PRNTDVS CONECTSET rmpath -d -l $LINE -p $PRNTDVS -w "$CONECTSET" done echo "hdisk $LINE done" Done 36 © 2013 IBM Corporation Script and content provided by Aydin Tasdeler

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Host Attachment Kits § SVC -To attach an IBM® System p® AIX® host, you must install the AIX host attachment script. – www. ibm. com/servers/storage/support/software/sdd/downloading. html § DS 8000 - For the most current information on supported hosts, operating systems, adapters, and switches, go to the System Storage® Interoperation Center (SSIC) website at • www. ibm. com/systems/support/storage/config/ssic/. – For additional SDD information, see the IBM System Storage Multipath Subsystem Device Driver User's Guide at – www. ibm. com/systems/support/storage/software/sdd – http: //www. redbooks. ibm. com/redbooks/pdfs/sg 248887. pdf § XIV -Install the proper Host Attachments Kit (Multipathing drivers) – http: //www-01. ibm. com/support/search. wss? q=ssg 1*&tc=STJTAG+HW 3 E 0&rs=1319&dc=D 400&dtm – ftp: //ftp. software. ibm. com/storage/XIV/Host_Attachment/AIX/ § V 7000 – For the most current information on supported hosts – Confirm the adapter driver and firmware levels are current by checking the System Storage Interoperation Center. • http: //www 03. ibm. com/systems/support/storage/config/ssic/displayesssearchwithoutjs. wss? start_over=yes – For host connectivity-Install the host attachment script as specified by the SDDPCM Readme file. • http: //www-01. ibm. com/support/docview. wss? rs=540&context=ST 52 G 7&uid=ssg 1 S 7000303 37 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Track data placement and Host Vdisk mapping Documentation – Does it matter? Why? § Disk mapping at a glance – Mapping becomes important § Spreading versus isolation Spreading 38 © 2013 IBM Corporation Isolation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Documentation – Why it matters § How do I achieve SVC node to Server Balance? • Use the SVCQTOOL listed under the tools section of this slide deck to produce a spread sheet similar to this Or • Use the script found in the speaker notes of this slide • Add a column for preferred node to host client 39 Spreadsheet developed by Keith Williams © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems How do I get the information Are there any automated storage inquiry tools out there that will help me understand my setup? § Storage tools – Gathers information such as, but not limited to: • LUN layout • LUN to Host mapping • Storage Pool maps • Fabric connectivity – DS 8 QTOOL • Go to the following Website to download the tool: http: //congsa. ibm. com/~dlutz/public/ds 8 qtool/index. htm – SVCQTOOL • Go to the following Website to download the tool: http: //congsa. ibm. com/~dlutz/public/svcqtool/index. htm 40 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Administrator - How do I improve disk performance on the Host? § Reduce the number of IOs – Bigger caches • Application, file system, disk subsystem • Use caches more efficiently – No file system logging – No access time updates § Improve average IO service times – Better data layout – Reduce locking for IOs – Buffer/queue tuning – Use SSDs or RAM disk – Faster disks/interfaces, more disks – Short stroke the disks and use the outer edge – Smooth the IOs out over time § Reduce the overhead to handle IOs 41 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Troubleshooting – What’s the most common thing that changes over time? Pool 1 An ounce of prevention is worth a pound of cure Rank • Pool 2 Rank Depending on the work load characteristics, isolating the workload may prove to be more Vdisk 1 beneficial and out perform a larger array. App A – There are 3 important principles for creating a logical configuration for the Storage Pools to optimize performance: Map Host A Rank • • • Workload isolation Workload resource-sharing Workload spreading Rank Some examples of I/O workloads or files/datasets which may have heavy and continuous I/O access patterns are: • Sequential workloads (especially those with large blocksize transfers) Rank Vdisk 2 • Log files or datasets • Sort/work datasets or files App B • Business Intelligence and Data Mining Map Host B • Disk copies (including Point in Time Copy background copies, remote mirroring target volumes, and tape simulation on disk) • Video/imaging applications Data Migration • Engineering/scientific applications Rank • Certain batch workloads • I always separate Log files from Data files for best performance. Apps sharing the same physical spindles on traditional arrays may peak at the same time 42 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Advanced Features – plan careful data placement How can this help me with data placement? SDD Drivers Flash. Copy ü Decoupling the relationship between servers and storage systems makes for efficient use of storage resources. ü Flash. Copy and Mirroring can be between dissimilar disk systems. IBM SAN VOLUME CONTROLLER IO-Group Node Node IO-Group Node ü Designed to offer flexible, SAN wide copy services. CPQ ESS 43 © 2013 IBM Corporation 0101010101 Remote Copy 010010101101001000 LCRB 0101010101 Copy Data 010010101101001000

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Stor. Admin – How do I improve disk performance? § Data layout affects IO performance more than any tunable IO parameter • If a bottleneck is discovered, then some of the things you need to do are: – Identify the hardware resources the heavy hitting volumes are on • • • Identify which D/A pair the rank resides on Identify which I/O enclosure the D/A pair resides on Identify which host adapters the heavy hitting volumes are using Identify which host server the problem volumes reside on Identify empty non used volumes on other ranks – storage pools – Move data off the saturated I/O enclosures to empty volumes residing on less used ranks/storage pools – Move data off the heavy hitting volumes to empty volumes residing on less used hardware resources and perhaps to the another Storage Device – Balance LUN mapping across • Backend and host HBAs • SVC iogrps • SVC preferred nodes – Change Raid type. 44 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Troubleshooting: What are some Storage Bottlenecks? • After verifying that the disk subsystem is causing a system bottleneck, a number of solutions are possible. These solutions include the following: • Consider using faster disks SDD will out perform HDD, etc. • Eventually change the RAID implementation if this is relevant to the server’s I/O workload characteristics. – For example, going to RAID-10 if the activity is heavy random writes may show observable gains. • Add more arrays/ranks to the Storage pool. – This will allow you to spread the data across more physical disks and thus improve performance for both reads and writes. • Add more RAM – Adding memory will increase system memory disk cache, which in effect improves disk response times. • Finally, if the previous actions do not provide the desired application performance: – Off-load/migrate - processing to another host system in the network (either users, applications, or services). 45 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Helpful URLs- One URL is missing, what is it? • • And………… IBM Systems Magazine § For Redbooks on the topics discussed here go to: • http: //www. redbooks. ibm. com/ – If any of the above links do not work for you, – In the Serach field – type in the Keyword you want to pull up information on, such as SVC, copy and paste this URL into your browser: DS 8000, XIV, V 7000 www. ibmsystemsmagpowersystemsdigital. com § SVC Information Center • page 34 and 35 • http: //publib. boulder. ibm. com/infocenter/svcic/v 3 r 1 m 0/index. jsp – V 7000 - Install the host attachment script as specified by the SDDPCM Readme file. • http: //www-01. ibm. com/support/docview. wss? rs=540&context=ST 52 G 7&uid=ssg 1 S 7000303 § V 7000 -Support Information Center • http: //publib. boulder. ibm. com/infocenter/storwize/ic/index. jsp – Support for Storwize V 7000 website at • www. ibm. com/storage/support/storwize/v 7000 § XIV Information Center • http: //publib. boulder. ibm. com/infocenter/ibmxiv/r 2/index. jsp – XIV Redbooks: Architecture, Implementation, & usage • http: //www. redbooks. ibm. com/abstracts/sg 247659. html • http: //www. redbooks. ibm. com/redpieces/abstracts/sg 247904. html? Open § DS 8000 Information Center – http: //publib. boulder. ibm. com/infocenter/dsichelp/ds 8000 ic/index. jsp – Redbook - http: //www. redbooks. ibm. com/redbooks/pdfs/sg 248887. pdf © 2013 IBM Corporation § Details about the what the RAID levels actually mean is available at URL below:

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Summary § § Knowing - what's inside will help you make informed decisions? You should make a list of the things you don’t know – § Talk to the Storage Administrator or those who do know A better Admin understands 1. 2. 3. 4. § § The backend physical makeup The backend virtual makeup What's in a Storage Pool for better data placement Avoid the Pitfalls associated with IO Tuning 5. Know where to go to get right device drivers 6. Know why documentation matters – – 7. Keep Topology Diagrams 8. keep Disk Mapping documentation 9. Be able to use Storage Inquiry Tools to find answers 10. Understand how to troubleshoot storage performance bottlenecks © 2013 IBM Corporation 47

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Questions-

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Thank

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Extras

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems You can make Tiers on the same Storage Platform Manual Mode (Volume Level) and Automatic Mode (Extent Level) SSD Rank Pools FC or SATA Rank Pools Merged Pool (SSD/FC or SSD/SATA) Manual Mode Volume Based Data Relocation Automatic Mode Extent Level Data Relocation Change disk class Change RAID type Change disk RPM Automatic extent level data relocation enabled in a Merged Extent Pool 51 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Tiered Storage Strategy Overview You should consider: Cost versus Performance Storage Pyramid Tier 0: Ultra High Performance Applications 1 -3% 15 -20% Tier 0 Tier 1: Mission critical, revenue generating applications Tier 2: Backup, recovery and vital data 20 -25% Ultra High Performance 50 -60% Tier 3: Archives and long term retention Tier 1 High Performance Mission Critical Cost Per Gigabyte Tier 2 Medium Performance Non-Mission Critical Tier 3 Low Performance Archival/Tape Performance 52 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Tiered Storage Classification TIER 0 – Solid State drives only Description Ultra High Performance. Meet Qo. S for High End Technical Examples – High level Guidance – Local variations on technology exist IBM (block) SVC Recommended for Open IBM (file) Nseries/So. NAS I/O Density Performance range capability DS 8870 -> Greater than 250, 000 IOPs/5, 300+ MBs DS 8870/SVC 400 GB recommended *** RAID 5 >1 -- Small Block Recommended – COST / PERFORMANCE / AVAILABILITY 720/820 Flash. System TIER 1(a) High Performance. DS 8870 w/SAS 300 GB 15 K disk drive Drive up utilization RAID 5/RAID 10 arrays *** of high-ended storage subsystems and -- DS 8700/DS 8800 removed from strategy -still maintain performance Qo. S objectives. N 7000 series/FAS 6 series units with HDD disk drive tech. 146 GB/300 GB SAS drives in RAID-DP arrays. XIV* (GEN 3) model 214 with 2 TB SAS drives (11 Module or greater unit) For low capacity requirements smaller less powerful devices may meet tier definition XIV* (GEN 3) model 214 with 3 TB SAS drives Medium Performance. Meet Qo. S for applications/data that resides here. TIER 1(b) DS 8870 w/SAS 600 GB 10 k disk drive (11 Module or greater unit) --- 11. 2 code version required ----- SSDs (solid state drives) required ----- XIV GEN 2 removed from strategy --- RAID 5/RAID 10 arrays V 7000 w/SAS 600 GB using RAID 5 V 7000 (Pure) Integrated Chassis Platform TIER 2 For low capacity requirements smaller less powerful devices may meet tier definition w/450 GB SAS N 6000 series/FAS 3 series units with HDD disk drive tech. 146 GB/300 GB/450 GB SAS drives in RAID-DP arrays. So. NAS** with SAS V 7000 Unified (Pending) XIV* (GEN 3) with 2 TB SAS drives (9 or 10 modules) XIV* (GEN 3) with 3 TB SAS drives (9 or 10 modules) -- No XIVs less than 9 modules recom -- Response times may increase exponentially above these ranges TIER Keep within I/O density ranges to maintain good (eg <10 ms) response times - 720/820 Flash. System Greater than 400, 000 IOPs mixed work load (70/30). 100% write the 720 should be used as it can maintain this while the 820 drops down significantly. 0. 5 DS 8870 -> 250, 000+ IOPs / 5800 MBs 1 N 7000/FAS 6 -> less than 60, 000 IOPs or less than 1, 500 MBs XIV 2 TB XIV (GEN 3 15 mod) less than 120, 000 IOPs/3, 000 MBs XIV (GEN 3 11 mod) less than 80, 000 IOPs/2, 600 MBs XIV 3 TB XIV (GEN 3 15 mod) less than 100, 000 IOPs/2, 800 MBs XIV (GEN 3 11 mod) less than 70, 000 IOPs/2, 400 MBs <0. 5 DS 8870 -> less than 50, 000 IOPs or less than 1, 500 MBs V 7000 (block) -> less than 50, 000 IOPs or less than 1, 500 MBs N 6000/FAS 3 -> less than 50, 000 IOPs or less than 1, 500 MBs So. NAS -> up to 25, 000 IOPs/1, 000 MBs XIV 2 TB XIV (GEN 3) less than 70, 000 IOPs/2, 400 MBs XIV 3 TB XIV (GEN 3) less than 50, 000 IOPs/2, 000 MBs * XIV should not be used for transaction intensive workloads (under 5 ms response) and CACHE unfriendly workloads. Use DS 88 for this need. ** So. NAS to be used as part of Cloud offering only *** Recommendation is to run DISK Magic to confirm drive selection/amount needed/DA Pairs needed © 2013 IBM Corporation Tier rating is based on performance AND reliability

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Tiered Storage Classification Keep within I/O density ranges to maintain good (eg <10 ms) response times Technical Examples – High level Guidance – Local variations on technology exist IBM (block) SVC Recommended for I/O IBM (file) Nseries/So. NAS Performance range capability Open Density Low Performance. Meet Qo. S for applications/data that resides here. DS 8870 with NL-SAS tech using RAID 6 N 3000 series/FAS 2 series. Any unit with SATA using RAID DP V 7000 with NL-SAS using RAID 6 So. NAS** with NL-SAS V 7000 Unified (Pending) Archival, long term retention, backup Virtual Engines, Tape ATLs, Protec. TIER 4 <0. 1 Response times may increase exponentially above these ranges Description TIER 3 COST / PERFORMANCE / AVAILABILITY TIER DS 8870 -> less than 30, 000 IOPs or less than 1, 000 MBs N 3000/FAS 2 -> 11, 000 IOPs/500 MBs thru 30, 000 IOPs/750 MBs So. NAS -> up to 5, 000 IOPs/400 MBs V 7000 (block) -> less than 30, 000 IOPs or less than 300 MBs N/A Tier based on features. * XIV should not be used for transaction intensive workloads (under 5 ms response) and CACHE unfriendly workloads. Use DS 88 for this need. ** So. NAS to be used as part of Cloud offering only *** Recommendation is to run DISK Magic to confirm drive selection/amount needed/DA Pairs needed © 2013 IBM Corporation Tier rating is based on performance AND reliability

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Thin Provisioning – What is it? § Users define volumes with any logical size § Users acquire only the physical capacity of XIV Storage needed for data that is actually written – The part of the volume that contains no data does not consume any physical space Real Capacity Volume 70 GB Amount of physical storage required Real Capacity Used Volume 70 GB Capacity 150 GB Volume 50 GB Volume 30 GB Actual Data Written 10 GB Actual Data Written 30 GB Virtual Actual Capacity Data Written 20 GB Fat provisioned 55 55 © 2013 IBM Corporation Used Capacity 60 GB Used Actual Data Capacity Written 10 GB Actual Data Written 30 GB Actual Data Written 20 GB Thin provisioned

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems IBM XIV Storage: Thin Provisioning § § § 56 Defining logical volumes bigger than physical capacity Installing physical capacity only if and when needed No space consumed when data is 0 Pools are used to manage quota Results: – Reduced overall direct storage cost – Storage expenses spread over time, exploiting price reductions – Easier management – Save 20 -50% of storage capacity © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Storage Problems and Limitations Today º Static relationship between servers and storage systems Inefficient use, storage not available to all servers º Space EMC Drivers Can standardize on one vendor – usually very expensive Or restrict servers to vendors Use SVC and TPC to address EMC Drivers RDAC Drivers Proprietary, non-interoperable Copy Services º Out of SDD Drivers Migration of data disruptive and time consuming º SDD Out of Drivers Space º º Flashcopy ? Out of Space DS 8000 Free capacity Remote Copy ? EMC DS 4000 0101010101 Data Migration 010010101101001000 Traditional SAN storage environments 57 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Connectivity - Host to SVC Zoning Best Practices • Good communication between the SA and the Storadmin, can uncover issues quickly – Correct datapathing has 3 factors • Proper zoning • Proper SVC Host definitions (SVC logical config of the host def) • Proper redundancy for the SVC preferred /non preferred pathing Incorrect Correct 58 © 2013 IBM Corporation DEV#: 3 DEVICE NAME: hdisk 3 TYPE: 2145 ALGORITHM: Load Balance SERIAL: 600507680181059 BA 00000005 ============================== Path# Adapter/Path Name State Mode Select Errors 0 fscsi 0/path 0 OPEN NORMAL 558254 0 1* fscsi 0/path 1 OPEN NORMAL 197 0 2* fscsi 0/path 2 OPEN NORMAL 197 0 3 fscsi 0/path 3 OPEN NORMAL 493559 0 4 fscsi 2/path 4 OPEN NORMAL 493330 0 5* fscsi 2/path 5 OPEN NORMAL 197 0 6* fscsi 2/path 6 OPEN NORMAL 197 0 7 fscsi 2/path 7 OPEN NORMAL 493451 0 8 fscsi 5/path 8 OPEN NORMAL 492225 0 9* fscsi 5/path 9 OPEN NORMAL 197 0 10* fscsi 5/path 10 OPEN NORMAL 197 0 11 fscsi 5/path 11 OPEN NORMAL 492660 0 12 fscsi 7/path 12 OPEN NORMAL 491988 0 13* fscsi 7/path 13 OPEN NORMAL 197 0 14* fscsi 7/path 14 OPEN NORMAL 197 0 15 fscsi 7/path 15 OPEN NORMAL 492943 0

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Tip - Queue Depth Tuning § Take some measurements § Make some calculations – (Storage port depth / total LUNs per host = queue depth) • If a single host with 10 assigned LUNs, is accessing the storage port supporting 4096 then calculate as (4096/10 = 409) or 256 in this case – Are there different calculations for the different storage devices? • For volumes on homogeneous hosts examples: SVC q = ((n × 7000) / (v×p×c)) DS 8000 = 2048 XIV= 1400 V 7000 q = ((n * 4000) / (v * p * c)) • Best thing to do is go to each device “Information Center” URLs listed in link slide – Don’t increase queue depths beyond what the disk can handle! • IOs will be lost and will have to be retried, which reduced performance § Note: – For more information on the info needed to make the calculations, please refer to the “Dan Braden” in the Extra slides at the end of this deck 59 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems More on Host Disk IO Tuning § Please refer to the following PPTs provided by Dan Braden § Disk IO Tuning § SANBoot 60 © 2013 IBM Corporation

Best Practices for Performance, Design and Troubleshooting IBM Storage connected to Power Systems Document VIO to LPAR mapping § Script Output sample to produce documentation Content provided by Aydin Y. Tasdeler 61 © 2013 IBM Corporation