QLogic 7104-3M-CABLE User Manuals

Table of Contents

• Table of Contents

  11
• 1 Introduction

  17
  • Who Should Read this Guide

    17
  • How this Guide is Organized

    18
  • Overview

    19
  • Interoperability

    19
  • Conventions Used in this Guide

    20
    • Table 1-1. Typographical Conventions

      20
  • Documentation

    21
  • Contact Information

    21
• 2 Feature Overview

  23
  • Features

    23
    • Other Changes

      24
    • Continued Support

      24
      • Table 2-1. QLogic Adapter Model Numbers

        25
  • Software Components

    26
• 3 Step-by-Step Installation Checklist

  27
  • Hardware Installation

    27
  • Software Installation

    28
• 4 Hardware Installation

  31
  • Hardware Installation Requirements

    31
    • Hardware

      31
      • Table 4-1. Adapter Models and Related Platforms

        31
      • Form Factors

        32
    • Cabling and Switches

      33
      • Table 4-2. QLogic InfiniBand Cables

        33
      • Optical Fibre Option

        34
    • Configuring the BIOS

      34
  • Safety with Electricity

    35
  • Unpacking Information

    35
    • Verify the Package Contents

      35
    • List of the Package Contents

      35
      • Figure 4-1. QLogic QLE7280 with IBA7220 ASIC

        37
      • Figure 4-2. QLogic QLE7140 Card with Riser, Top View

        37
      • Figure 4-3. QLogic QHT7040/QHT7140 Full and Low Profile Cards with Riser, Top View

        38
    • Unpacking the QLogic Adapter

      38
  • Hardware Installation

    39
    • Hardware Installation for QLE7240, QLE7280, or QLE7140 with PCI Express Riser

      39
      • Dual Adapter Installation

        39
      • Installation Steps

        39
        • Figure 4-4. PCIe Slot in a Typical Motherboard

          40
        • Figure 4-5. QLogic PCIe Host Channel Adapter Assembly with Riser Card

          41
        • Figure 4-6. Assembled PCIe Host Channel Adapter with Riser

          42
    • Hardware Installation for QHT7140 with HTX Riser

      42
      • Figure 4-7. HTX Slot

        43
      • Figure 4-8. QLogic QHT7140 Adapter with Riser Card

        44
      • Figure 4-9. Assembled QHT7140 with Riser

        45
    • Hardware Installation for QLE7240, QLE7280, and QLE7140 Without a PCI Express Riser

      45
    • Hardware Installation for the QHT7140 Without an HTX Riser

      46
      • Figure 4-10. QHT7140 Without Riser Installed in a 3U Chassis

        47
  • Switch Configuration and Monitoring

    47
  • Cabling the Adapter to the InfiniBand Switch

    47
  • Completing the Installation

    48
• 5 Software Installation

  49
  • Cluster Setup

    49
    • Types of Nodes in a Cluster Environment

      49
    • Supported Linux Distributions

      50
      • Table 5-1. InfiniPath/OpenFabrics Supported Distributions and Kernels

        50
    • Distribution Identifiers

      51
      • Table 5-2. Distribution Identifiers

        51
  • Compiler Support

    51
  • Set Up Your Environment

    51
    • Table 5-3. Required Operating System Packages

      52
    • Table 5-4. Specific Component Requirements

      52
  • Choose the Appropriate Download Files

    53
    • Table 5-5. Available Packages for QLogic OFED 1.4 Release

      53
  • Install QLogicIB-Basic with the Installer Tool

    56
    • Table 5-6. INSTALL Options

      61
  • About rpm Installation

    62
    • Using rpm to Install InfiniPath and OpenFabrics

      63
    • RPM Organization

      65
  • Install QLogic OFED User-level Software with the rpm Command

    66
  • Rebuild or Reinstall the kernel-ib Driver with rpm After a Kernel Upgrade

    68
  • Rebuild the kernel-ib Driver on an Unsupported Distribution or an Unsupported Distribution/Kernel Pair

    68
  • Install QLogic OFED Using Rocks

    69
    • Install Frontend and Compute Nodes

      69
    • Rocks Installation on an Existing Frontend Node

      70
  • Install QLogic OFED Using a Platform OCS Kit

    71
  • Install the QLogic InfiniBand Fabric Suite Software

    72
  • Install Lustre Software

    72
  • Installed Layout

    72
  • Remove Software Packages

    74
    • Uninstall Using the Installer Tool

      74
    • Uninstall InfiniPath and OpenFabrics RPMs

      74
    • Uninstall Software with Rocks or Platform OCS

      74
    • Install a Previous Version of QLogicIB-Basic

      75
    • Downgrading RPMs

      75
• 6 Configuring Drivers and Services

  77
  • InfiniPath and OpenFabrics Driver Overview

    77
  • OpenFabrics Drivers and Services Configuration and Startup

    77
    • Configure the IPoIB Network Interface

      78
    • OpenSM

      79
    • SRP

      80
      • Using QLogic SRP

        80
      • Using OFED SRP

        80
    • Configuring and Administering the VNIC Interface

      82
    • MPI over uDAPL

      89
  • Other Configuration: Changing the MTU Size

    90
  • Managing the InfiniPath Driver

    90
    • Configure the InfiniPath Driver State

      91
    • Start, Stop, or Restart InfiniPath

      91
    • Unloading the Driver/Modules Manually

      92
  • Further Information on Configuring and Loading Drivers

    93
• 7 Installation Verification and Additional Settings

  95
  • LED Link and Data Indicators

    95
    • Table 7-1. LED Link and Data Indicators

      95
  • Adapter and Other Settings

    96
  • Customer Acceptance Utility

    97
    • Table 7-2. ipath_checkout Options

      98
• A Installation Troubleshooting

  99
  • Hardware Issues

    99
    • Node Spontaneously Reboots

      99
    • Some HTX Motherboards May Need Two or More CPUs in Use

      99
  • BIOS Settings

    100
    • Enable Advanced Configuration and Power Interface (ACPI)

      100
    • Issue with Supermicro® H8DCE-HTe and the QHT7040

      100
  • Software Installation Issues

    100
    • Missing Kernel RPM Errors

      100
    • Resolving Conflicts

      101
    • openmpi_gcc Fails to Install Because of Dependency on gfortran (RHEL 4)

      102
    • mpirun Installation Requires 32-bit Support

      102
    • Lockable Memory Error on Initial Installation of InfiniPath

      102
  • Configuration Issues

    103
    • ibsrpdm Command Hangs when Two Adapters are Installed but only Unit 1 is Connected to the Switch

      103
    • Outdated ipath_ether Configuration Setup Generates Error

      103
• B Write Combining

  105
  • Introduction

    105
  • Verify Write Combining is Working

    105
  • PAT and Write Combining

    106
  • MTRR Mapping and Write Combining

    106
    • Edit BIOS Settings to Fix MTRR Issues

      106
    • Use the ipath_mtrr Script to Fix MTRR Issues

      107
• C Configuration Files

  109
  • Table C-1. Configuration Files

    109
• D Package Descriptions

  111
  • Package Names with the QLogicIB-Basic Download

    111
  • Different Nodes May Use Different RPMs

    112
  • InfiniPath RPM Version Numbers and Identifiers

    112
  • OpenFabrics RPM Names

    112
  • InfiniPath and OpenFabrics RPMs

    113
  • Documentation RPMs

    113
    • Table D-1. Documentation RPMs

      113
  • InfiniPath RPMs

    113
    • Table D-2. InfiniPath RPMs

      113
    • Table D-3. InfiniPath-Devel/RPMs

      114
    • Table D-4. InfiniPath-MPI/RPMs

      114
  • OpenFabrics RPMs

    115
    • Table D-5. OpenFabrics/RPMs

      115
    • Table D-6. OpenFabrics-Devel/RPMs

      118
    • Table D-7. OpenSM/RPM

      119
    • Table D-8. OpenSM-Devel/RPM

      119
  • Other Adapters

    119
    • Table D-9. Other Host Channel Adapters/RPMs

      119
    • Table D-10. Other Host Channel Adapters-Devel/RPMs

      120
  • Other MPIs

    120
    • Table D-11. OtherMPIs/RPMs

      120
• Index

  123
  • Symbols

    123
  • A

    123
  • B

    123
  • C

    123
  • D

    123
  • E

    124
  • F

    124
  • H

    124
  • I

    124
  • K

    125
  • L

    125
  • M

    125
  • N

    125
  • O

    125
  • P

    125
  • Q

    126
  • R

    126
  • S

    126
  • T

    126
  • U

    127
  • V

    127
  • W

    127