This README is for the Linux Fibre Channel HBA failover device driver version 8.00.02 for the IBM FAStT Host Adapter Driver,the IBM FAStT FC-2 and the IBM FAStT FC2-133 Host Bus Adapters (HBAs). IMPORTANT: This version of the Linux Fibre Channel HBA device driver should only be used with the 2.6 kernel. Products supported: --------------------------------------------------------------------------- | FAStT Adapter | Qlogic Adapter | IBM Feature Code | IBM Option P/N | ---------------------------------------------------------------------------- |FAStT Host Adapter| QLA2200F/66-IBM-SP | FC2102 | 00N6881 | |FAStT FC2 | QLA2310FL-IBM-SP | FC2130 | 19K1246 | |FAStT FC2-133 | QLA2340-IBM-SP | FC2104 | 24P0960 | --------------------------------------------------------------------------- Last Update: 07/13/2005 Note: The 07/13/2005 update to the readme is to add support for RedHat4. ======================================================================= Contents -------- 1.0 OS Support 2.0 Supported Features 3.0 Release History 4.0 Host Adapter configuration 4.1 Update IBM FAStT Host Adapter BIOS 4.2 Configure the NOVRAM Setting for the IBM FAStT Host Adapter 5.0 Creating the Driver Diskette 5.1 Driver Disk for Adding Driver to Existing OS 6.0 Building a Driver from the Sources Code 6.1 Install the kernel source and header packages 6.2 Install the IBM FAStT (Qlogic) driver source code 6.3 Building the device driver 7.0 Loading and Configuring the driver 7.1 Enable more than 1 SCSI device per adapter 7.2 Install FAStT_MSJ 7.3 Modify the module load time options 7.4 Loading the Driver manually 7.5 Loading the Driver using a ramdisk image 7.6 Rebuilding the ramdisk image after configuration changes 8.0 Failover Support 8.1 How to enable the Failover support in the Driver 8.2 Configuration Changes Made via FAStT MSJ 8.3 Persistent Binding 9.1 Proc Filesystem Support 10.0 Driver file Contents 11.0 WEB Sites and Support Phone Number 12.0 Trademarks and Notices 13.0 Disclaimer ======================================================================= 1.0 OS Support -------------- The 8.00.02 device driver was tested with the following Linux versions and kernel versions: --------------------------------------------------------------------- | Suse Linux Enterprise Server 9 | 2.6.5-7.151 | | 32-bit | | |--------------------------------|------------------------------------| | Suse Linux Enterprise Server 9 | 2.6.5-7.151 | | IA-64 (Homogeneous Only) | | |--------------------------------|------------------------------------| | Suse Linux Enterprise Server 9 | 2.6.5-7.151 | | AMD-64 (Homogeneous Only) | | |--------------------------------|------------------------------------| | RedHat 4.0 32-bit | 2.6.9-11.EL | |--------------------------------|------------------------------------| | RedHat 4.0 IA-64 | 2.6.9-11.EL | | (Homogeneous Only) | | |--------------------------------|------------------------------------| | RedHat 4.0 AMD-64 | 2.6.9-11.EL | | (Homogeneous Only) | | --------------------------------------------------------------------- Earlier or later versions of the vendor kernels or generic kernels have not been tested with this device driver version and may not be supported with this release. The Qlogic device driver in the Linux vendor packages have not been tested and are not supported by IBM. This driver package contains the Qlogic driver source code only. The device driver must be compiled from the source code for your specific Linux installation. This readme file makes an assumption that the user has already configured their storage subsystem properly for a Linux server with storage partitioning enabled and configured with the host type set to LINUX if the driver is configured as LUN failover/failback (fo) driver. If this driver is configured as non-failover/non-failback (non-fo) driver, the host type set to LNXCL and a multipath driver for LUN failover and failback, like IBM DS4000 Linux RDAC driver, must also be installed. Only like type adapters can be configured as a failover pair. An IBM FAStT Host Adapter cannot failover to an IBM FAStT FC-2 Host Bus Adapter. IBM FAStT Host Adapter Driver uses the qla2200.ko device driver. The IBM FAStT FC-2 and FC2-133 Host Bus Adapters use the qla2300.ko device driver. This readme will use the term IBM FAStT Host Adapter to refer to the IBM FAStT Host Adapter, the IBM FAStT FC-2 Host Bus Adapter and the IBM FAStT FC2-133 Host Bus Adapter. ======================================================================= 2.0 Supported Features ---------------------- 2.1 ISP2x00 Features -------------------- * FCAL - direct attach loop * Point-to-point * Fabric support * Initiator mode only * Fault recovery on down loops * Persistent binding * Extended LUN support up to 255 LUNs * FC tape support * Multi-path failover support * Hot Add LUNs ======================================================================= 3.0 Release History ------------------- Version 8.00.02 dated 5/17/2005 - Updated ISP23XX firmware to v3.03.11 - Correct issue with 'selective exclusion of failover' patch. - Do not perform any failover operations on tape storage. - Add support for selective exclusion of failover devices based on known types. - Updated Options Rom region code definition. - Sync up ioctl definition files with 2.02beta5 api lib. - Code formatting changes. - Add support for new NVRAM parts. - Add 'extended_error_logging' modules-parameter support to enable DEBUG/DEBUG2. - Added checking and returning of "missing" devices that are still found in persistent binding configuration in qla2x00_std_get_tgt(). - Added returning of PCI Domain value used to identify PCI devices in 2.6 kernel via query_chip ioctl. - Added more port speed definition in exioct.h, updated exioctln.h with latest version, and corrected ha->link_data_rate assignment. - Corrected return status value used when copy error. - Corrected ISP23xx beacon support. - Corrected additional compilation warnings with certain kernel distributions. - Added checking of ConfigRequired flag in fcport_bind in order to support "unconfigure" of devices. - Add shell script to assist in driver compilation: extras/build.sh. - Fixed issue reading tpg_id from pg 83 of inquiry. buffer too small. - Add code to exclude non-disk storage from failover processing and handling. - Return the proper IOCTL state during a loop-dead condition. - Resync with Linux Kernel 2.6.10-rc2. - Additional code cleanup of unsused functions. - [PATCH] qla2xxx: remove dead code, add missing statics. - Add fix to report the correct target port group based on state. - Add fix to address the panic for devices which does not support Tgt Port Group(TPG) -- initialize the TPG List irrespective whether devices support TPG or not. - Add fix to handle the transition wait time from standby to active state for DS400 storage during the execution of the set_target_port_grp(). - Added the support for devices which support Target Port Group. - Fixed the segmentation fault while freeing tgt_port/lu_path memory - Static Load balancing turned off by default. Default behaviour is to expose all the luns on the first active path. - Fixed the incorrect masking off ALUA field from inquiry data. - Add the luns to active lu_path irrespective of ALUA support. - Free the tgt_port_grp/lu_path/mp_port memory during unload. - Added the lu_path to describe path on a per lun basis. - Mark the fclun as active if tgt_port_grp is in active state. - Added the support for Target Port Groups and Set Target Port Groups. - Correct failover issue while dealing with handling multi-controller/multi- ported storage. - Moved inline functions from qla_32ioctl.c to qla_32ioctl.h to eliminate compile error on gcc 3.4.2. - Add support for HBA portname aliasing. - Properly iounmap() allocations. - Sparse __iomem annotations. - Packed the loopback related ioctl structures so the sizes are consistent between user and kernel spaces on certain platforms. - Use proper return codes in qla2x00_fabric_login() to ensure DPC process login requests correctly. - Calculate the checksum of the incoming binary file AFTER writing the ascii file and return an exit status of 2 for an invalid checksum. - When verifying the portname of the incoming ascii file, return an exit status of 1 on error. - Return an exit status of 3 on command line argument errors. - Handle PLOGI reject failures during qla2x00_fabric_login(). Version 8.00.00 dated 3/22/2005 - Initial Release to support the linux 2.6 kernel. (SLES9) ======================================================================= 4.0 Host Adapter configuration ------------------------------ 4.1 Update IBM FAStT Host Adapter, IBM FAStT FC-2 Host Bus Adapter or IBM FAStT FC2-133 Host Bus Adapter BIOS --------------------------------------------------------------------- 4.1.1 BIOS update using the DOS bootdisk ---------------------------------------- The adapter BIOS can be updated by booting the server to the BIOS update diskette, available from the IBM Support website, then run the following commands: flasutil /f /l Note: The FAStT adapter BIOS update program will only update like adapters. If you have a server that contains IBM FAStT Host Adapters and IBM FAStT FC-2 Host Bus Adapters you may only update one adapter type at a time. 4.1.2 EFI Driver Update For IA64 Environment -------------------------------------------- IMPORTANT: The IBM FAStT Host Adapter (2200) is not supported in an IA-64 Environment. Download the latest EFI Driver update package from the IBM FAStT Support website. Please refer to the Readme in the EFI Driver update package for installation instructions. 4.2 Configure the NOVRAM settings for the IBM FAStT FC-2 Host Bus Adapter and IBM FAStT FC2-133 Host Bus Adapter. ------------------------------------------------------------------------- Note: Currently the only method to change the adapter NOVRAM settings in an IA-64 environment are with FAStT_MSJ. All settings, except for the following, should maintain the IBM defaults. - Host Adapter settings Loop reset delay - 8. - Advanced Adapter Settings LUNs per target - 0 Enable Target Reset - Yes Port down retry count - 12 1. As the host boots, press when prompted. 2. After the Fast!Util program loads, the display will depend on whether there are multiple IBM FAStT Adapters installed. If there are multiple IBM FAStT Adapters, a list of addresses occupied by those Host Adapters will appear. Using the arrow keys, select the desired adapter and press ENTER. The Fast!Util Options menu will then appear. For further information refer to the IBM FAStT Host Bus Adapter publication. ======================================================================= 5.0 Download the IBM FAStT Host Adapter driver source code for Linux and IBM FAStT_MSJ for Linux from the IBM Support website. ------------------------------------------------------------------------ Download the IBM FAStT Host Adapter driver source code for Linux from the IBM TotalStorage Support website. If prompted "What would you like to do with this file?" choose "Save this file to disk". Insert a blank formatted diskette and download to the diskette directly. This file will be in the .tgz file compression format. Download the IBM FAStT_MSJ for Linux from the IBM Support website. If prompted "What would you like to do with this file?" choose "Save this file to disk". This file will need to be saved to your hard disk drive. This file will be in the .tgz file compression format. ======================================================================= 6.0 Building a Driver from the Source Code ------------------------------------------ From the source code, you can build a qla2xxx.ko and a qla2300.ko, qla6312.ko, or qla6322.ko for your host system, and load the driver manually or automatically using a RAMDISK image during system boot time. 6.1 Install the kernel source, header, and ncurses packages ----------------------------------------------------------- 1. Install the kernel-source RPM files for the supported kernel. These files are available from your Linux vendor or your OS installation CD set. Also, be sure that the ncurses packages and the gcc packages have been installed. Kernel installation instructions for different linux versions may vary. Please refer to your linux vendor documentation for procedures to update the kernel source code for your linux version. # rpm -iv kernel-source*.rpm (SLES9) # rpm -iv kernel-syms*.rpm (SLES9) # rpm -iv kernel-devel*.rpm (RH4) # rpm -iv kernel-utils*.rpm # rpm -iv ncurses*.rpm # rpm -iv ncurses-devel*.rpm # rpm -iv gcc*.rpm # rpm -iv libgcc*.rpm Verify that kernel-source, ncurses and gcc RPMS are installed. # rpm -qa | grep kernel # rpm -qa | grep ncurses # rpm -qa | grep gcc 6.2 Install the qlogic driver source code for the IBM FAStT Host Adapters ------------------------------------------------------------------------- 1. Using the adapter driver diskette you created in Section 5, copy the qla2xxx-v8.00.02-dist.tgz file to /qla2xxx-v8.00.02. Follow these steps: # mkdir /qla2xxx-v8.00.02 # cd /qla2xxx-v8.00.02 # mount /mnt/floppy # cp /mnt/floppy/*.tgz . (the period at the end is required) # tar -xvzf *.tgz This will create a folder named "qla2xxx-v8.00.02" and extract out the following files to the "qla2xxx-v8.00.02" folder: drvrinstall Script file to copy driver source files included in the driver source tgz file. qla2xxx-src-.tgz Compressed binary distribution file for driver sources. This file is the same type of driver source tgz file as the ones used for distributing earlier versions of the QLA2XXX drivers. libinstall Script file to install/setup HBA API library. libremove Script file to remove HBA API library. qlapi--rel.tgz Compressed binary distribution file for API library. 2. Execute the following commands: # cd qlogic # ./drvrsetup (this will extract the driver source files to the current directory) # cd qla2xxx-8.00.02 6.3 Building the device driver ----------------------------------------------------------- This section makes extensive use of the build.sh script located in driver source (extras/build.sh). This script currently supports driver compilation (installation and updates) on SLES9 and RedHat4 distributions on four main hardware platforms: (x86, x86_64, ia64 and ppc64). The build.sh script supports for following directives: # ./extras/build.sh Build the driver sources based on the standard 2.6 kernel build environment. # ./extras/build.sh clean Clean driver source directory of all build files (i.e. *.ko, *.o, etc). # ./extras/build.sh new Rebuild the driver sources from scratch. This is essentially a shortcut for: # ./build.sh clean # ./build.sh # ./extras/build.sh install Build and install the driver module files. This command performs the following: 1. Builds the driver .ko files. 2. Copies the .ko files to the appropriate /lib/modules/... directory. 3. Adds the appropriate directive in the modprobe.conf[.local] to remove the qla2xxx_conf module when the qla2xxx modules in unloaded. 4. Updates the newly built qla2xxx_conf.ko module with any previously saved data in /etc/qla2xxx.conf. NOTE: For SLES9 distributions the "modprobe.conf.local" file is used. For RedHat4 distributions the "modprobe.conf" file is used. For Suse Linux Enterprise Server 9 (SLES9) Distribution: You will need to modify the /etc/sysconfig/kernel file to specify which modules will be added during initrd creation. NOTE: Please ensure the conf module is listed before the actual driver module. All "qla" modules should be added to the end of the INITRD_MODULES string. For example: INITRD_MODULES="aic7xxx qla2xxx_conf qla2xxx qla2300" or INITRD_MODULES="aic7xxx qla2xxx_conf qla2xxx qla2200" For RedHat 4 Advanced Server Distribution: You will need to modify the /etc/modprobe.conf file to load the correct device driver modules. Add the following lines to the /etc/modules.conf file depending on the type of adapters you have in your system. alias scsi_hostadapter0 qla2xxx_conf alias scsi_hostadapter1 qla2xxx alias scsi_hostadapter2 qla2200 or alias scsi_hostadapter0 qla2xxx_conf alias scsi_hostadapter1 qla2xxx alias scsi_hostadapter2 qla2300 ======================================================================= 7.0 Load and configure the driver --------------------------------- 7.1 Enable more than 1 scsi device per adapter ---------------------------------------------- Support for multiple LUNs can be configured in one of three ways. Currently, the maximum number of LUNs that can be scanned for each device is 256. (1) The kernel must be configured to have multiple LUN support enabled in order for non-zero LUNs to be configured and accessible. Use "make menuconfig" to build a kernel which has the option under SCSI Support enabled to probe all LUNs on SCSI devices. NOTE: If you have multiple adapters, set max_scsi_luns to the largest number of LUNs supported by any one of these adapters. (2) If the SCSI Mid-Layer is compiled in the kernel, the boot loader can be configured to scan for multiple LUNs each time the system boots. On IA-32 and AMD-64 ------------------- For RedHat Distribution : a) Append the max_scsi_luns parameters to each of the kernel images listed in the /etc/grub.conf file. For example: kernel /vmlinux-2.6.9-5 ro root=/dev/hda2 max_scsi_luns=256 b) Reboot the system. For SuSE Distribution : a) Append the max_scsi_luns parameters to each of the kernel images listed in the /boot/grub/menu.lst file. For example: kernel (hd0,1)/boot/vmlinuz root=/dev/hda2 max_scsi_luns=256 b) Reboot the system. On IA-64 -------- For RedHat Distribution : a) Add the following line to each of the kernel images listed in the /boot/efi/efi/redhat/elilo.conf append="max_scsi_luns=256" b) Reboot the system. For SuSE Distribution : a) Add the following line to each of the kernel images listed in the /boot/efi/SuSE/elilo.conf append="max_scsi_luns=256" b) Reboot the system. (3) If the SCSI Mid-Layer is compiled as a module, add the following line to the /etc/modprobe.conf[.local] file to scan for multiple LUNs at each boot: options scsi_mod max_scsi_luns=256 Refer to your Linux distribution documentation to verify the steps to rebuild the boot image and ramdisk for your distribution. Section 7.5 of this readme does cover some of the steps required to rebuild your initrd image. 7.2 Install FAStT_MSJ --------------------- If FAStT_MSJ for Linux is not currently installed you will need to Install it now. FAStT_MSJ and the qlremote agent are needed to configure this device driver and the host adapters for Multi-path failover. See the FAStT_MSJ publications for instructions on the installation procedure for FAStT_MSJ and the qlremote agent for Linux. FAStT_MSJ is required for proper multi-path and failover configuration. 7.3 Modify the module load time options --------------------------------------- IMPORTANT: Incorrect changes to these settings may cause driver to malfunction. This device driver has multiple load time options that will need to be added to change driver values without having to recompile the device driver. When connecting to a FAStT Storage Server, you will need to add ql2xsuspendcount=70 and ql2xretrycount=60 to your modprobe.conf.[local] options string as shown later in this section. The two entries are to be added to the file after finishing the Load Balancing procedure. IMPORTANT: Please note that the /etc/modules.conf file has been replaced with /etc/modprobe.conf.local in SLES9 and /etc/modprobe.conf in RH4. Note: For FAStT FC HBA drivers version earlier 7.00.61-fo, the ql2xsuspendcount is set to 40 instead of 70 as indicated. The change was made to accommodate longer delay time of SATA drives. The driver gets its parameters from the command line itself or from modprobe 'option' directive found in the modprobe.conf[.local] file. The parameters are in simple =value format, i.e. ql2xfailover=1. Where is one of the following option parameters: Usage: insmod qla2xxx.ko =value ql2xfailover - This parameter defines whether failover mode is enabled or disabled. 0 to disable; 1 to enable. ql2xmaxqdepth - Maximum queue depth to report for target devices. ql2xretrycount - Maximum number of mid-layer retries allowed for a command. The default value in non-failover mode is 20, for failover mode the default is 30. ql2xsuspendcount - Number of 6-second suspend iterations to perform while a target returns a status. The default is 10 iterations. which is equal to 60 seconds. This value is required to be modified to 40 for the FAStT Storage Servers. displayConfig - If set to '1' then display the configuration used in /etc/modprobe.conf.[local]. max_srbs - Maximum number of simultaneous commands allowed for an HBA. The default value is set to 4096 in the driver source. A comprehensive list of parameters can be found with the following command line: # /sbin/modinfo qla2xxx.ko These values can be changed in the modprobe.conf.local file as part of the options string created by FAStT_MSJ. For example, you could add the following into your options string in the /etc/modprobe.conf.local file. ql2xretrycount=60 and ql2xsuspendcount=70 as shown below. "options qla2xxx ql2xfailover=1 ConfigRequired=1 ql2xretrycount=60 ql2xsuspendcount=70 7.4 Load the Driver Manually using INSMOD or MODPROBE ----------------------------------------------------------- Before loading the driver manually, first build the driver binary from the driver source files as described in sections 5.1. To load the driver directly from the local build directory, type the following in order: # insmod qla2xxx_conf.ko ; insmod qla2xxx.ko ; insmod qla2300.ko To load the driver using modprobe: 1. Install the driver modules (*.ko) files to the appropriate kernel module directory: # ./extras/build.sh install 2. Type the following to load the driver for qla2xxx HBAs: # modprobe -v qla2xxx_conf # modprobe -v qla2300 The modprobe -v qla2300 command will automatically load qla2xxx.ko the component. To unload the driver using modprobe: 1. # modprobe -r qla2300 This will unload qla2300.ko and qla2xxx.ko modules. 2. # modprobe -r qla2xxx_conf This will unload qla2xxx_conf.ko Verify that the driver is loaded correctly. # lsmod (This command will display the loaded modules. qla2xxx_conf, qla2xxx and qla2300 should be in the loaded modules listing.) # cd /proc/scsi/qla2xxx # cat 1 (Displays driver information for Adapter 1) 7.5 Loading the Driver using a ramdisk image -------------------------------------------- 1. Linux may detect new adapters during the system boot, if the adapters are configured during the boot process. For Suse Linux Enterprise Server 9 (SLES9) Distribution: You will need to modify the /etc/sysconfig/kernel file to specify which modules will be added during initrd creation. NOTE: Please ensure the conf module is listed before the actual driver module. All "qla" modules should be added to the end of the INITRD_MODULES string. For example: INITRD_MODULES="aic7xxx qla2xxx_conf qla2xxx qla2300" or INITRD_MODULES="aic7xxx qla2xxx_conf qla2xxx qla2200" For RedHat 4 Advanced Server Distribution: You will need to modify the /etc/modprobe.conf file to load the correct device driver modules. Add the following lines to the /etc/modules.conf file depending on the type of adapters you have in your system. alias scsi_hostadapter0 qla2xxx_conf alias scsi_hostadapter1 qla2xxx alias scsi_hostadapter2 qla2200 or alias scsi_hostadapter0 qla2xxx_conf alias scsi_hostadapter1 qla2xxx alias scsi_hostadapter2 qla2300 2. You will then need to run depmod from the command prompt to update your /lib/modules/KERNEL_VERSION/modules.dep file with the information added in your /etc/modprobe.conf.[local] file. # depmod -a 3. You will now need to load the device driver. To load the driver manually, type the following command: # modprobe -v qla2xxx_conf # modprobe -v qla2300 The modprobe -v qla2300 command will automatically load qla2xxx.ko the component. 4. Type the following command: For RedHat Distribution: # mkinitrd -f NOTE: This step will overwrite the original ramdisk image file if executed within the /boot directory. Specify a unique ramdisk image name to preserve the original ramdisk image. On IA-32 and AMD-64 ------------------- - Copy the newly built file to /boot. On IA-64 -------- - Copy the newly built file to /boot/efi/efi/redhat For SLES9 Distribution: # /sbin/mk_initrd By default, the RAMDISK images created are: /boot/initrd /boot/initrd.suse NOTE: This step will overwrite the original ramdisk image file. To preserve the original ramdisk image specify a unique ramdisk image name as follows: # /sbin/mk_initrd -k -i 5. Configure the boot loader with the new RAMDISK image. For RedHat4: Add "initrd=/boot/" in /etc/grub.conf under one of the kernel entries to use the RAMDISK image. For SLES9: Add "initrd=/boot/" in /boot/grub/menu.lst under one of the kernel entries to use the RAMDISK image. 7. Reboot the system, the qla2xxx and qla2300 driver will be loaded by the ramdisk image at boot time. 8. You will now need to configure your device driver for Multi- path I/O using FAStT_MSJ and the qlremote agent. Refer to the FAStT_MSJ publication and readme.txt for these instructions. 9. After the system is configured for multi-path I/O steps 3 - 7 above need to be repeated so that during system boot the proper driver configuration is loaded. 7.6 Rebuilding the ramdisk image after configuration changes ------------------------------------------------------------ To rebuild your ramdisk image after adding a new LUN or other configuration changes these steps may be followed: 1. To unload the driver using modprobe: 1. # modprobe -r qla2300 This will unload qla2300.ko and qla2xxx.ko modules. 2. # modprobe -r qla2xxx_conf This will unload qla2xxx_conf.ko 2. Delete the options string in /etc/modprobe.conf.[local] that is added by the FAStT MSJ load-balancing process. This is typically the last line in the /etc/modprobe.conf.[local] file: "options qla2xxx ConfigRequired=1 3. In the terminal windows type: depmod -a 4. Reload the qla2xxx and qla2300 device drivers typing: # modprobe -v qla2xxx_conf # modprobe -v qla2300 The modprobe -v qla2300 command will automatically load qla2xxx.ko the component. 5. Run the SMclient to redistribute the logical volumes. 6. Open a terminal windows and run: qlremote 7. Open another terminal window and run: /usr/FAStT_MSJ 8. Connect to local host, configure LUNs to match the preferred paths shown in IBM FAStT Storage manager. Running Storage Subsystem->Profile in the Subsystem Management window will display the world-wide port name for each controller. Ensure that the preferred path in the FAStT_MSJ LUN configuration window matches the controller assignment in IBM FAStT Storage manager. 9. Apply the configuration and exit MSJ. 10. Switch to the terminal window that has qlremote running and to stop qlremote. 11. In the terminal windows type: depmod -a 12. Unload the qla2xxx and qla2300 driver and then reload the driver using modprobe so the driver will pull in the new option string that was added by FAStT_MSJ. 13. Type the following command: For RedHat Distribution: # mkinitrd -f NOTE: This step will overwrite the original ramdisk image file if executed within the /boot directory. Specify a unique ramdisk image name to preserve the original ramdisk image. On IA-32 and AMD-64 ------------------- - Copy the newly built file to /boot. On IA-64 -------- - Copy the newly built file to /boot/efi/efi/redhat For SLES9 Distribution: # /sbin/mk_initrd By default, the RAMDISK images created are: /boot/initrd /boot/initrd.suse NOTE: This step will overwrite the original ramdisk image file. To preserve the original ramdisk image specify a unique ramdisk image name as follows: # /sbin/mk_initrd -k -i 14. Reboot the system to the ram disk image just created. ======================================================================= 8.0 Failover Support --------------------- 8.1 How to enable the Failover support in the Driver ----------------------------------------------------- Failover support can be enabled in the qla2xxx driver by loading the driver using the ql2xfailover module parameter: # insmod qla2xxx.ko ql2xfailover=1 ; insmod qla2300.ko Non-failover support can also be enabled in the qla2xxx driver by loading the driver using the ql2xfailover module parameter: # insmod qla2xxx.ko ql2xfailover=0 ; insmod qla2300.ko These values can be changed in the /etc/modprobe.conf.[local] file as part of the options string created by FAStT_MSJ. For example, you could add the following into your options string in the /etc/modprobe.conf.[local] file, ql2xfailover=1 as shown below. "options qla2xxx ql2xfailover=1 ConfigRequired=1 ql2xretrycount=60 ql2xsuspendcount=70 8.2 Configuration Changes Made via FAStT MSJ -------------------------------------------- 1. LUN Masking For the new LUN masking configuration to take effect, the driver must be reloaded. The following is an example of the sequence of actions to take: - Load the driver: modprobe - Load the qlremote agent. - Start the GUI and connect it to the destination system. - Make LUN masking changes. - Disconnect the host from GUI and stop qlremote agent. - Unload the driver: modprobe -r - Reload the driver: modprobe - Load qlremote agent again. - Start the GUI and connect it to the destination system. Now you should see the updated LUN masking configuration. Please note that when using modprobe to load the driver, the length of the option line specified in /etc/modprobe.conf.local file has a limit of 2K characters. Any longer option line will cause a string overflow error from modprobe. 8.3 Persistent Binding ---------------------- The Persistent Binding information consists of some adapter parameter entries along with some target entries. However, the Linux entries have been shorten to save space on the command line. Currently, there is no limit on the size of the command line when using modprobe. But, if you embedded the driver in the kernel you are using lilo that has a string size limitation. Persistent Binding can be specified in two ways. Manually or using FAStT MSJ. We recommend using FAStT MSJ for ease of use. The following is the procedure to manually add persistent binding commands: The driver displays the current configuration when the displayConfig command line option is specified. The persistent binding configuration is found in /var/log/messages file. It prints the configuration information in the format required by the driver. The best way to extract configuration messages is to use grep and direct the output to a file. You need to remove the Linux timestamp at the beginning of each message and combine them together on single line. For example #insmod qla2300.ko displayConfig=1 #grep "scsi-qla" /var/log/messages > /tmp/info.cfg The format of the persistent binding commands is as follows: Device descriptions scsi-qla<#>-adapter-port=; The designated by qla<#>, where the <#> is the adapter instance number. The parameter specifies the FC port name to be used for the adapter. where is the FC port name value in hexa- decimal format. If this entry is not specified in the conf file, the default value is the adapter's port name as saved in the NVRAM. Example: scsi-qla00-adapter-port=210000e08b01158d\; host adapter instance 0 has a portname of 210000e08b01158d scsi-qla<#1>-tgt-<#2>-di-<#3>-node=; This parameter associates the specified with the SCSI target ID value specified by <#2> and a device id value specified by <#3>. where type is the FC node name of the device, and <#2> is the SCSI target ID to be assigned to the device and <#3> is the device unique id. Where <#1> Specifies the adapter instance number <#2> Specifies the SCSI ID of Target <#3> Specifies the path/device id scsi-qla<#1>-tgt-<#2>-di-<#3>-port=; This parameter associates the specified with the SCSI target ID value specified by <#2> and a device id value specified by <#3>. where type is the FC port Where <#1> Specifies the adapter instance number <#2> Specifies the SCSI ID of Target <#3> Specifies the path/device id (always 0 for non-failover) name of the device, and <#2> is the SCSI target ID to be assigned to the device and <#3> is the device unique id. scsi-qla<#1>-tgt-<#2>-di-<#3>-disabled=<256 bit mask>; This parameter associates the specified <256 bit mask> with the SCSI target ID value specified by <#2> and a device id value specified by <#3>. Where <#1> Specifies the adapter instance number <#2> Specifies the SCSI ID of Target <#3> Specifies the path/device id <256 bit mask> msb lsb 000000000000000000000000000000000000000000000000000000000000000F the mask above will make the first four luns, 3, 2, 1, and 0 of a given Target disabled on that target/path. This mask specification is heavily type checked to be a sequence of 64 hex digits. 8.4 Configuration Data ---------------------- 8.4.1 Limitations with /etc/modprobe.conf.local ----------------------------------------------- Due to size constraints inherent in the user-space applications which load kernel modules, the total amount of configuration data that could be passed via modprobe.conf.local by the modprobe application was around 4096 bytes of information (with minor tuning of the modutil package). Of course, as densities of SANs increase, larger configuration spaces are needed to accommodate the information. In general, the following formula can be used to compute an approximate size in bytes of the configuration data needed to store information pertaining to 'M' HBAs and 'N' targets/device paths: 75 + 42*M + 381*N Plugging in values for common configurations returns some sample results: 2 same type HBAs - (75 + 2*42 == 159) ---------------- 1 target - 75+2*42+381*1 = 540 bytes 2 targets - 75+2*42+381*2 = 921 bytes 3 targets - 75+2*42+381*3 = 1302 bytes 4 targets - 75+2*42+381*4 = 1683 bytes 5 targets - 75+2*42+381*5 = 2064 bytes ... 3 same type HBAs (75 + 3*42 == 201) ---------------- 1 target - 75+2*42+381*1 = 582 bytes 2 targets - 75+2*42+381*2 = 963 bytes 3 targets - 75+2*42+381*3 = 1344 bytes 4 targets - 75+2*42+381*4 = 1725 bytes 5 targets - 75+2*42+381*5 = 2086 bytes ... Please note, a target in this case does not always indicate a distinct piece of storage -- it could represent 'n' paths to the same storage, as is the case with failover in a true-cloud configuration. As an example, the configuration data size needed for two storage devices with four paths (via two HBAs) to each storage (4*2 paths) would need approximately 3200 bytes (75 + 42*2 + 381*8) of configuration space. 8.4.2 QLA_OPTS as an alternative -------------------------------- Modutil (namely modprobe) loads 'option' data present in the modprobe.conf.local file by first loading the module, parsing the 'options' directive of the newly loaded module for parameters (parameters are simple key=value directives, i.e. ql2xfailover=1), for each key, scan the memory area where the module was loaded for the location of the 'key' parameter, and finally, writing the key's 'value' directly into the pre-defined memory space. This basic mechanism is similar in nature to the mechanism employed by the QLA_OPTS application, but does not suffer from the relatively small size constraints within the modutil package. There are two important difference between the modutil and QLA_OPTS mechanism: 1) Configuration data is read from a configuration file in /etc/ with a name based on the ISP type: Configuration File Module name ------------------ ----------- /etc/qla2xxx.conf qla2300 2) Option values are written directly (branded) to the module's '.o' binary file. Approximately 300K of configuration space has been pre-allocated within the driver. 8.4.3 Compatibility with FAStT_MSJ (FAStT Management Suite Java) ---------------------------------------------------------------- QLA_OPTS will work seamlessly with updated FAStT_MSJ applications. Originally, when a FAStT_MSJ application would save a configuration the corresponding data would be written to the 'options' section of the modprobe.conf.local file in an form similar to the following: ql2xopts=scsi-qla0-adapter-port=210000e08b000000\;scsi-qla0-tgt-1-di- 0-node=20000020371682e7\;scsi-qla0-tgt-1-di-0-port=21000020371682e7\ ;scsi-qla0-tgt-1-di-0-pid=0000e2\;scsi-qla0-tgt-1-di-0-control=00\; Now, the information is written to the appropriate qla2xxx.conf file in /etc and then branded to the binary file of the currently executing module (qla2300.ko): /lib/modules//kernel/drivers/scsi Where CURRENT_KERNEL_VERSION is the the result of the command 'uname -r'. This operation is performed automatically and requires no user intervention. Of course, if the driver was loaded from an initrd image, as before with the modprobe.conf.local interface, the user would be required to rebuild the initrd image after updating a configuration. 8.4.4 Persisting configuration data while upgrading drivers ----------------------------------------------------------- To maintain configuration data during a driver update (e.g. 7.00.61b11 -> 8.00.02), the 'install' directive during the make process will read the proper qla2xxx.conf file and write the previous configuration into the newly built driver binaries. When updating device drivers from earlier versions you will be required to have the latest version of FAStT_MSJ installed and then re-save your failover configuration so that the qla2xxx.conf file can be written out as needed. This is required due to the change in method that the persistent configuration data is read by the device driver. 8.5 Hot Add new LUNs -------------------- This version supports hot adding new luns, etc. Please see text below on how to perform the lun hot add procedure. The latest driver-v8.00.02 has the mechanism which allows the user to force the driver to do re-scan of the devices to allow a new device to be added. This triggers the driver to initiate lun discovery process. To do this from the command line: #echo "scsi-qlascan" > /proc/scsi// (qlogic driver will re-scan) Where can be either one : qla2xxx/qla2300 is the instance number of the HBA. Once that has been done , user then can force the scsi mid layer to do its own scan and build the device table entry for the new device: # echo "scsi add-single-device 0 1 2 3" >/proc/scsi/scsi (scsi mid layer will re-scan) with "0 1 2 3" replaced by your "Host Channel Id Lun". The scanning has to be done in the above mentioned order. First the driver (qla2xxx/qla2300 driver etc) and then the Linux scsi mid layer. ======================================================================= 9.0 Limitations --------------- * Failover adapters must be of the same type. An IBM FAStT Host Adapter and an IBM FAStT FC-2 Host Bus Adapter cannot be a failover pair. * The qlremote agent must not be loaded when disk I/O's are running. * Lun masking is not functional in this release. * In the Storage Manager Client Storage Partitioning must be enabled with the host type of 'linux' selected and the UTM must be removed from the 'Linux' storage partition. Otherwise the correct drive information is not presented to the Linux operating system. * Every time a change is made to your configuration you will need to generate a new boot image. This includes changes to LUN ownership and adding LUNs to the storage subsystem. * With sequential LUN numbering, if one LUN is missing due to a controller failure the system associates the wrong SCSI device entry with the LUNs after the failed LUN. The result could be that the wrong LUN could be mounted to an incorrect mount point. You should always be aware of this because it can also be caused by adding a LUN out of sequence. You should ensure that the LUNs are number in sequence starting from 0 in storage partitioning. If there is a gap (for example LUN 0, LUN1, LUN3) Linux will quit probing at the gap where LUN 2 should be and all of the following LUNs will be missing. * When updating Firmware or NVSRAM with high disk activity a controller may become unresponsive during the update. IBM recommends that disk I/O be stopped during these code updates. * When adding logical drives to the storage subsystem with the SMclient you will need to unload the device driver and then modprobe the driver to scan the new logical drives. You will then need to reconfigure using FAStT_MSJ and the qlremote agent so these disks will be available for the OS to utilize. After the driver is reconfigured with FAStT_MSJ you will need to repeat section 7.6 in this readme to rebuild your boot image with the new option string created by FAStT_MSJ. * If a configuration change has been made using FAStT_MSJ and you are not able to APPLY or SAVE the configuration you will need to check the options string in your modprobe.conf.local file. On occasion this string may get corrupted and need to be deleted before your new configuration will be saved properly. See section 7.6 above. ======================================================================= 9.1 Proc Filesystem Support --------------------------- The /proc filesystem for the driver can be found in the /proc/scsi/qla2xxx/ directory. This directory contains a file for each IBM FAStT Host Adapter in the system. Each file will present information about the adapter and transfer statistics for each discovered LUN. ======================================================================= 10.0 Driver file Contents ------------------------- qla2xxx-src-v8.00.02.tgz ql2100_fw.c ql6322.c qla_dbg.h qla_inioct.c qla_rscn.c ql2200.c ql6322_fw.c qla_def.h qla_init.c qla_settings.h Kconfig ql2200_fw.c qla2xxx_conf.c qla_devtbl.h qla_inline.h qla_sup.c Makefile ql2300.c qla_32ioctl.c qla_fo.c qla_iocb.c qla_version.h exioct.h ql2300_fw.c qla_32ioctl.h qla_fo.h qla_isr.c qla_xioct.c exioctln.h ql2322.c qla_cfg.c qla_foln.c qla_listops.h qlfo.h extras ql2322_fw.c qla_cfg.h qla_foln.h qla_mbx.c qlfolimits.h inioct.h ql6312.c qla_cfgln.c qla_gbl.h qla_opts.h qlfoln.h ql2100.c ql6312_fw.c qla_dbg.c qla_gs.c qla_os.c revision.notes ======================================================================= 11.0 WEB Sites and Support Phone Number --------------------------------------- 11.1 IBM TotalStorage™ Disk Storage Systems Technical Support web site: http://www.ibm.com/servers/storage/support/disk/ 11.2 IBM TotalStorage™ Marketing Web Site: http://www.ibm.com/servers/storage/disk 11.3 If you have any questions about this update, or problem applying the update go to the following HelpCenter World Telephone Numbers URL: http://www.ibm.com/planetwide ======================================================================= 12.0 Trademarks and Notices --------------------------- The following terms are trademarks of the IBM Corporation in the United States or other countries or both: HelpCenter IBM eServer IBM xSeries IBM TotalStorage UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Linux is a registered trademark of Linus Torvalds. Other company, product, and service names may be trademarks or service marks of others. ======================================================================= 13.0 Disclaimer --------------- THIS DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. IBM DISCLAIMS ALL WARRANTIES, WHETHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE AND MERCHANTABILITY WITH RESPECT TO THE INFORMATION IN THIS DOCUMENT. BY FURNISHING THIS DOCUMENT, IBM GRANTS NO LICENSES TO ANY PATENTS OR COPYRIGHTS. Note to U.S. Government Users -- Documentation related to restricted rights -- Use, duplication or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation.