Found excellent solution about the nasty problem described in subject and I’m reporting it here for the records.
Short description of the issue I’ve written on OpenFiler Forum:
I’m using Openfiler 2.3 as iSCSI server for 2 ESXi 3.5.
The system is configured with one 80 Gb internal disk where I defined a 72 GB partition as iSCSI volume, plus a 500 MB external disk connected via USB2 defined as another iSCSI volume.
Both are formatted as VMFS and accesses simultaneously on a Gbps network by the 2 ESXs.
Everything works fine at initial setup, until you reboot.
Then what happens is:
1) in boot process you see the following errors:
iSCSI Enterprise Target Software – version 0.4.16
iscsi_trgt: Registered io type fileio
iscsi_trgt: Registered io type blockio
iscsi_trgt: Registered io type nullio
iscsi_trgt: blockio_open_path(166) Can’t open device /dev/raidusb2/ofraid, error -2
iscsi_trgt: blockio_attach(340) Error attaching Lun 1 to Target iqn.2006-01.com.openfiler:tsn.bebb933b3331
2) ESX servers are still seeing the OpenFiler internal disk volume but not anymore the external USB disk.
Check in OF Administration interface, I’ve noticed that the USB2 disk is detected at boot and appears in the “Status” screen but unfortunately it does not appear anymore in “Volumes” screen as a volume group.
Bizarre thing is that if I click on iSCSI target from Volume screen, then I re-click on Volume screen the Volume Group appears back.
Despite this, iSCSI scan made by ESX is still detecting OF Internal Volume Group but not anymore the external USB disk.
Only way to fix it is to delete the volume group and recreate it from scratch, losing all data on it.
There is a solution I’ve found in a long post, therefore I’ve decided to sum it up here in five easy steps:
- from /root directory on standard OpenFiler installation, issue the following command (all on one line):
for file in `dir /lib/modules`; do mkinitrd --allow-missing --preload ohci-hcd --preload ehci-hcd --with=usb-storage --with=async_tx --with=xor --with=async_xor --with=async_memcpy --with=raid456 -f /boot/initrd-$file.img $file; done
- To retrieve your volume structure you have to create a command (I named it remake_volinfo) on OpenFiler containing the following lines:
#!/bin/bash # strip the /mnt lines from fstab as we will be rebuilding them grep /mnt /etc/fstab -v > _fstab # create the new volumes.xml file echo -e "<?xml version=\"1.0\" ?>\n<volumes>" > _volumes.xml # find all logical volumes and loop for i in `lvdisplay | grep "LV Name" | sed 's/[^\/]*//'`; do fstype=`vol_id $i -t 2> /dev/null`; mntpoint=`echo $i | sed 's/\/dev\//\/mnt\//'`/ vgname=`echo $i | cut -d '/' -f3` volid=`echo $i | cut -d '/' -f4` args=""; if [ "$fstype" == "" ]; then # assume iscsi since filesystem is unknown fstype="iscsi" mntpoint="" fi if [ $fstype == ext3 ] ; then args=",acl,user_xattr" fi if [ $fstype == reiserfs ] ; then args=",acl" fi if [ $fstype != "iscsi" ]; then echo "$i $mntpoint $fstype defaults,usrquota,grpquota$args 0 0" >> _fstab echo " <volume id=\"$volid\" name=\"$volid\" mountpoint=\"$mntpoint\" vg=\"$vgname\" fstype=\"$fstype\" />" >> _volumes.xml echo "Mounting $mntpoint" mkdir -p $mntpoint > /dev/null 2> /dev/null umount $mntpoint 2> /dev/null mount $mntpoint else echo "$i - assuming iSCSI" echo " <volume id=\"$volid\" name=\"$volid\" mountpoint=\"\" vg=\"$vgname\" fstype=\"$fstype\" incominguser=\"\" incomingpassword=\"\" outgoinguser=\"\" outgoingpassword=\"\" />" >> _volumes.xml fi done; echo "</volumes>" >> _volumes.xml mv -f _fstab /etc/fstab mv -f _volumes.xml /opt/openfiler/etc/volumes.xml chown openfiler.openfiler /opt/openfiler/etc/volumes.xml
- Chmod it to 700
- Run command
Once you’ve done this, your USB disk volume groups will survive the reboots and you’ll have back your volume structure.