Udev label removes corresponding fdisk, sfdisk or lsvdev entry

TBC Cosmo · 03-02-2009, 12:11 PM

I'm curious about the behavior where any udev labeled device causes that corresponding listing to disappear from fdisk, sfdisk, or in the case of RDAC, lsvdev.

I have seen this on both EMC clariion and Sun Storagetek/Engenio 6540 arrays.

We use RHEL5.1 and udev to create persistent labels for Oracle devices.

Example udev rule:

Code:

# cat /etc/udev/rules.d/55-oracle-naming.rules
# Configure persistent, user-defined Oracle Clusterware device file names
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b8000476694000004704974f509", NAME="ocr1", OWNER="root", GROUP="oinstall", MODE="0640"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b80004793a2000004ae4974e92d", NAME="ocr2", OWNER="root", GROUP="oinstall", MODE="0640"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b8000478034000003e04974f6f0", NAME="ocr3", OWNER="root", GROUP="oinstall", MODE="0640"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b80004793da000004904974e9b1", NAME="vot1", OWNER="oracle", GROUP="oinstall", MODE="0640"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b8000476694000004734974f778", NAME="vot2", OWNER="oracle", GROUP="oinstall", MODE="0640"
KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id", RESULT=="3600a0b80004793a2000004b14974ea59", NAME="vot3", OWNER="oracle", GROUP="oinstall", MODE="0640"

So the result in fdisk -l etc are that these 6 LUNS are missing, though the block devices are always accessible via the label.

I'm comfortable that we can always reference the /dev/ocr* or /dev/vot* devices, I'm just looking for how to explain this behavior. Is there some mechanism under the hood causing it? Is it so someone doesn't do things with those fdisk listed devices and inadvertently wipe out data on udev labeled ones?

Thx

syg00 · 03-03-2009, 07:54 AM

What does "cat /proc/partitions" give ?.

TBC Cosmo · 03-03-2009, 08:39 AM

Code:

[root@racmeta1 ~]# diff -y fdisk.txt procpartitions.txt
sdb                                                             sdb
sdc                                                             sdc
sdd                                                             sdd
sde                                                             sde
sdf                                                             sdf
sdg                                                             sdg
sdh                                                             sdh
sdi                                                             sdi
sdj                                                             sdj
sdk                                                             sdk
sdl                                                             sdl
sdm                                                             sdm
sdn                                                             sdn
sdo                                                             sdo
sdp                                                             sdp
sdq                                                             sdq
sdr                                                             sdr
sds                                                             sds
                                                              > sdt
                                                              > sdu
sdv                                                             sdv
                                                              > sdw
sdx                                                             sdx
                                                              > sdy
sdz                                                             sdz
                                                              > sdaa
sdab                                                            sdab
                                                              > sdac
sdad                                                            sdad
sdae                                                            sdae
sdaf                                                            sdaf
sdag                                                            sdag
sdah                                                            sdah
sdai                                                            sdai
sdaj                                                            sdaj
sdak                                                            sdak
sdal                                                            sdal
sdam                                                            sdam
sdan                                                            sdan
sdao                                                            sdao
sdap                                                            sdap
sdaq                                                            sdaq
sdar                                                            sdar
sdas                                                            sdas
sdat                                                            sdat
sdau                                                            sdau
[root@racmeta1 ~]# scsi_id -s /block/sdt
3600a0b80004793a2000004ae4974e92d
[root@racmeta1 ~]# scsi_id -s /block/sdu
3600a0b80004793da000004904974e9b1
[root@racmeta1 ~]# scsi_id -s /block/sdw
3600a0b80004793a2000004b14974ea59
[root@racmeta1 ~]# scsi_id -s /block/sdy
3600a0b8000476694000004704974f509
[root@racmeta1 ~]# scsi_id -s /block/sdaa
3600a0b8000478034000003e04974f6f0
[root@racmeta1 ~]# scsi_id -s /block/sdac
3600a0b8000476694000004734974f778

TBC Cosmo · 03-03-2009, 08:50 AM

Code:

[root@racmeta1 ~]# for i in sdt sdu sdw sdy sdaa sdac; do dd if=/dev/$i of=/dev/null bs=1M count=10; done
dd: opening `/dev/sdt': No such file or directory
dd: opening `/dev/sdu': No such file or directory
dd: opening `/dev/sdw': No such file or directory
dd: opening `/dev/sdy': No such file or directory
dd: opening `/dev/sdaa': No such file or directory
dd: opening `/dev/sdac': No such file or directory

[root@racmeta1 ~]# ls -l /dev/ocr* /dev/vot*
brw-r----- 1 root   oinstall 65, 129 Mar  2 16:54 /dev/ocr1
brw-r----- 1 root   oinstall 65,  49 Mar  2 16:54 /dev/ocr2
brw-r----- 1 root   oinstall 65, 161 Mar  2 16:54 /dev/ocr3
brw-r----- 1 oracle oinstall 65,  65 Mar  3 09:47 /dev/vot1
brw-r----- 1 oracle oinstall 65, 193 Mar  3 09:47 /dev/vot2
brw-r----- 1 oracle oinstall 65,  97 Mar  3 09:47 /dev/vot3

[root@racmeta1 ~]# for i in `seq 1 3`; do dd if=/dev/ocr$i of=/dev/null bs=1M count=10; done
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.009242 seconds, 1.1 GB/s
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.009082 seconds, 1.2 GB/s
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.052977 seconds, 198 MB/s
[root@racmeta1 ~]# for i in `seq 1 3`; do dd if=/dev/vot$i of=/dev/null bs=1M count=10; done
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.220546 seconds, 47.5 MB/s
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.230579 seconds, 45.5 MB/s
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.236041 seconds, 44.4 MB/s

syg00 · 03-04-2009, 02:41 AM

Whilst I appreciate that you offered what you thought was useful, it isn't what I asked for nor need.

TBC Cosmo · 03-04-2009, 07:25 AM

Quote:

Originally Posted by syg00

Whilst I appreciate that you offered what you thought was useful, it isn't what I asked for nor need.

Actually it is, but I added what I know to be relevant, anticipating follow on questions. So it's not just for you to look at, it's actually for anyone else who cares to jump in and assist. But thanks anyway. Moving on...

/proc/partitions shows me the missing 6 devices from fdisk, as illustrated by the diff output. These devices are not accessible as named in /dev/. But scsi_id gives the unique ID and the udev labeled devices are accessible via their label in /dev. So back to the original question. What mechanism accounts for this behavior?

syg00 · 03-04-2009, 07:34 AM

I'd suggest you go look at the source for fdisk. I see no point in my trying to assist further.

TBC Cosmo · 03-04-2009, 07:59 AM

Quote:

Originally Posted by syg00

I'd suggest you go look at the source for fdisk. I see no point in my trying to assist further.

And in three posts you've added no value whatsoever to this thread.