[SOLVED] RHEL8 udev & partprob Error: Partition(s):1,2,3.. /dev/sdc1 have been written, but we have been unable to inform the kernel of the change

scofieldd · 10-26-2019, 04:34 AM

Hi guys ,

I was trying to add shared disks using udev in my rhel8 vm on virtualbox,but I am having a problem when applying the rules along with partprob on the partitions . Does udev configuration change in red hat 8 ?
Thx in advance

The shared disks are required for my oracle database install .
- It seems that rhel8 doesn't allow partprob to read the partition table online after its modification (only applies change in kernel and not in userland). for every disk the system keeps requesting a reboot to link these modification to the kernel (see error) :

Code:

#/sbin/partprobe /dev/sdc1
#/sbin/partprobe /dev/sdd1
#/sbin/partprobe /dev/sde1
#/sbin/partprobe /dev/sdf1

Quote:

default: ******************************************************************************
default: Configure udev. Fri Oct 25 20:02:46 UTC 2019
default: ******************************************************************************
default: Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 5
4, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 on /dev/sdc1 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should
reboot now before making further changes.
default: Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 5
4, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 on /dev/sdd1 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should
reboot now before making further changes.
default: Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 5
4, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 on /dev/sde1 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should
reboot now before making further changes.
default: Error: Partition(s) 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 5
4, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64 on /dev/sdf1 have been written, but we have been unable to inform the kernel of the change, probably because it/they are in use. As a result, the old partition(s) will remain in use. You should
reboot now before making further changes.

• udev rules :

Quote:

[root@london2 vagrant]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB10a62cf9-e1a4aafe", SYMLINK+="oracleasm/asm-disk1", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB28814cf8-c552fb1e", SYMLINK+="oracleasm/asm-disk2", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VBd90d74ee-9d444a1a", SYMLINK+="oracleasm/asm-disk3", OWNER="oracle", GROUP="dba", MODE="0660"
KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/usr/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="1ATA_VBOX_HARDDISK_VB90ac3161-58c1b131", SYMLINK+="oracleasm/asm-disk4", OWNER="oracle", GROUP="dba", MODE="0660"

• The disks :
the desired permissions are not visible

Code:

[root@london2 vagrant]# ll /dev/oracleasm/asm*
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk1 -> ../sdc
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk2 -> ../sdd
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk3 -> ../sde
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk4 -> ../sdf

• udev status :
On the other hand I also notice an error from the udev service

Quote:

[root@london2 vagrant]# systemctl status systemd-udevd.service
● systemd-udevd.service - udev Kernel Device Manager
Loaded: loaded (/usr/lib/systemd/system/systemd-udevd.service; static; vendor preset: disabled)
Active: active (running) since Sat 2019-10-26 08:18:35 UTC; 2s ago
Docs: man:systemd-udevd.service(8)
man:udev(7)
Main PID: 1958 (systemd-udevd)
Status: "Processing with 12 children at max"
Tasks: 1
Memory: 3.0M
CGroup: /system.slice/systemd-udevd.service
└─1958 /usr/lib/systemd/systemd-udevd

Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Stopped udev Kernel Device Manager.
Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Starting udev Kernel Device Manager...
Oct 26 08:18:35 london2.evilcorp.com systemd-udevd[1958]: Network interface NamePolicy= disabled on kernel command line, ignoring.
Oct 26 08:18:35 london2.evilcorp.com systemd-udevd[1958]: Error resolving user 'oracle': Bad message
Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Started udev Kernel Device Manager.

the user oracle exists

Code:

[root@london2 vagrant]# groups oracle
oracle : oinstall racdba dba oper backupdba dgdba kmdba

but even when I tried with the below commands than partprob to apply the changes It didn't work.

Code:

/sbin/partx -u  /dev/sdc
OR /sbin/hdparm -z /dev/sdc

Any insight would be really appreciated as I haven't found any resource online regarding custom udev rules issues in RHEL8

TB0ne · 10-27-2019, 11:24 AM

Quote:

Originally Posted by scofieldd

Hi guys ,
I was trying to add shared disks using udev in my rhel8 vm on virtualbox,but I am having a problem when applying the rules along with partprob on the partitions . Does udev configuration change in red hat 8 ? Thx in advance

The shared disks are required for my oracle database install. It seems that rhel8 doesn't allow partprob to read the partition table online after its modification (only applies change in kernel and not in userland). for every disk the system keeps requesting a reboot to link these modification to the kernel (see error) :

Code:

#/sbin/partprobe /dev/sdc1
#/sbin/partprobe /dev/sdd1
#/sbin/partprobe /dev/sde1
#/sbin/partprobe /dev/sdf1

• udev rules :

• The disks :
the desired permissions are not visible

Code:

[root@london2 vagrant]# ll /dev/oracleasm/asm*
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk1 -> ../sdc
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk2 -> ../sdd
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk3 -> ../sde
lrwxrwxrwx. 1 root root 6 Oct 26 07:30 /dev/oracleasm/asm-disk4 -> ../sdf

• udev status :
On the other hand I also notice an error from the udev service the user oracle exists

Code:

[root@london2 vagrant]# groups oracle
oracle : oinstall racdba dba oper backupdba dgdba kmdba

but even when I tried with the below commands than partprob to apply the changes It didn't work.

Code:

/sbin/partx -u  /dev/sdc
OR /sbin/hdparm -z /dev/sdc

Any insight would be really appreciated as I haven't found any resource online regarding custom udev rules issues in RHEL8

Well, the message clearly says to reboot before you do anything else; did you??? And RHEL with Oracle is a well-supported configuration; have you contacted either Oracle or Red Hat support? You are PAYING for those commercial products, right?

And is the oracle user a member of the plugdev group?

scofieldd · 10-27-2019, 02:14 PM

Quote:

Well, the message clearly says to reboot before you do anything else; did you???

yes and it didn't fix the problem . in the meantime why should I (new partitions aren't mounted ... partprob is supposed to read the changes and apply the udev rules along the way )

Quote:

And RHEL with Oracle is a well-supported configuration; have you contacted either Oracle or Red Hat support?

no it's a lab based on a RHEL8 server (trial) for education purpose

Quote:

You are PAYING for those commercial products, right?

not really since it's not on production but lab vms

Quote:

And is the oracle user a member of the plugdev group?

I wasn't aware it was required .
plus I read somewhere that

Quote:

The group 'plugdev' is a misguided Ubuntu-only feature to solve hotplug device
permissions. Other distributions do not need this, and should not start using
any group assignments.

I actually installed oracle products on rhel 7 (as well as rhel6) using the same scripts (to create new partitions from attached shared disks) and I didn't have any problem .

thx

TB0ne · 10-27-2019, 04:28 PM

Quote:

Originally Posted by scofieldd

yes and it didn't fix the problem . in the meantime why should I (new partitions aren't mounted ... partprob is supposed to read the changes and apply the udev rules along the way )

Why you should is covered in the change notes for RHEL8

Quote:

no it's a lab based on a RHEL8 server (trial) for education purpose

..and...

Quote:

not really since it's not on production but lab vms

And neither of those things matter; either talk to the sales reps and get trial licenses (with support), or use something else.

Quote:

I wasn't aware it was required . plus I read somewhere that I actually installed oracle products on rhel 7 (as well as rhel6) using the same scripts (to create new partitions from attached shared disks) and I didn't have any problem .
thx

And RHEL8 is NOT RHEL 7 or 6. You don't say what file-system you're using, but this is a known issue (resolved) in RHEL7, and will be for 8 if you have the updates/patches loaded (which you won't, since you're not paying, and don't have access to the online repositories).
https://access.redhat.com/solutions/199573

scofieldd · 10-27-2019, 06:01 PM

Quote:

Originally Posted by TB0ne

Why you should is covered in the change notes for RHEL8

You mean there is no way to run partprobe and apply the changes without a reboot ? in this case I don't think it's enough since the error occured after the restart.

Quote:

..and...And neither of those things matter; either talk to the sales reps and get trial licenses (with support), or use something else.

support for a simple partprobe run that went wrong in a lab

?

Quote:

And RHEL8 is NOT RHEL 7 or 6. You don't say what file-system you're using, but this is a known issue (resolved) in RHEL7

the file-system is the Oracle based one (ASM) so no need for the newly created partitions to be formatted , just mapped (via udev) so ASM can manage them .

Code:

[root@london2 vagrant]# lsblk
NAME                MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT  FSTYPE
sda                   8:0    0   32G  0 disk
+-sda1                8:1    0    1G  0 part /boot         xfs
+-sda2                8:2    0   31G  0 part               LVM2_member
  +-rhel_rhel8-root 253:0    0 28.9G  0 lvm  /             xfs
  +-rhel_rhel8-swap 253:1    0  2.1G  0 lvm  [SWAP]        swap
sdb                   8:16   0  100G  0 disk
+-sdb1                8:17   0  100G  0 part /u01
sdc                   8:32   0   20G  0 disk
+-sdc1                8:33   0   20G  0 part
sdd                   8:48   0   20G  0 disk
+-sdd1                8:49   0   20G  0 part
sde                   8:64   0   20G  0 disk
+-sde1                8:65   0   20G  0 part
sdf                   8:80   0   20G  0 disk
+-sdf1                8:81   0   20G  0 part
sr0                  11:0    1 1024M  0 rom

Code:

[root@london2 vagrant]#  fsck -N /dev/sd?1
fsck from util-linux 2.32.1
[/sbin/fsck.xfs (1) -- /boot] fsck.xfs /dev/sda1
[/sbin/fsck.xfs (2) -- /u01] fsck.xfs /dev/sdb1
[/sbin/fsck.ext2 (3) -- /dev/sdc1] fsck.ext2 /dev/sdc1
[/sbin/fsck.ext2 (4) -- /dev/sdd1] fsck.ext2 /dev/sdd1
[/sbin/fsck.ext2 (5) -- /dev/sde1] fsck.ext2 /dev/sde1
[/sbin/fsck.ext2 (6) -- /dev/sdf1] fsck.ext2 /dev/sdf1

I said I happened to already install Oracle RAC in RHEL7 where the attached disks were partitioned and partprobe ran successfully as expected (see link )

Quote:

, and will be for 8 if you have the updates/patches loaded (which you won't, since you're not paying, and don't have access to the online repositories).

I do have access to rhel 8 beta repository if that's what you want to check and have a solution to suggest

Code:

vi /etc/yum.repos.d/rhel8.repo     
[rhel-8-for-x86_64-baseos-beta-rpms]
name = Red Hat Enterprise Linux 8 for x86_64 - BaseOS Beta (RPMs)
baseurl = https://downloads.redhat.com/redhat/rhel/rhel-8-beta/baseos/x86_64/
enabled = 1
gpgcheck = 0

[rhel-8-for-x86_64-appstream-beta-rpms]
name = Red Hat Enterprise Linux 8 for x86_64 - AppStream Beta (RPMs)
baseurl = https://downloads.redhat.com/redhat/rhel/rhel-8-beta/appstream/x86_64/
enabled = 1

by the way I had read the that note before opening this thread .
[CODE]https://access.redhat.com/solutions/199573[/QUOTE]

partx didn't help since my udev rules aren't applied and the ownership of the partitions is still root.

Code:

# partx -u /dev/vda

• I thought SELINUX was the cause (it was previously enforced) but the issue is still there when i set it to permissive

• It seems that the group part of the udev rules is applied but not the owner (which may be connected to the udev service error mentioned earlier in this thread ( Error resolving user 'oracle': Bad message) .

Quote:

[root@london2 vagrant]# ll /dev/sd* |grep dba
brw-rw----. 1 root dba 8, 33 Oct 27 18:21 /dev/sdc1
brw-rw----. 1 root dba 8, 49 Oct 27 18:05 /dev/sdd1
brw-rw----. 1 root dba 8, 65 Oct 27 18:05 /dev/sde1
brw-rw----. 1 root dba 8, 81 Oct 27 18:05 /dev/sdf1

thanks

scofieldd · 11-08-2019, 11:02 AM

Following up on my issue here . I finally used partx -u on the partitions after creation .

I then got stuck on the udev rules not beeing fully applied (only the group ownership was ).

see below udev status : the error is visible on the udev service

Quote:

[root@london2 vagrant]# systemctl status systemd-udevd.service
● systemd-udevd.service - udev Kernel Device Manager
Loaded: loaded (/usr/lib/systemd/system/systemd-udevd.service; static; vendor preset: disabled)
Active: active (running) since Sat 2019-10-26 08:18:35 UTC; 2s ago
Docs: man:systemd-udevd.service(8)
man:udev(7)
Main PID: 1958 (systemd-udevd)
Status: "Processing with 12 children at max"
Tasks: 1
Memory: 3.0M
CGroup: /system.slice/systemd-udevd.service
└─1958 /usr/lib/systemd/systemd-udevd

Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Stopped udev Kernel Device Manager.
Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Starting udev Kernel Device Manager...
Oct 26 08:18:35 london2.evilcorp.com systemd-udevd[1958]: Network interface NamePolicy= disabled on kernel command line, ignoring.
Oct 26 08:18:35 london2.evilcorp.com systemd-udevd[1958]: Error resolving user 'oracle': Bad message
Oct 26 08:18:35 london2.evilcorp.com systemd[1]: Started udev Kernel Device Manager.

After many attempts I realized something was wrong with the user creation script . The vagrant box was packed with oracle user in it . I then dropped the user and recreated it during the vm provisioning .
that was it all is working fine now .

thanks

Thread can be closed