The customer’s RAC environment is Huawei’s storage, and the shared disk is /dev/sd*. At first glance, it is suspected that the multipath configuration has not been performed. Actually, the host engineer has already configured it. We can use the upadmin show vlun command. View to:
[ root@xxdb01 ~]# upadmin show vlun
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Vlun ID Disk Name Lun WWN Status Capacity Ctrl(Own/Work) Array Name Dev Lun ID No.ofPaths(Available/Total)0 sdb LUN_Oracle_400G_0000 6acb3b510041191b0de7bcdd0000000f Normal 400.00GB 0D/0D Huawei.18500V5 158/81 sdc LUN_Oracle_400G_0001 6acb3b510041191b0de7be3900000010 Normal 400.00GB 0A/0A Huawei.18500V5 168/82 sdd LUN_Oracle_400G_0002 6acb3b510041191b0de7bec100000011 Normal 400.00GB 0B/0B Huawei.18500V5 178/83 sde LUN_Oracle_400G_0003 6acb3b510041191b0de7bfc900000012 Normal 400.00GB 0C/0C Huawei.18500V5 188/84 sdf LUN_Oracle_400G_0004 6acb3b510041191b0de7c03f00000013 Normal 400.00GB 0D/0D Huawei.18500V5 198/85 sdg LUN_Oracle_400G_0005 6acb3b510041191b0de7c09e00000014 Normal 400.00GB 0A/0A Huawei.18500V5 208/86 sdh LUN_Oracle_400G_0006 6acb3b510041191b0de7c0e900000015 Normal 400.00GB 0B/0B Huawei.18500V5 218/87 sdi LUN_Oracle_400G_0007 6acb3b510041191b0de7c12e00000016 Normal 400.00GB 0C/0C Huawei.18500V5 228/88 sdj LUN_Oracle_5G_0003 6acb3b510041191b0de893a0000000da Normal 5.00GB 0C/0C Huawei.18500V5 2188/89 sdk LUN_Oracle_5G_0004 6acb3b510041191b0de8941e000000db Normal 5.00GB 0D/0D Huawei.18500V5 2198/810 sdl LUN_Oracle_5G_0005 6acb3b510041191b0de894a4000000dc Normal 5.00GB 0A/0A Huawei.18500V5 2208/811 sdm LUN_Oracle_100G_0002 6acb3b510041191b0de80f1f00000039 Normal 100.00GB 0B/0B Huawei.18500V5 578/8--------------------------------------------------------------------------------------------------------------------------------------------------------------------------[root@xxdb01 ~]#
In fact, it is also possible to use these disks directly, but considering the specifications, refer to the previous customer udev binding rules and specifications:
- - not available
KERNEL=="sd*",BUS=="scsi",PROGRAM=="/sbin/scsi_id i --whitelisted --device=/dev/$name",RESULT=="36000c29b263ed2452f80e9848bdf2fa5",NAME="asm-2g-2fa5-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
You can see that the naming method of the alias is: the last four digits of asm-lunsize-id-disk group name + number. In this way, if you encounter operations such as adding/deleting disks in the future, you can quickly help the DBA to confirm.
However, because the above udev syntax is for RHEL 6, it does not apply to CentOS 7. The corresponding syntax for 7 is:
- - ok!
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
Considering the relatively large number of disks, writing one by one is time-consuming and error-prone. When I think of the earlier years when installing RAC, I often refer to a method of maclean, which is to write a script for this work:
vi /u01/asmdisk.sh
for i in b c d e f g h i j k l m;do
echo "KERNEL==\"sd*\",SUBSYSTEM==\"block\",PROGRAM==\"/lib/udev/scsi_id -g -u -d /dev/\$name\",RESULT==\"`/lib/udev/scsi_id -g -u -d /dev/sd$i`\",SYMLINK+=\"asm-5g-xxxx-grid1\",OWNER=\"grid\",GROUP=\"asmadmin\",MODE=\"0660\""
done
Execute the script: sh /u01/asmdisk.sh, the result is:
- - script-result
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-5g-xxxx-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
Use a text editor with column editing to quickly modify the following, and then copy it to the /etc/udev/rules.d/99-oracle-asmdevices.rules configuration file:
- - modify
[ root@xxdb01 ~]# cat /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bcdd0000000f",SYMLINK+="asm-400g-000f-data1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7be3900000010",SYMLINK+="asm-400g-0010-data2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bec100000011",SYMLINK+="asm-400g-0011-data3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7bfc900000012",SYMLINK+="asm-400g-0012-data4",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c03f00000013",SYMLINK+="asm-400g-0013-data5",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c09e00000014",SYMLINK+="asm-400g-0014-data6",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c0e900000015",SYMLINK+="asm-400g-0015-data7",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de7c12e00000016",SYMLINK+="asm-400g-0016-data8",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de893a0000000da",SYMLINK+="asm-5g-00da-grid1",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de8941e000000db",SYMLINK+="asm-5g-00db-grid2",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de894a4000000dc",SYMLINK+="asm-5g-00dc-grid3",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="sd*",SUBSYSTEM=="block",PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$name",RESULT=="36acb3b510041191b0de80f1f00000039",SYMLINK+="asm-100g-0039-arch1",OWNER="grid",GROUP="asmadmin",MODE="0660"
Here you see the result of /lib/udev/scsi_id -g -u -d /dev/sd* and the Lun WWN found by storage multipath. Except for the result of scsi_id query, there is an extra 3 in the first place, followed by complete the same.
At this time, you can use udevadm to apply rules:
udevadm control --reload
udevadm trigger
Then check the results:
[ root@xxdb01 ~]# ls -l /dev/asm*
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-100g-0039-arch1 -> sdm
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-000f-data1 -> sdb
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0010-data2 -> sdc
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0011-data3 -> sdd
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0012-data4 -> sde
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0013-data5 -> sdf
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0014-data6 -> sdg
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0015-data7 -> sdh
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-400g-0016-data8 -> sdi
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-5g-00da-grid1 -> sdj
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-5g-00db-grid2 -> sdk
lrwxrwxrwx.1 root root 3 Sep 810:19/dev/asm-5g-00dc-grid3 -> sdl
[ root@xxdb01 ~]# ls -l /dev/sd*
brw-rw----.1 root disk 8,0 Sep 810:10/dev/sda
brw-rw----.1 root disk 8,1 Sep 810:10/dev/sda1
brw-rw----.1 root disk 8,2 Sep 810:10/dev/sda2
brw-rw----.1 grid asmadmin 8,16 Sep 816:02/dev/sdb
brw-rw----.1 grid asmadmin 8,32 Sep 810:19/dev/sdc
brw-rw----.1 grid asmadmin 8,48 Sep 810:19/dev/sdd
brw-rw----.1 grid asmadmin 8,64 Sep 810:19/dev/sde
brw-rw----.1 grid asmadmin 8,80 Sep 810:19/dev/sdf
brw-rw----.1 grid asmadmin 8,96 Sep 810:19/dev/sdg
brw-rw----.1 grid asmadmin 8,112 Sep 810:19/dev/sdh
brw-rw----.1 grid asmadmin 8,128 Sep 810:19/dev/sdi
brw-rw----.1 grid asmadmin 8,144 Sep 816:02/dev/sdj
brw-rw----.1 grid asmadmin 8,160 Sep 816:02/dev/sdk
brw-rw----.1 grid asmadmin 8,176 Sep 816:02/dev/sdl
brw-rw----.1 grid asmadmin 8,192 Sep 816:02/dev/sdm
Finally use asmca to create a disk group, the final result is:
[ grid@xxdb01 ~]$ asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 5124096419430410240010227601022760 N ARCH/
MOUNTED EXTERN N 5124096419430432768003276620032766200 N DATA/
MOUNTED NORMAL N 512409641943041536014320512046000 Y GRID/[grid@xxdb01 ~]$
It can be seen that the name of the disk group and the disk alias defined above can be well managed and maintained. I personally think that the customer's specifications are worth learning from.
Recommended Posts