This blogpost is about how to clean up the failed Grid Infrastructure installation,When Grid Infrastructure installation fails, especially while running the root.sh script, Failure maybe due to various reasons. My failure reason was that, I was installing GI and in my VM I had allocated very little memory, due to which the root.sh script hanged, patiently waited for 3 hours, nothing moved, had to abruptly shutdown the VM and go home. C:\>"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" list hdds UUID: 3326c82d-4a75-4f5a-a306-5762ec45db86 Parent UUID: base State: locked write Type: normal (base) Location: C:\RAC11g\ajithn1\ajithn1.vdi Storage format: VDI Capacity: 30720 MBytes Encryption: disabled 1) When checked, it was a half baked situation, In node “ajithn1” ASM instance was up and running and all crs services were up Using username "oracle". Last login: Sun Apr 10 16:57:19 2016 [oracle@ajithn1 ~]$ ps -ef|grep pmon oracle 3697 1 0 08:21 ? 00:00:00 asm_pmon_+ASM1 oracle 4423 4346 0 08:32 pts/1 00:00:00 grep pmon [oracle@ajithn1 ~]$ su - Password: [root@ajithn1 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [root@ajithn1 ~]# 2) In node “ajithn2”, again ASM instance was up but cluster services were not. Using username "oracle". Last login: Sun Apr 10 16:58:58 2016 [oracle@ajithn2 ~]$ ps -ef|grep pmon oracle 3733 1 0 08:32 ? 00:00:00 asm_pmon_+ASM2 oracle 3846 3607 0 08:32 pts/1 00:00:00 grep pmon [oracle@ajithn2 ~]$ crsctl check crs -bash: crsctl: command not found [oracle@ajithn2 ~]$ su - Password: [root@ajithn2 ~]# crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [root@ajithn2 ~]# 3) Now, how to cleanup your GI installation? To do a cleanup execute the below perl command on all the nodes but not the last node (I have only 2 nodes, so second node will have a slightly different command) [oracle@ajithn1 ~]$ perl /u01/grid/oracle/product/11.2.0/grid_1/crs/install/rootcrs.pl -verbose -deconfig -force 2016-04-11 08:35:03: Parsing the host name 2016-04-11 08:35:03: Checking for super user privileges You must be logged in as root to run this script. 2016-04-11 08:35:03: ###### Begin Error Stack Trace ###### 2016-04-11 08:35:03: Package File Line Calling 2016-04-11 08:35:03: --------------- -------------------- ---- ---------- 2016-04-11 08:35:03: 1: crsconfig_lib s_crsconfig_lib.pm 121 crsconfig_lib::error 2016-04-11 08:35:03: 2: crsconfig_lib crsconfig_lib.pm 856 crsconfig_lib::s_check_SuperUser 2016-04-11 08:35:03: 3: main rootcrs.pl 311 crsconfig_lib::check_SuperUser 2016-04-11 08:35:03: ####### End Error Stack Trace ####### Log in as root and rerun this script. 2016-04-11 08:35:03: ###### Begin Error Stack Trace ###### 2016-04-11 08:35:03: Package File Line Calling 2016-04-11 08:35:03: --------------- -------------------- ---- ---------- 2016-04-11 08:35:03: 1: crsconfig_lib s_crsconfig_lib.pm 122 crsconfig_lib::error 2016-04-11 08:35:03: 2: crsconfig_lib crsconfig_lib.pm 856 crsconfig_lib::s_check_SuperUser 2016-04-11 08:35:03: 3: main rootcrs.pl 311 crsconfig_lib::check_SuperUser 2016-04-11 08:35:03: ####### End Error Stack Trace ####### 2016-04-11 08:35:03: Not running as authorized user Insufficient privileges to execute this script 2016-04-11 08:35:03: ###### Begin Error Stack Trace ###### 2016-04-11 08:35:03: Package File Line Calling 2016-04-11 08:35:03: --------------- -------------------- ---- ---------- 2016-04-11 08:35:03: 1: main rootcrs.pl 313 crsconfig_lib 2016-04-11 08:35:03: ####### End Error Stack Trace ####### [oracle@ajithn1 ~]$ su - Password: [root@ajithn1 ~]# perl /u01/grid/oracle/product/11.2.0/grid_1/crs/install/rootcr 2016-04-11 08:35:23: Parsing the host name 2016-04-11 08:35:23: Checking for super user privileges 2016-04-11 08:35:23: User has super user privileges Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/i VIP exists.:ajithn1 VIP exists.: /ajithn1-vip/192.168.78.61/255.255.255.0/eth0 GSD exists. ONS daemon exists. Local port 6100, remote port 6200 eONS daemon exists. Multicast port 17212, multicast IP address 234.96.173.129, l ACFS-9200: Supported CRS-2613: Could not find resource 'ora.registry.acfs'. CRS-4000: Command Stop failed, or completed with errors. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resourc CRS-2673: Attempting to stop 'ora.crsd' on 'ajithn1' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ajit CRS-2673: Attempting to stop 'ora.DATA.dg' on 'ajithn1' CRS-2677: Stop of 'ora.DATA.dg' on 'ajithn1' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'ajithn1' CRS-2677: Stop of 'ora.asm' on 'ajithn1' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ajithn1' has completed CRS-2677: Stop of 'ora.crsd' on 'ajithn1' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'ajithn1' CRS-2673: Attempting to stop 'ora.ctssd' on 'ajithn1' CRS-2673: Attempting to stop 'ora.evmd' on 'ajithn1' CRS-2673: Attempting to stop 'ora.asm' on 'ajithn1' CRS-2673: Attempting to stop 'ora.mdnsd' on 'ajithn1' CRS-2677: Stop of 'ora.cssdmonitor' on 'ajithn1' succeeded CRS-2677: Stop of 'ora.evmd' on 'ajithn1' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'ajithn1' succeeded CRS-2677: Stop of 'ora.asm' on 'ajithn1' succeeded CRS-2677: Stop of 'ora.ctssd' on 'ajithn1' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'ajithn1' CRS-2677: Stop of 'ora.cssd' on 'ajithn1' succeeded CRS-2673: Attempting to stop 'ora.gpnpd' on 'ajithn1' CRS-2673: Attempting to stop 'ora.diskmon' on 'ajithn1' CRS-2677: Stop of 'ora.gpnpd' on 'ajithn1' succeeded CRS-2673: Attempting to stop 'ora.gipcd' on 'ajithn1' CRS-2677: Stop of 'ora.diskmon' on 'ajithn1' succeeded CRS-2677: Stop of 'ora.gipcd' on 'ajithn1' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ajithn1' has completed CRS-4133: Oracle High Availability Services has been stopped. Successfully deconfigured Oracle clusterware stack on this node [root@ajithn1 ~]# 4) Now, run the below perl command in the last node of your cluster, in my case it is node “ajithn2” , I have only 2 nodes. [oracle@ajithn2 ~]$ perl /u01/grid/oracle/product/11.2.0/grid_1/crs/install/rootcrs.pl -verbose -deconfig -force -lastnode 2016-04-11 08:37:29: Parsing the host name 2016-04-11 08:37:29: Checking for super user privileges You must be logged in as root to run this script. 2016-04-11 08:37:29: ###### Begin Error Stack Trace ###### 2016-04-11 08:37:29: Package File Line Calling 2016-04-11 08:37:29: --------------- -------------------- ---- ---------- 2016-04-11 08:37:29: 1: crsconfig_lib s_crsconfig_lib.pm 121 crsconfig_lib::error 2016-04-11 08:37:29: 2: crsconfig_lib crsconfig_lib.pm 856 crsconfig_lib::s_check_SuperUser 2016-04-11 08:37:29: 3: main rootcrs.pl 311 crsconfig_lib::check_SuperUser 2016-04-11 08:37:29: ####### End Error Stack Trace ####### Log in as root and rerun this script. 2016-04-11 08:37:29: ###### Begin Error Stack Trace ###### 2016-04-11 08:37:29: Package File Line Calling 2016-04-11 08:37:29: --------------- -------------------- ---- ---------- 2016-04-11 08:37:29: 1: crsconfig_lib s_crsconfig_lib.pm 122 crsconfig_lib::error 2016-04-11 08:37:29: 2: crsconfig_lib crsconfig_lib.pm 856 crsconfig_lib::s_check_SuperUser 2016-04-11 08:37:29: 3: main rootcrs.pl 311 crsconfig_lib::check_SuperUser 2016-04-11 08:37:29: ####### End Error Stack Trace ####### 2016-04-11 08:37:29: Not running as authorized user Insufficient privileges to execute this script 2016-04-11 08:37:29: ###### Begin Error Stack Trace ###### 2016-04-11 08:37:29: Package File Line Calling 2016-04-11 08:37:29: --------------- -------------------- ---- ---------- 2016-04-11 08:37:29: 1: main rootcrs.pl 313 crsconfig_lib::error 2016-04-11 08:37:29: ####### End Error Stack Trace ####### [oracle@ajithn2 ~]$ su - Password: [root@ajithn2 ~]# perl /u01/grid/oracle/product/11.2.0/grid_1/crs/install/rootcrs.pl -verbose -deconfig -force -lastnode 2016-04-11 08:37:44: Parsing the host name 2016-04-11 08:37:44: Checking for super user privileges 2016-04-11 08:37:44: User has super user privileges Using configuration parameter file: /u01/grid/oracle/product/11.2.0/grid_1/crs/install/crsconfig_params GSD exists. ONS daemon exists. Local port 6100, remote port 6200 eONS daemon exists. Multicast port 17212, multicast IP address 234.96.173.129, listening port 2016 PRKO-2439 : VIP does not exist. ACFS-9200: Supported CRS-2613: Could not find resource 'ora.registry.acfs'. CRS-4000: Command Stop failed, or completed with errors. CRS-2613: Could not find resource 'ora.registry.acfs'. CRS-4000: Command Delete failed, or completed with errors. CRS-2673: Attempting to stop 'ora.crsd' on 'ajithn2' CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'ajithn2' CRS-2673: Attempting to stop 'ora.DATA.dg' on 'ajithn2' CRS-2677: Stop of 'ora.DATA.dg' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.asm' on 'ajithn2' CRS-2677: Stop of 'ora.asm' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.eons' on 'ajithn2' CRS-2673: Attempting to stop 'ora.ons' on 'ajithn2' CRS-2677: Stop of 'ora.ons' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.eons' on 'ajithn2' succeeded CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'ajithn2' has completed CRS-2677: Stop of 'ora.crsd' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'ajithn2' CRS-2673: Attempting to stop 'ora.ctssd' on 'ajithn2' CRS-2673: Attempting to stop 'ora.evmd' on 'ajithn2' CRS-2673: Attempting to stop 'ora.asm' on 'ajithn2' CRS-2677: Stop of 'ora.cssdmonitor' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.evmd' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.ctssd' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.asm' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'ajithn2' CRS-2677: Stop of 'ora.cssd' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.diskmon' on 'ajithn2' CRS-2677: Stop of 'ora.diskmon' on 'ajithn2' succeeded CRS-2613: Could not find resource 'ora.drivers.acfs'. CRS-4000: Command Modify failed, or completed with errors. CRS-2672: Attempting to start 'ora.cssdmonitor' on 'ajithn2' CRS-2676: Start of 'ora.cssdmonitor' on 'ajithn2' succeeded CRS-2672: Attempting to start 'ora.cssd' on 'ajithn2' CRS-2672: Attempting to start 'ora.diskmon' on 'ajithn2' CRS-2676: Start of 'ora.diskmon' on 'ajithn2' succeeded CRS-2676: Start of 'ora.cssd' on 'ajithn2' succeeded CRS-4611: Successful deletion of voting disk +DATA. CRS-2672: Attempting to start 'ora.ctssd' on 'ajithn2' CRS-2676: Start of 'ora.ctssd' on 'ajithn2' succeeded CRS-2672: Attempting to start 'ora.asm' on 'ajithn2' CRS-2676: Start of 'ora.asm' on 'ajithn2' succeeded ASM de-configuration trace file location: /u01/app/oracle/cfgtoollogs/asmca/asmcadc_clean4904428430281726239.log ASM Clean Configuration START ASM Clean Configuration END ASM with SID +ASM1 deleted successfully. Check /u01/app/oracle/cfgtoollogs/asmca/asmcadc_clean4904428430281726239.log for details. CRS-2613: Could not find resource 'ora.drivers.acfs'. CRS-4000: Command Delete failed, or completed with errors. CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'ajithn2' CRS-2673: Attempting to stop 'ora.mdnsd' on 'ajithn2' CRS-2673: Attempting to stop 'ora.gpnpd' on 'ajithn2' CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'ajithn2' CRS-2673: Attempting to stop 'ora.ctssd' on 'ajithn2' CRS-2673: Attempting to stop 'ora.asm' on 'ajithn2' CRS-2677: Stop of 'ora.cssdmonitor' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.mdnsd' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.gpnpd' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.asm' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.ctssd' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.cssd' on 'ajithn2' CRS-2677: Stop of 'ora.cssd' on 'ajithn2' succeeded CRS-2673: Attempting to stop 'ora.diskmon' on 'ajithn2' CRS-2673: Attempting to stop 'ora.gipcd' on 'ajithn2' CRS-2677: Stop of 'ora.gipcd' on 'ajithn2' succeeded CRS-2677: Stop of 'ora.diskmon' on 'ajithn2' succeeded CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'ajithn2' has completed CRS-4133: Oracle High Availability Services has been stopped. error: package cvuqdisk is not installed Successfully deconfigured Oracle clusterware stack on this node [root@ajithn2 ~]# 5) My ASM instance was also up, But when I try to reinstall the GI, I will not be able to use the already used ASM devices (sdb1, sdc1), So, let’s scribble something on the headers of the devices to erase the details of ASM DG and make it reusable. (Using dd command to erase the ASM diskgroup details from device headers) [root@ajithn1 ~]# /etc/init.d/oracleasm listdisks BACKUP DATA [root@ajithn1 ~]# dd if=/dev/zero of=/dev/sdb1 bs=1024 count=100 100+0 records in 100+0 records out 102400 bytes (102 kB) copied, 0.267837 seconds, 382 kB/s [root@ajithn1 ~]# dd if=/dev/zero of=/dev/sdc1 bs=1024 count=100 100+0 records in 100+0 records out 102400 bytes (102 kB) copied, 0.357379 seconds, 287 kB/s [root@ajithn1 ~]# 6) Removing the ASM Diskgroups [root@ajithn1 ~]# /etc/init.d/oracleasm deletedisk DATA /dev/sdb1 Removing ASM disk "DATA": [ OK ] [root@ajithn1 ~]# /etc/init.d/oracleasm deletedisk BACKUP /dev/sdc1 Removing ASM disk "BACKUP": [ OK ] [root@ajithn1 ~]# /etc/init.d/oracleasm listdisks [root@ajithn1 ~]# 7) Recreating the ASM diskgroups. [root@ajithn1 ~]# /etc/init.d/oracleasm createdisk data /dev/sdb1 Marking disk "data" as an ASM disk: [ OK ] [root@ajithn1 ~]# /etc/init.d/oracleasm createdisk backup /dev/sdc1 Marking disk "backup" as an ASM disk: [ OK ] [root@ajithn1 ~]# /etc/init.d/oracleasm listdisks BACKUP DATA [root@ajithn1 ~]# Once the ASM Diskgroups are recreated, we can restart the GI installation and it should be successful, and I had a successful installation after I increased my VM memory. HAPPY LEARNING!
↧