Written By Nassyam Basha Introduction This article explains you how to upgrade the Data Guard environment from 12.1.0.1 to 12.1.0.2 when the Far Sync is part of the configuration. The database have container database with pluggable database and the protection mode is maximum performance mode. Data Guard Environment Fully configured configuration of Primary Database, Physical standby and Far Sync instance on 12.1.0.1 RDBMS, To upgrade the Data Guard environment of course we have the DBMS_ROLLING procedure which was introduced from 12.1.0.1 with simple steps and less risk but I have already explained how to deal with DBMS_ROLLING in URL https://www.toadworld.com/platforms/oracle/w/wiki/10994.simple-database-upgrades-with-12c-dbms-rolling , but it falls under Oracle Active Data Guard license and nobody expects that everyone have the license of ADG. Our goal is to ensure the Data Guard environment is available and well functioning after the upgrade from 12.1.0.1 to 12.1.0.2, In order to achieve this being DBA we have to take smart steps. I mean there are many steps involved to move the instance configuration files, network files to the new Oracle Home of all the three sites and take care of listener changes. Where to start and how to do? That is what going to cover in this article. As we know the basic rule first we have to complete the work on standby databases and there is no matter of availability of the standby databases all the time, because the more importance is to reduce downtime of production database and upgrade the environment to finish in smooth manner. Protection Mode: Maximum Performance Primary database: CANADA - ORA-C1 Standby Database: INDIA - ORA-C3 Far SYNC: CANFAR - ORA-C2 Version: 12.1.0.1 Upgrade Method: Manual Please note that this article does not show about the prerequisites prior to upgrade and which patchset you have to download and so on. Our goal is to how to upgrade Data Guard environment with Far Sync. Software and Installation Method Prior to upgrade the database, it is mandatory to install the 12.1.0.2 new home in all the locations because it is not going to harm the current configuration, the installation of 12.1.0.2 can be done either using the silent method or the OUI with the available options. In this article the installation of 12.1.0.2 using the runInstaller and the brief steps of GUI can find below. The software will can be downloaded from OTN. Configure Security Updates Installation options - install database software only Grid Installation Options - Single instance database installation Product languages, Database edition, Installation location OS Groups Fulfill prerequisite checks Click for Install Run post script Review of Database and Pre-Upgrade script Before performing the upgrade of the database, Oracle have preupgrade utility which will give information on the existing problems which needs to be fixed prior to the upgrade, if it is resolvable by Oracle then the fixup scripts will be available in the location mentioned in the catcon script. SQL> alter pluggable database all open; Pluggable database altered. SQL> show pdbs CON_ID CON_NAME OPEN MODE RESTRICTED ---------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 SAPDB READ WRITE NO SQL> [oracle@ORA-C1 admin]$ $ORACLE_HOME/perl/bin/perl catcon.pl -d /u01/app/oracle/product/12.1.0/dbhome_1/rdbms/admin -l /home/oracle/nassyam -b DG_FS preupgrd.sql [oracle@ORA-C1 admin]$ [oracle@ORA-C1 nassyam]$ pwd /home/oracle/nassyam [oracle@ORA-C1 nassyam]$ ls -ltr total 20 -rw-r--r--. 1 oracle oinstall 4822 Dec 29 02:11 DG_FS0.log -rw-r--r--. 1 oracle oinstall 2657 Dec 29 02:11 DG_FS1.log -rw-r--r--. 1 oracle oinstall 498 Dec 29 02:11 DG_FS2.log -rw-r--r--. 1 oracle oinstall 498 Dec 29 02:11 DG_FS3.log [oracle@ORA-C1 nassyam]$ Fixup Scripts [oracle@ORA-C1 cfgtoollogs]$ cd /u01/app/oracle/cfgtoollogs/CANADA/preupgrade/ [oracle@ORA-C1 preupgrade]$ ls -ltr total 16 -rw-r--r--. 1 oracle oinstall 4012 Dec 29 02:11 preupgrade_fixups.sql -rw-r--r--. 1 oracle oinstall 1969 Dec 29 02:11 postupgrade_fixups.sql -rw-r--r--. 1 oracle oinstall 5819 Dec 29 02:11 preupgrade.log [oracle@ORA-C1 preupgrade]$ Current configuration Prior to Upgrade DGMGRL> show configuration Configuration - ckpt12c Protection Mode: MaxPerformance Databases: canada - Primary database canfar - Far Sync india - Physical standby database Fast-Start Failover: DISABLED Configuration Status: SUCCESS DGMGRL> show database india Database - india Role: PHYSICAL STANDBY Intended State: APPLY-ON Transport Lag: 0 seconds (computed 1 second ago) Apply Lag: 0 seconds (computed 1 second ago) Apply Rate: 97.00 KByte/s Real Time Query: ON Instance(s): india Database Status: SUCCESS DGMGRL> Container Database – PreUpgrade Script The preupgrade script we have to run both at the container level and also the pluggable database as well. As mentioned earlier this can be fixed using scripts or manually we may have to fix depending on the existing issues. SQL> @preupgrd.sql Loading Pre-Upgrade Package... *************************************************************************** Executing Pre-Upgrade Checks in CDB$ROOT... *************************************************************************** ************************************************************ ====>> ERRORS FOUND for CDB$ROOT > PRE-UPGRADE RESULTS for CDB$ROOT Pluggable Database – Preupgrade Script SQL> show pdbs 2 PDB$SEED READ ONLY NO 3 SAPDB READ WRITE NO SQL> alter session set container=sapdb 2 ; SQL> @preupgrd.sql Loading Pre-Upgrade Package... *************************************************************************** Executing Pre-Upgrade Checks in SAPDB... *************************************************************************** ************************************************************ ====>> ERRORS FOUND for SAPDB > PRE-UPGRADE RESULTS for SAPDB Running fixup scripts From the preupgrade utility fixup scripts, run from the root level. SQL> conn / as sysdba Connected. SQL> @ /u01/app/oracle/cfgtoollogs/CANADA/preupgrade/preupgrade_fixups.sql Pre-Upgrade Fixup Script Generated on 2015-12-29 05:32:48 Version: 12.1.0.2 Build: 006 Beginning Pre-Upgrade Fixups... Executing in container CDB$ROOT ********************************************************************** Check Tag: SYNC_STANDBY_DB Check Summary: Check for unsynced database Fix Summary: Standby databases should be synced prior to upgrade. ********************************************************************** Fixup Returned Information: INFORMATION: --> Standby database not synced Sync standby database prior to upgrade. Your standby databases should be synched prior to upgrading. ********************************************************************** ********************************************************************** Check Tag: APEX_UPGRADE_MSG Check Summary: Check that APEX will need to be upgraded. Fix Summary: Oracle Application Express can be manually upgraded prior to database upgrade. ********************************************************************** Fixup Returned Information: INFORMATION: --> Oracle Application Express (APEX) can be manually upgraded prior to database upgrade APEX is currently at version 4.2.0.00.27 and will need to be upgraded to APEX version 4.2.5 in the new release. Note 1: To reduce database upgrade time, APEX can be manually upgraded outside of and prior to database upgrade. Note 2: See MOS Note 1088970.1 for information on APEX installation upgrades. ********************************************************************** ********************************************************************** [Pre-Upgrade Recommendations] ********************************************************************** ***************************************** ********* Dictionary Statistics ********* ***************************************** Please gather dictionary statistics 24 hours prior to upgrading the database. To gather dictionary statistics execute the following command while connected as SYSDBA: EXECUTE dbms_stats.gather_dictionary_stats; ^^^ MANUAL ACTION SUGGESTED ^^^ ************************************************** ************* Fixup Summary ************ 2 fixup routines generated INFORMATIONAL messages that should be reviewed. **************** Pre-Upgrade Fixup Script Complete ********************* SQL> EXECUTE dbms_stats.gather_dictionary_stats; SQL> Fixup Script from Pluggable Database. SQL> alter session set container=sapdb 2 ; SQL> @/u01/app/oracle/cfgtoollogs/CANADA/preupgrade/preupgrade_fixups.sql Pre-Upgrade Fixup Script Generated on 2015-12-29 05:34:27 Version: 12.1.0.2 Build: 006 Beginning Pre-Upgrade Fixups... Executing in container SAPDB ********************************************************************** Check Tag: SYNC_STANDBY_DB Check Summary: Check for unsynced database Fix Summary: Standby databases should be synced prior to upgrade. ********************************************************************** Fixup Returned Information: INFORMATION: --> Standby database not synced Sync standby database prior to upgrade. Your standby databases should be synched prior to upgrading. ********************************************************************** ********************************************************************** Check Tag: APEX_UPGRADE_MSG Check Summary: Check that APEX will need to be upgraded. Fix Summary: Oracle Application Express can be manually upgraded prior to database upgrade. ********************************************************************** Fixup Returned Information: INFORMATION: --> Oracle Application Express (APEX) can be manually upgraded prior to database upgrade APEX is currently at version 4.2.0.00.27 and will need to be upgraded to APEX version 4.2.5 in the new release. Note 1: To reduce database upgrade time, APEX can be manually upgraded outside of and prior to database upgrade. Note 2: See MOS Note 1088970.1 for information on APEX installation upgrades. ********************************************************************** ********************************************************************** [Pre-Upgrade Recommendations] ********************************************************************** ***************************************** ********* Dictionary Statistics ********* ***************************************** Please gather dictionary statistics 24 hours prior to upgrading the database. To gather dictionary statistics execute the following command while connected as SYSDBA: EXECUTE dbms_stats.gather_dictionary_stats; ^^^ MANUAL ACTION SUGGESTED ^^^ ************************************************** ************* Fixup Summary ************ 2 fixup routines generated INFORMATIONAL messages that should be reviewed. **************** Pre-Upgrade Fixup Script Complete ********************* SQL> EXECUTE dbms_stats.gather_dictionary_stats; SQL> Check Standby and FarSync Synchronization with Primary Prior to upgrade it is necessary to have all the data was received on farsync and also applied on standby database. Primary SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- CANADA PRIMARY ID STATUS DB_MODE TYPE RECOVERY_MODE PROTECTION_MODE SRLs ACTIVE ARCHIVED_SEQ# ---------- --------- --------------- ---------- ----------------------- -------------------- ---------- ---------- ------------- 1 VALID OPEN ARCH IDLE MAXIMUM PERFORMANCE 0 0 56 2 VALID MOUNTED-STANDBY LGWR IDLE MAXIMUM PERFORMANCE 7 1 56 SQL> select thread#,max(sequence#) from v$archived_log group by thread#; THREAD# MAX(SEQUENCE#) ---------- -------------- 56 Far Sync SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- CANFAR FAR SYNC ID STATUS DB_MODE TYPE RECOVERY_MODE PROTECTION_MODE SRLs ACTIVE ARCHIVED_SEQ# ---------- --------- --------------- ---------- ----------------------- -------------------- ---------- ---------- ------------- 1 VALID MOUNTED ARCH IDLE MAXIMUM PERFORMANCE 0 0 56 2 VALID OPEN_READ-ONLY LGWR MANAGED REAL TIME APPLY MAXIMUM PERFORMANCE 4 1 56 Standby SQL> select thread#,max(sequence#) from v$archived_log where applied='YES' group by thread#; THREAD# MAX(SEQUENCE#) ---------- -------------- 1 56 Destination Settings: SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- CANADA PRIMARY SQL> select name,value from v$parameter where name='log_archive_dest_2'; NAME VALUE -------------------- -------------------------------------------------------------------------------- log_archive_dest_2 service="canfar", ASYNC NOAFFIRM delay=0 optional compression=disable max_failur e=0 max_connections=1 reopen=300 db_unique_name="canfar" net_timeout=30, valid_f or=(online_logfile,all_roles) SQL> connect sys@canfar as sysdba Enter password: Connected. SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- CANFAR FAR SYNC SQL> select name,value from v$parameter where name='log_archive_dest_2'; NAME VALUE -------------------- -------------------------------------------------------------------------------- log_archive_dest_2 service="india", ASYNC NOAFFIRM delay=0 optional compression=disable max_failure =0 max_connections=1 reopen=300 db_unique_name="india" net_timeout=30, valid_for =(standby_logfile,all_roles) Copy Configuration files and take care of network file s Disable the broker from all the sites using below commands, so that broker will not attempt to enable the redo transport. SQL> alter system set dg_broker_start=false; System altered. SQL> Defer the remote destinations to send the redo from primary to FarSync, so that no redo will send until the upgrade process is completed. SQL> alter system set log_archive_dest_state_2='defer'; System altered. SQL> [oracle@ORA-C1 dbs]$ cp * /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/ [oracle@ORA-C1 dbs]$ cd /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/ [oracle@ORA-C1 admin]$ ls listener.ora samples shrept.lst sqlnet.ora tnsnames.ora [oracle@ORA-C1 admin]$ cp -r * /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/ [oracle@ORA-C1 admin]$ [oracle@ORA-C1 admin]$ lsnrctl stop listener LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 28-DEC-2015 18:18:09 Copyright (c) 1991, 2013, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) The command completed successfully [oracle@ORA-C1 admin]$ Primary: Update the new ORACLE_HOME location in listener.ora [oracle@ORA-C1 admin]$ cd /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/ [oracle@ORA-C1 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) [oracle@ORA-C1 admin]$ pwd /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin [oracle@ORA-C1 admin]$ vi listener.ora [oracle@ORA-C1 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) [oracle@ORA-C1 admin]$ Standby: Cancel the recovery on Physical Standby/terminal database. SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- INDIA PHYSICAL STANDBY SQL> alter database recover managed standby database cancel; Database altered. SQL> Move FarSync to New Home There are no upgrade special steps involved in FarSync or Standby database, we have to stop all the services from old home and then start from New home as below. SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- CANFAR FAR SYNC SQL> shut immediate ORA-01109: database not open Database dismounted. ORACLE instance shut down. SQL> [oracle@ORA-C3 ~]$ ps -ef|grep tns root 15 2 0 02:46 ? 00:00:00 [netns] oracle 14230 1 2 18:18 ? 00:00:00 /u01/app/oracle/product/12.1.0/dbhome_1/bin/tnslsnr LISTENER -inherit oracle 14233 14190 0 18:18 pts/2 00:00:00 grep tns [oracle@ORA-C3 ~]$ lsnrctl stop listener LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 28-DEC-2015 18:18:09 Copyright (c) 1991, 2013, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) The command completed successfully [oracle@ORA-C3 ~]$ [oracle@ORA-C3 dbs]$ cp * /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/ [oracle@ORA-C3 dbs]$ cd /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/ [oracle@ORA-C3 admin]$ ls listener.ora samples shrept.lst sqlnet.ora tnsnames.ora [oracle@ORA-C3 admin]$ cp -r * /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/ [oracle@ORA-C3 admin]$ Update the new ORACLE_HOME location in listener.ora [oracle@ORA-C3 admin]$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 [oracle@ORA-C3 admin]$ export PATH=$ORACLE_HOME/bin:$PATH [oracle@ORA-C3 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) [oracle@ORA-C3 admin]$ vi listener.ora [oracle@ORA-C3 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) [oracle@ORA-C3 admin]$ Start the FarSync instance after above changes [oracle@ORA-C3 admin]$ vi tnsnames.ora [oracle@ORA-C3 admin]$ export ORACLE_SID=canfar [oracle@ORA-C3 admin]$ sqlplus / as sysdba SQL*Plus: Release 12.1.0.2.0 Production on Mon Dec 28 18:14:36 2015 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount ORACLE instance started. Total System Global Area 838860800 bytes Fixed Size 2929936 bytes Variable Size 608176880 bytes Database Buffers 222298112 bytes Redo Buffers 5455872 bytes Database mounted. SQL> [oracle@ORA-C3 ~]$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 [oracle@ORA-C3 ~]$ export PATH=$ORACLE_HOME/bin:$PATH [oracle@ORA-C3 ~]$ lsnrctl start listener LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 28-DEC-2015 18:19:05 Copyright (c) 1991, 2014, Oracle. All rights reserved. Starting /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 12.1.0.2.0 - Production System parameter file is /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Log messages written to /u01/app/oracle/diag/tnslsnr/ORA-C3/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.85)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) STATUS of the LISTENER ------------------------ Alias listener Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production Start Date 28-DEC-2015 18:19:05 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Listener Log File /u01/app/oracle/diag/tnslsnr/ORA-C3/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.85)(PORT=1521))) Services Summary... Service "PLSExtProc" has 1 instance(s). Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service... Service "canfar_DGMGRL" has 1 instance(s). Instance "canfar", status UNKNOWN, has 1 handler(s) for this service... The command completed successfully [oracle@ORA-C3 ~]$ [oracle@ORA-C3 ~]$ ps -ef|grep tns root 15 2 0 02:46 ? 00:00:00 [netns] oracle 14240 1 0 18:19 ? 00:00:00 /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr listener -inherit oracle 14246 14190 0 18:19 pts/2 00:00:00 grep tns [oracle@ORA-C3 ~]$ ps -ef|grep pmon oracle 14091 1 0 18:14 ? 00:00:00 ora_pmon_canfar oracle 14248 14190 0 18:19 pts/2 00:00:00 grep pmon [oracle@ORA-C3 ~]$ pwdx 14091 14091: /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs [oracle@ORA-C3 ~]$ Move the Physical Standby to new Home The steps are same like FarSync, because we are not going to run any upgrade scripts on physical standby. Whenever the upgrade performed on primary database the changes will be applied on standby database whenever the redo is received. Hence the data dictionary will be updated. We will see how that works at the end of the article. SQL> select db_unique_name,database_role from v$database; DB_UNIQUE_NAME DATABASE_ROLE ------------------------------ ---------------- INDIA PHYSICAL STANDBY SQL> shut immediate; Database closed. Database dismounted. ORACLE instance shut down. SQL> [oracle@ORA-C2 trace]$ lsnrctl stop listener LSNRCTL for Linux: Version 12.1.0.1.0 - Production on 29-DEC-2015 03:27:22 Copyright (c) 1991, 2013, Oracle. All rights reserved. Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) The command completed successfully [oracle@ORA-C2 trace]$ [oracle@ORA-C2 dbs]$ cp * /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs/ [oracle@ORA-C2 dbs]$ cd ../network/admin/ [oracle@ORA-C2 admin]$ ls listener.ora samples shrept.lst sqlnet.ora tnsnames.ora [oracle@ORA-C2 admin]$ cp -r * /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/ [oracle@ORA-C2 admin]$ cd /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/ [oracle@ORA-C2 admin]$ ls listener.ora samples shrept.lst sqlnet.ora tnsnames.ora [oracle@ORA-C2 admin]$ Perform changes in listener.ora file [oracle@ORA-C2 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0/dbhome_1) [oracle@ORA-C2 admin]$ vi listener.ora [oracle@ORA-C2 admin]$ cat listener.ora |grep 12.1 # listener.ora Network Configuration File: /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1) [oracle@ORA-C2 admin]$ [oracle@ORA-C2 admin]$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/dbhome_1 [oracle@ORA-C2 admin]$ export PATH=$ORACLE_HOME/bin:$PATH [oracle@ORA-C2 admin]$ which sqlplus /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/sqlplus [oracle@ORA-C2 admin]$ sqlplus / as sysdba SQL*Plus: Release 12.1.0.2.0 Production on Tue Dec 29 03:30:29 2015 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to an idle instance. SQL> startup mount ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance ORACLE instance started. Total System Global Area 1409286144 bytes Fixed Size 2924352 bytes Variable Size 922747072 bytes Database Buffers 469762048 bytes Redo Buffers 13852672 bytes Database mounted. SQL> select db_unique_name,database_role,version from v$database, v$instance; DB_UNIQUE_NAME DATABASE_ROLE VERSION ------------------------------ ---------------- ----------------- INDIA PHYSICAL STANDBY 12.1.0.2.0 SQL> alter database recover managed standby database disconnect from session System altered. SQL> [oracle@ORA-C2 admin]$ lsnrctl start listener LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 29-DEC-2015 03:32:13 Copyright (c) 1991, 2014, Oracle. All rights reserved. Starting /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 12.1.0.2.0 - Production System parameter file is /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Log messages written to /u01/app/oracle/diag/tnslsnr/ORA-C2/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.90)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) STATUS of the LISTENER ------------------------ Alias listener Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production Start Date 29-DEC-2015 03:32:13 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Listener Log File /u01/app/oracle/diag/tnslsnr/ORA-C2/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.90)(PORT=1521))) Services Summary... Service "PLSExtProc" has 1 instance(s). Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service... Service "canada_DGMGRL" has 1 instance(s). Instance "india", status UNKNOWN, has 1 handler(s) for this service... Service "india_DGMGRL" has 1 instance(s). Instance "india", status UNKNOWN, has 1 handler(s) for this service... The command completed successfully [oracle@ORA-C2 admin]$ [oracle@ORA-C2 admin]$ ps -ef|grep tns root 15 2 0 Dec27 ? 00:00:00 [netns] psoracle 18327 1 0 03:32 ? 00:00:00 /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr listener -inherit oracle 18330 16898 0 03:32 pts/5 00:00:00 grep tns [oracle@ORA-C2 admin]$ ps -ef|grep pmon oracle 18016 1 0 03:30 ? 00:00:00 ora_pmon_india oracle 18332 16898 0 03:32 pts/5 00:00:00 grep pmon [oracle@ORA-C2 admin]$ pwdx 18016 18016: /u01/app/oracle/product/12.1.0.2/dbhome_1/dbs [oracle@ORA-C2 admin]$ Manual upgrade of Primary Database If we are performing the upgrade using DBUA then oracle will take care everything to stop and start the database to new home so on, this method is purely on manual upgrade and now we will see how to upgrade the environment. SQL> shut immediate; Pluggable Database closed. SQL> conn / as sysdba Connected. SQL> shut immediate; Database closed. Database dismounted. ORACLE instance shut down. SQL> [oracle@ORA-C1 admin]$ sqlplus / as sysdba SQL*Plus: Release 12.1.0.2.0 Production on Tue Dec 29 05:43:22 2015 Copyright (c) 1982, 2014, Oracle. All rights reserved. Connected to an idle instance. SQL> startup upgrade ORACLE instance started. Total System Global Area 1409286144 bytes Fixed Size 2924352 bytes Variable Size 905969856 bytes Database Buffers 486539264 bytes Redo Buffers 13852672 bytes Database mounted. Database opened. SQL> alter pluggable database all open upgrade; Pluggable database altered. SQL> As we know the upgrade setup is entirely changed from 12c and we will see the scripts to upgrade the database. [oracle@ORA-C1 admin]$ /u01/app/oracle/product/12.1.0.2/dbhome_1/perl/bin/perl /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin/catctl.pl -d $ORACLE_HOME/rdbms/admin -l /home/oracle/nassyam -n 2 catupgrd.sql Argument list for [/u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin/catctl.pl] SQL Process Count n = 2 SQL PDB Process Count N = 0 Input Directory d = /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin Phase Logging Table t = 0 Log Dir l = /home/oracle/nassyam Script s = 0 Serial Run S = 0 Upgrade Mode active M = 0 Start Phase p = 0 End Phase P = 0 Log Id i = 0 Run in c = 0 Do not run in C = 0 Echo OFF e = 1 No Post Upgrade x = 0 Reverse Order r = 0 Open Mode Normal o = 0 Debug catcon.pm z = 0 Debug catctl.pl Z = 0 Display Phases y = 0 Child Process I = 0 catctl.pl version: 12.1.0.2.0 Oracle Base = /u01/app/oracle Analyzing file /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin/catupgrd.sql Log files in /home/oracle/nassyam catcon: ALL catcon-related output will be written to /home/oracle/nassyam/catupgrd_catcon_5816.lst catcon: See /home/oracle/nassyam/catupgrd*.log files for output generated by scripts catcon: See /home/oracle/nassyam/catupgrd_*.lst files for spool files, if any Number of Cpus = 2 Parallel PDB Upgrades = 2 SQL PDB Process Count = 2 SQL Process Count = 2 [CONTAINER NAMES] CDB$ROOT PDB$SEED SAPDB PDB Inclusion:[PDB$SEED SAPDB] Exclusion:[] ------------------------------------------------------ Phases [0-73] Container Lists Inclusion:[CDB$ROOT] Exclusion:[] Serial Phase #: 0 Files: 1 Serial Phase #: 1 Files: 3 Time: 85s CDB$ROOT Restart Phase #: 2 Files: 1 Time: 0s CDB$ROOT Parallel Phase #: 3 Files: 18 Time: 6s CDB$ROOT Restart Phase #: 4 Files: 1 Time: 1s CDB$ROOT Serial Phase #: 5 Files: 5 Time: 39s CDB$ROOT Serial Phase #: 6 Files: 1 Time: 19s CDB$ROOT …………….. LOG FILES: (catupgrd*.log) Upgrade Summary Report Located in: /u01/app/oracle/product/12.1.0.2/dbhome_1/cfgtoollogs/CANADA/upgrade/upg_summary.log Grand Total Upgrade Time: [0d:0h:41m:20s] [oracle@ORA-C1 trace]$ After the completion of the upgrade, you can refer to the logs available under the directory which we have mentioned in the catctl.pl script. Total Upgrade Time: 00:40:44 [CDB$ROOT] PL/SQL procedure successfully completed. Elapsed: 00:00:00.22 Upgrade Times Sorted In Descending Order Total Upgrade Time: 00:47:05 [CDB$ROOT] Total Upgrade Time: 00:40:44 [CDB$ROOT] Grand Total Upgrade Time: [0d:0h:41m:20s] -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15349.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15351.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15350.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15352.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15354.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15353.done -rw-r--r--. 1 oracle oinstall 42 Dec 29 09:24 catupgrd_catcon_15355.done -rw-r--r--. 1 oracle oinstall 390 Dec 29 09:24 catupgrd_datapatch_upgrade.log -rw-r--r--. 1 oracle oinstall 294 Dec 29 09:24 catupgrd_datapatch_upgrade.err -rw-r--r--. 1 oracle oinstall 18602376 Dec 29 09:26 catupgrd0.log -rw-r--r--. 1 oracle oinstall 512 Dec 29 09:26 catupgrd_catcon_12519.lst -rw-r--r--. 1 oracle oinstall 2553907 Dec 29 09:26 catupgrd7.log -rw-r--r--. 1 oracle oinstall 2970208 Dec 29 09:26 catupgrd2.log -rw-r--r--. 1 oracle oinstall 4386125 Dec 29 09:26 catupgrd3.log -rw-r--r--. 1 oracle oinstall 3720956 Dec 29 09:26 catupgrd6.log -rw-r--r--. 1 oracle oinstall 2904249 Dec 29 09:26 catupgrd1.log -rw-r--r--. 1 oracle oinstall 2333390 Dec 29 09:26 catupgrd5.log -rw-r--r--. 1 oracle oinstall 2082325 Dec 29 09:26 catupgrd4.log [oracle@ORA-C1 nassyam]$ pwd /home/oracle/nassyam [oracle@ORA-C1 nassyam]$ If the upgrade process returned errors or the catuppst script haven’t run during the upgrade process, then the script can be execute in manual method like below. [oracle@ORA-C1 admin]$ /u01/app/oracle/product/12.1.0.2/dbhome_1/perl/bin/perl catcon.pl -n 1 -e -b catuppst -d /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin catuppst.sql catcon: ALL catcon-related output will be written to catuppst_catcon_20583.lst catcon: See catuppst*.log files for output generated by scripts catcon: See catuppst_*.lst files for spool files, if any catcon.pl: completed successfully [oracle@ORA-C1 admin]$ After the upgrade of database, utlrp.sql script can run on database to validate the objects. This script can run multiple times if required to do. [oracle@ORA-C1 admin]$ /u01/app/oracle/product/12.1.0.2/dbhome_1/perl/bin/perl catcon.pl -n 1 -e -b utlrp -d /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin utlrp.sql catcon: ALL catcon-related output will be written to utlrp_catcon_20649.lst catcon: See utlrp*.log files for output generated by scripts catcon: See utlrp_*.lst files for spool files, if any catcon.pl: completed successfully [oracle@ORA-C1 admin]$ To ensure the upgrade is successful or not, it can be determined using the script utlu121s.sql available under the $ORACLE_HOME/rdbms/admin. After reviewing the logs we can take the required actions to fix the issues. [oracle@ORA-C1 admin]$ /u01/app/oracle/product/12.1.0.2/dbhome_1/perl/bin/perl catcon.pl -n 1 -e -b utlu121s -d /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin utlu121s.sql catcon: ALL catcon-related output will be written to utlu121s_catcon_20769.lst catcon: See utlu121s*.log files for output generated by scripts catcon: See utlu121s_*.lst files for spool files, if any catcon.pl: completed successfully [oracle@ORA-C1 admin]$ [oracle@ORA-C1 admin]$ ls -ltr utlu*lst -rw-r--r--. 1 oracle oinstall 348 Dec 29 17:28 utlu121s_catcon_20769.lst [oracle@ORA-C1 admin]$ ls -ltr utlu*log -rw-r--r--. 1 oracle oinstall 32615 Dec 29 17:28 utlu121s0.log [oracle@ORA-C1 admin]$ pwd /u01/app/oracle/product/12.1.0.2/dbhome_1/rdbms/admin [oracle@ORA-C1 admin]$ After performing complete upgrade now we can start the database in Read Write mode SQL> shut immediate Database closed. Database dismounted. ORACLE instance shut down. SQL> startup ORACLE instance started. Total System Global Area 1409286144 bytes Fixed Size 2924352 bytes Variable Size 905969856 bytes Database Buffers 486539264 bytes Redo Buffers 13852672 bytes Database mounted. Database opened. SQL> alter pluggable database all open; Pluggable database altered. SQL> Start the Listener and ensure it is running from new home. [oracle@ORA-C1 admin]$ lsnrctl start LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 29-DEC-2015 20:02:20 Copyright (c) 1991, 2014, Oracle. All rights reserved. Starting /u01/app/oracle/product/12.1.0.2/dbhome_1/bin/tnslsnr: please wait... TNSLSNR for Linux: Version 12.1.0.2.0 - Production System parameter file is /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Log messages written to /u01/app/oracle/diag/tnslsnr/ORA-C1/listener/alert/log.xml Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.80)(PORT=1521))) Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) STATUS of the LISTENER ------------------------ Alias LISTENER Version TNSLSNR for Linux: Version 12.1.0.2.0 - Production Start Date 29-DEC-2015 20:02:21 Uptime 0 days 0 hr. 0 min. 0 sec Trace Level off Security ON: Local OS Authentication SNMP OFF Listener Parameter File /u01/app/oracle/product/12.1.0.2/dbhome_1/network/admin/listener.ora Listener Log File /u01/app/oracle/diag/tnslsnr/ORA-C1/listener/alert/log.xml Listening Endpoints Summary... (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521))) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.0.80)(PORT=1521))) Services Summary... Service "PLSExtProc" has 1 instance(s). Instance "PLSExtProc", status UNKNOWN, has 1 handler(s) for this service... Service "canada_DGMGRL" has 1 instance(s). Instance "canada", status UNKNOWN, has 1 handler(s) for this service... Service "india_DGMGRL" has 1 instance(s). Instance "canada", status UNKNOWN, has 1 handler(s) for this service... The command completed successfully [oracle@ORA-C1 admin]$ So far we have performed all the steps as part of the uprade and also the Production database availability, now we have to take care of FarSync and Physical standby database to receive all the redo we have stopped earlier. Primary : Enable the redo transport SQL> show parameter log_archive_dest_state_2 NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ log_archive_dest_state_2 string DEFER SQL> alter system set log_archive_dest_state_2='enable'; System altered. SQL> show parameter log_archive_dest_state_2 NAME TYPE VALUE ------------------------------------ ----------- ------------------------------ log_archive_dest_state_2 string enable As soon as the enabled the remote destination from primary, we can see below from FarSync instance that redo is receiving and FarSync should bypass redo to the Physical standby database. Far Sync Tue Dec 29 11:06:12 2015 Archived Log entry 320 added for thread 1 sequence 231 rlc 899524424 ID 0x68aa1ac5 dest 2: RFS[30]: Opened log for thread 1 sequence 234 dbid 1755994053 branch 899524424 Tue Dec 29 11:06:13 2015 Archived Log entry 321 added for thread 1 sequence 232 rlc 899524424 ID 0x68aa1ac5 dest 2: Tue Dec 29 11:06:13 2015 Archived Log entry 322 added for thread 1 sequence 233 rlc 899524424 ID 0x68aa1ac5 dest 2: RFS[28]: Opened log for thread 1 sequence 236 dbid 1755994053 branch 899524424 RFS[29]: Opened log for thread 1 sequence 235 dbid 1755994053 branch 899524424 Tue Dec 29 11:06:14 2015 Archived Log entry 323 added for thread 1 sequence 234 rlc 899524424 ID 0x68aa1ac5 dest 2: RFS[30]: Opened log for thread 1 sequence 237 dbid 1755994053 branch 899524424 Tue Dec 29 11:06:14 2015 Archived Log entry 324 added for thread 1 sequence 235 rlc 899524424 ID 0x68aa1ac5 dest 2: Standby: From physical standby database we can see the archives/redo is receiving from the FarSync and applying without any issues. Tue Dec 29 20:15:16 2015 Media Recovery Log /u01/app/oracle/fast_recovery_area/INDIA/archivelog/2015_12_29/o1_mf_1_269_c84dlbwk_.arc Tue Dec 29 20:15:16 2015 Media Recovery Log /u01/app/oracle/fast_recovery_area/INDIA/archivelog/2015_12_29/o1_mf_1_270_c84dlcdm_.arc Tue Dec 29 20:15:20 2015 Media Recovery Log /u01/app/oracle/fast_recovery_area/INDIA/archivelog/2015_12_29/o1_mf_1_271_c84d5bc8_.arc Resize operation completed for file# 3, old size 757760K, new size 778240K Media Recovery Waiting for thread 1 sequence 272 (in transit) Tue Dec 29 20:15:21 2015 Recovery of Online Redo Log: Thread 1 Group 7 Seq 272 Reading mem 0 Mem# 0: /u01/app/oracle/oradata/INDIA/onlinelog/o1_mf_7_c7xlzcot_.log Mem# 1: /u01/app/oracle/fast_recovery_area/INDIA/onlinelog/o1_mf_7_c7xlzct0_.log Review of Version We can see Primary, Standby and FarSync are upgraded and running without any issues or without any lag. SQL> select db_unique_name,database_role,version from v$database,v$instance; DB_UNIQUE_NAME DATABASE_ROLE VERSION ------------------------------ ---------------- ----------------- CANADA PRIMARY 12.1.0.2.0 SQL> select db_unique_name,database_role,version from v$database,v$instance; DB_UNIQUE_NAME DATABASE_ROLE VERSION ------------------------------ ---------------- ----------------- CANFAR FAR SYNC 12.1.0.2.0 SQL> select db_unique_name,database_role,version from v$database, v$instance; DB_UNIQUE_NAME DATABASE_ROLE VERSION ------------------------------ ---------------- ----------------- INDIA PHYSICAL STANDBY 12.1.0.2.0 Start Broker Now we can consider to start broker from all the sites and check for the configuration, It is recommended to disable the configuration and enable back. SQL> alter system set dg_broker_start=true; System altered. SQL> [oracle@ORA-C1 admin]$ dgmgrl / DGMGRL for Linux: Version 12.1.0.2.0 - 64bit Production Copyright (c) 2000, 2013, Oracle. All rights reserved. Welcome to DGMGRL, type "help" for information. Connected as SYSDG. DGMGRL> disable configuration; Disabled. DGMGRL> enable configuration; Enabled. DGMGRL> show configuration Configuration - ckpt12c Protection Mode: MaxPerformance Members: canada - Primary database canfar - Far sync instance india - Physical standby database Fast-Start Failover: DISABLED Configuration Status: SUCCESS (status updated 44 seconds ago) DGMGRL> Summary We’ve seen how to upgrade the Data Guard configuration which consists of Primary, Far Sync and the Physical standby database. In fact the more part is to deal on primary database but remaining stuff is based on how we manage it when Data Guard Broker is involved. It is not mandatory to do the steps in same way like above, it depends whether standby changes to perform after the upgrade of Primary. But it is recommended and suggestible to upgrade first on Physical standby database. Regarding the broker, either we can remove the configuration to configure later or the configuration can be copied to new Oracle Home which is already described in this article. Reference: Complete checklist for manual upgrade from 12.1.0.1 to 12.1.0.N (Full CDB Upgrade) (Doc ID 1932762.1)
↧