Oracle 10g Dataguard Best Practice and setup steps

How it works:

As the primary database processes transactions, the ARCH process on the primary database is configured to transmit the redo data to the standby.  On standby, one or more RFS processes are used to receive redo data and MRP process applies the redo data to the standby.  The primary database also has the Fetch Archive Log (FAL) process to provide a client-server mechanism for transmitting archived logs to the standby following a communication loss between the primary and standby for automatic gap resolution and resynchronization.

Configuration Steps:

This section provides step-by-step instructions on how to setup 10g Dataguard.  Additional information has been provided as notes to elaborate or clarify a point. All steps shown were tested on linux.

Primary host   : prod-lnx-01
Standby host   : stdby-linx-01
ORACLE_SID=SILVER
Kernel   :Linux x86 2.4.21-27.EL (CEL 5)
Service names : SILVER_primary, SILVER_stdby

Initial Setup:

Initialization Parameters:

Primary database init.ora:

instance_name = SILVER
DB_UNIQUE_NAME  = 'SILVER_primary'
_LOG_ARCHIVE_CALLOUT='LOCAL_FIRST=TRUE'
##COMMON TO BOTH PRIMARY AND STANDBY ROLES
log_archive_config ='DG_CONFIG=(SILVER_primary, SILVER_stdby)'
log_archive_dest_1='LOCATION=/oracle/archive/SILVER/SILVER.arch VALID_FOR=(ALL_LOGFILES, ALL_ROLES) DB_UNIQUE_NAME=SILVER_primary'
log_archive_dest_2='SERVICE=SILVER_stdby ARCH VALID_FOR=(ONLINE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=SILVER_stdby max_connections=4 delay=30'
log_archive_dest_state_1=enable
log_archive_dest_state_2=enable
log_archive_max_processes=20
log_archive_format='%t_%s_%r.dbf'
remote_login_passwordfile=EXCLUSIVE
#SPECIFIC TO STANDBY ROLE
FAL_CLIENT=SILVER_primary
FAL_SERVER=SILVER_stdby
standby_archive_dest=/oracle/archive/SILVER/SILVER.arch
standby_file_management=auto

NOTE: Values for log_archive_max_processes, max_connections and delay are just recommended values and can be changed as applicable to the env based on the amount of generated redo. You can refer to the test cases to get some idea on what the values should be.

Standby database init.ora:
instance_name = SILVER
DB_UNIQUE_NAME   = 'SILVER_stdby'
log_archive_config ='DG_CONFIG=(SILVER_primary, SILVER_stdby)'
log_archive_dest_1='LOCATION=/oracle/archive/SILVER/SILVER.arch VALID_FOR=(ALL_LOGFILES, ALL_ROLES) DB_UNIQUE_NAME=SILVER_stdby'
log_archive_dest_2='SERVICE=SILVER_primary VALID_FOR=(ONLINE_LOGFILES, PRIMARY_ROLE) DB_UNIQUE_NAME=SILVER_primary max_connections=4 delay=30'
log_archive_dest_state_1=enable
log_archive_dest_state_2=enable
log_archive_max_processes=20
log_archive_format='%t_%s_%r.dbf'
remote_login_passwordfile=EXCLUSIVE
#SPECIFIC TO STANDBY ROLE
FAL_CLIENT='SILVER_stdby'
FAL_SERVER='SILVER_primary'
standby_archive_dest='/oracle/archive/SILVER/SILVER.arch'
standby_file_management='auto'
# SPECIFIC TO RECOVERY
parallel_max_servers=16
parallel_execution_message_size=16384
recovery_parallelism=4

Note:
Create just one initialization parameter file that contains parameters used in both Roles (primary / standby). Parameters specific to that role will be used eg: FAL_ parameter will be used only when the database is in standby role. For Log_archive_dest_n the VALID_FOR attribute will differentiate the roles, if not specified the default is (ALL_LOGFILES,ALL_ROLES). This VALID_FOR attribute allows us to use the same initialization parameter file for both the primary and standby roles. This will allow to failover without any changes

Note:
- The parameters specific to recovery should be set only on standby database and  they enhance the performance of the recovery. These specific parameters are not recommended on primary.
- Max_connection setting has issue. Refer Issues mentioned at the start of the document
- Settings like Log_archive_max_processes, delay can vary based on the env needs

Configure Net Services:

Note:

The recommendation is to use ports between 1910 to 1920 for Data Guard traffic for DG listener as there is a requirement  for asynchronous traffic over WAN to be marked for Scavenger class and these ports are included in scavenger class Qos list.

What is Scavenger class?
Scavenger class traffic is defined as limited percent of the total bandwidth available. When there is available bandwidth the scavenger class will exceed its  allocation and is in no way policed or shaped. The Qos policies will kick in only when there is no available bandwidth, and in that scenario the traffic will be limited to that allocation (which isn't often).  What it means is during the periods of congestion, the Scavenger class policies get kicked in and the traffic marked for this class will get limited bandwidth, packets might get dropped etc..prior to impacting the critical transactional data

Verify the TCP send/receive size are set to 4665000 (for improved efficiency in log shipping).
cat /proc/sys/net/core/wmem_max
cat /proc/sys/net/core/rmem_max
Add SDU = 32768 in tnsnames.ora  in both Primary and DR.
Setup separate listener for DG
 
On Primary database host:
/etc/listener.ora
LISTENER_SILVER_DG =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST =erpdb-prd-06)(PORT = 1523))
      )
    )
  )
SID_LIST_LISTENER_SILVER_DG =
  (SID_LIST =
    (SID_DESC =
      (SDU=32768)
      (SID_NAME = SILVER)
      (ORACLE_HOME = /oracle/product/10.2.0.2-64)
    )
  )

/etc/tnsnames.ora
SILVER_stdby =
 (DESCRIPTION =
  (SDU = 32768)
   (ADDRESS_LIST =
       (ADDRESS =
         (PROTOCOL = TCP)
         (HOST =hpquadopt)
         (PORT = 1523)
        )
    )
   (CONNECT_DATA =
     (SERVICE_NAME = SILVER)
   )
)
On standby host:
/etc/listener.ora
LISTENER_SILVER_DG =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS_LIST =
        (ADDRESS = (PROTOCOL = TCP)(HOST =hpquadopt)(PORT = 1523))
      )
    )
  )
SID_LIST_LISTENER_SILVER_DG =
  (SID_LIST =
    (SID_DESC =
      (SDU=32768)
      (SID_NAME = SILVER)
      (ORACLE_HOME = /oracle/product/10.2.0.2-64)
    )
  )
SILVER_primary =
 (DESCRIPTION =
  (SDU = 32768)
   (ADDRESS_LIST =
       (ADDRESS =
         (COMMUNITY = portal.com)
         (PROTOCOL = TCP)
         (HOST =erpdb-prd-06)
         (PORT = 1523)
        )
    )
   (CONNECT_DATA =
     (SERVICE_NAME = SILVER)
   )
)

Configure Primary Database

Ensure initialization parameters and net services configuration in place (refer initial setup)
Shutdown the Database
Setup Remote OS authentication
Create the password file if not in place already
orapwd file=/oracle/admin/<SID>/pfile/orapw<SID> password=<sys password>
Ensure sym link is created from $ORACLE_HOME/dbs to /oracle/admin/<SID>/pfile/orapw<SID>
Copy the orapw<sid> from primary to same location on standby
Ensure the Database is in archivelog mode
Startup the Database

Configure Standby Database

1. Ensure initialization parameters and net services configuration in place (refer initial setup)
2. Add entry in /etc/oratab. Put automatic startup as N
3. Copy the datafiles and required archives from cold/hot backup to DR server
4. Create the standby controlfile on Primary host
SQL>ALTER DATABASE CREATE STANDBY CONTROFLE AS '/tmp/stdby.ctl';
Copy the stdby.ctl file to the standby host. Make sure the control_file parameter reflects the correct name/path
Startup the standby database
SQL>STARTUP MOUNT
(optional) Create the SRL's (standby redo logs). Number of SRL’s should be one more than the number of redo logs on primary and the log size should be same as primary
                SQL>ALTER DATABASE ADD STANDBY LOGFILE GROUP 4 size 50 M;
                                 
                SQL>ALTER DATABASE ADD STANDBY LOGFILE GROUP 5 size 50 M;
                 
                SQL>ALTER DATABASE ADD STANDBY LOGFILE GROUP 6 size 50 M;

Note:

The archiver (ARCn) process on the primary database can optionally transmit redo data to remote standby redo logs (SRL’s) if  SRL’s are created.  In a failover situation, you will have access to the information already written in the Standby Redo Logs rather than wait until it is archived completely in case of no SRL’s and hence reducing the risk of loosing data.
MAXLOGFILE defaults to 16 , To create online redologs + standby redologs more than this ensure that you recreate the control file to modify maxlogfile to accomodate this number.

The POC testing was done with and without SRL’s. No issues were noticed with either option. But, if you can not afford to loose any data in case of a failover and you need standby recovered real time, you have to use SRL’s

Verify log shipping

Check to see if the redo from the primary is being shipped to the standby    
            On the primary do the following:
            SQL>ALTER SYSTEM SWITCH LOGFILE;
            SQl>SELECT SEQUENCE#, FIRST_TIME, NEXT_TIME, ARCHIVED, DEST_ID  FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;

            SEQUENCE# FIRST_TIME  NEXT_TIME   ARC    DEST_ID
      ---------- ----------- ----------- --- ----------
      1243 01-OCT-2006 07-OCT-2006 YES          1
      1243 01-OCT-2006 07-OCT-2006 YES          2
      1244 01-OCT-2006 07-OCT-2006 YES          1
      1244 01-OCT-2006 07-OCT-2006 YES          2

Note

In this configuration, arch process is used for log_archive_dest_2 and SRL’s are created on the standby. On the primary database, after the ARC0 process successfully archives the local online redo log to the local destination (LOG_ARCHIVE_DEST_1), the ARC1 process transmits redo from the local archived redo log files (instead of the online redo log files) to the remote standby destination (LOG_ARCHIVE_DEST_2). On the remote destination, the remote file server process (RFS) will in turn write the redo data to an archived redo log file from a standby redo log file.

Verify log apply

Start managed recovery
SQL>ALTER DATABASE RECOVER MANAGED STANDBY DATABASE DISCONNECT;
Verify if the archived logs are applied successfully on the standby
SQL>SELECT SEQUENCE#,APPLIED FROM V$ARCHIVED_LOG ORDER BY SEQUENCE#;

Note:

In our setup, the log apply services on standby are delayed by 90 mins. If we need real time apply enabled, the log apply services can apply redo data as it is received, without waiting for the current standby redo log file to be archived. On the standby cancel out of the current managed recovery and place it back in recovery with Real Time Apply
SQL>ALTER DATABASER RECOVER MANAGED STANDBY DATABASE CANCEL;    
SQL>ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;

Detailed and simple steps for Schema Migration using exp/imp,pipe,gz between two databases

Detailed and simple steps for Schema Migration using exp/imp,pipe,gz between two databases

Start of Migration

On the Source Database
==================
SQL> select name,open_mode from v$database;

NAME      OPEN_MODE
--------- ----------
SOURCE    READ WRITE

1. SQL> select owner,sum(bytes)/1024/1024 from dba_segments where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by owner;

2. SQL> select tablespace_name,sum(bytes)/1024/1024 from dba_segments where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by tablespace_name;

3. SQL> select default_tablespace,username from dba_users where username in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');

4. SQL> select owner,status,object_type,count(*) from dba_objects where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by object_type,owner,status;

5. SQL> select object_type,count(*) from dba_objects where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by object_type;

6. SQL> select count(*),status from dba_objects where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by status;

7. SQL> select object_type,count(*),status from dba_objects where owner in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5') group by status,object_type;

8. SQL> select tablespace_name,sum(bytes)/1024/1024/1000 from dba_data_files  where tablespace_name in('TS1','TS2','TS3','TS4','TS5','TS6','TS7') group by tablespace_name;

9. Export Schemas:
  
   vi exp_SOURCE_schemas.sh

   #!/bin/ksh
   export NLS_LANG=AMERICAN_AMERICA.UTF8
   mknod /tmp/exp_pipe1 p
   gzip -cNf SOURCE_schemas.dmp.gz &
   exp system/sysmic32 file=/tmp/exp_pipe1 buffer=2097152 log=exp_SOURCE_schemas.log statistics=none owner=SCHEMA1,SCHEMA2,SCHEMA3,SCHEMA4,SCHEMA5
   rm -f /tmp/exp_pipe1
   unset NLS_LANG

10. nohup ./exp_SOURCE_schemas.sh > exp_SOURCE_schemas.log 2>&1

11. Copy the SOURCE_schemas.dmp.gz to Target Database server.
    scp -p SOURCE_schemas.dmp.gz ganesh@target_server:/oracle/export


On the Target Database
==================
SQL> select name,open_mode from v$database;

NAME      OPEN_MODE
--------- ----------
TARGET    READ WRITE


1. Create tablespaces as per the output from Step 7 you got on Source database

2. Create users on Target database
   Run the following script on the source database and execute the resultant script on the Target database

   set linesize 300
   set pagesize 300
   spool create_users.sql
   select 'create user '||username||' identified by values '''||password||''' default tablespace '||default_tablespace||' temporary tablespace temp profile '||profile||';' from dba_users where username in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   spool off

3. Grant Quotas on Target database
   Run the following script on the source database and execute the resultant script on the Target database

   spool create_tablespace_quota.sql
   select 'alter user '||username||' quota unlimited on '||tablespace_name||';' from dba_ts_quotas where MAX_BYTES='-1' and username in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   select 'alter user '||username||' quota '||max_bytes||' on '||tablespace_name||';' from dba_ts_quotas where MAX_BYTES!='-1' and username in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   spool off

4. Grant Roles on Target database
   Run the following script on the source database and execute the resultant script on the Target database
   spool create_grant_roles.sql
   select 'grant '||GRANTED_ROLE||' to '||GRANTEE||';' from dba_role_privs where ADMIN_OPTION='NO' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');    
   select 'grant '||GRANTED_ROLE||' to '||GRANTEE||' with admin option;' from dba_role_privs where ADMIN_OPTION='YES' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   spool off

5. Grant System privs on Target database
   spool create_sys_privs.sql
   Run the following script on the source database and execute the resultant script on the Target database
   select 'grant '||PRIVILEGE||' to '||GRANTEE||';' from dba_sys_privs where  ADMIN_OPTION='NO' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   select 'grant '||PRIVILEGE||' to '||GRANTEE||'  with admin option;' from dba_sys_privs where ADMIN_OPTION='YES' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
   spool off


6. Import Schemas

   vi imp_TARGET_schemas.sh

   #!/bin/ksh
   export NLS_LANG=AMERICAN_AMERICA.AL32UTF8
   mknod /tmp/imp_pipe1 p
   gunzip -c SOURCE_schemas.dmp.gz >/tmp/imp_pipe1 &
   imp system/manager99 file=/tmp/imp_pipe1  buffer=20971520 log=imp_TARGET_schemas.log full=y
   rm -f /tmp/imp_pipe1
   unset NLS_LANG

7. Verify the logs

8. Do step 1 to step 7 that you have done on the source database.

9.  Grant Table privs on the Target
    Run the following script on the source database and execute the resultant script on the Target database
    spool create_tab_privs.sql
    select 'grant '||privilege||' on '||owner||'.'||table_name||' to '||grantee||';' from dba_tab_privs where GRANTABLE='NO' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
    select 'grant '||privilege||' on '||owner||'.'||table_name||' to '||grantee||' with grant option;' from dba_tab_privs where GRANTABLE='YES' and grantee in ('SCHEMA1','SCHEMA2','SCHEMA3','SCHEMA4','SCHEMA5');
    spool off


9. EXEC DBMS_UTILITY.COMPILE_SCHEMA ('SCHEMA1'); -> Do for all schemas that you have imported

10. exec dbms_stats.gather_schema_stats('SCHEMA1', cascade=>TRUE); -> Do for all schemas that you have imported

End of Migration

Where are the log files located in R12?

Where are the log files located in R12?

For DBA’s who mostly worked on 11i environments, finding the log files (Concurrent manager log, apache log etc.,) on a R12 environment might pose a threat initially, because these log files no longer reside in their old location ie., $APPLCSF/$APPLLOG or $APACHE_TOP/Apache/logs.
In R12, the log files are located in $LOG_HOME (which translates to $INST_TOP/logs)
Concurrent Reqeust related logs
$LOG_HOME/appl/conc -> location for concurrent requests log and out files
$LOG_HOME/appl/admin -> location for mid tier startup scripts log files
Apache Logs (10.1.3 Oracle Home which is equivalent to iAS Oracle Home)
$LOG_HOME/ora/10.1.3/Apache -> Location for Apache Error and Access log files
$LOG_HOME/ora/10.1.3/j2ee -> location for j2ee related log files
$LOG_HOME/ora/10.1.3/opmn -> location for opmn related log files
Forms & Reports related logs (10.1.2 Oracle home which is equivalent to 806 Oracle Home)
$LOG_HOME/ora/10.1.2/forms
$LOG_HOME/ora/10.1.2/reports
Related metalink notes to enable additional debugging
419839.1 – How to enable Apache, OC4J and OPMN logging in Oracle Applications R12
422419.1 – R12 – How To Enable and Collect Debug for HTTP, OC4J and OPMN

Source : http://oraclepitstop.wordpress.com

How to enable Apache and Jserv Debugging in Oracle Apps 11i?

PURPOSE:-If a user experiences an error, request a screen shot of the error and any further error details that they can report.If the error is reproducible, enable the following middle tier debug settings and collect diagnostic information appropriately and upload to support for troubleshooting.

Looking for Jserv.log at $IAS_ORACLE_HOME/Apache/Jserv/logs then enable…
For Temporary purpose manually editing configuration files.
1) Shut Down Apache Server
2) Backup the existing files in the following files:
$IAS_ORACLE_HOME/Apache/Apache/logs
$IAS_ORACLE_HOME/Apache/Jserv/logs
$IAS_ORACLE_HOME/Apache/Jserv/logs/jvm
3) Enable debug logging for Apache
s_apache_loglevel=debug (In Context File)
Or
loglevel debug (In $IAS_ORACLE_HOME/Apache/Apache/conf/httpd.conf)
4) Enable debug logging for mod_jserv
s_apjservloglevel=debug (In Context File)
Or
ApJServLogLevel debug (In $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.conf)
5) Enable debug logging for Jserv
s_oacorelog=true (In Context File)
Or
log=true log.channel=true log.channel.info=true log.channel.debug=true
(In $IAS_ORACLE_HOME/Apache/Jserv/etc/jserv.properties)
6) Enable debug logging for Forms Listener Servlet JVM. (Optional)
s_oaformslog=true (In Context File)
Or
log=true log.channel=true log.channel.info=true log.channel.debug=true
(In $IAS_ORACLE_HOME/Apache/Jserv/etc/forms.properties)
7) Restart Apache
(If Context File had been edited in above steps then run autoconfig to update the configuration files.) 8) Replicate the Problem.
9) Review the following files from the $IAS_ORACLE_HOME/Apache/Jserv/logs directory
a) mod_jserv.log
b) jserv.log
c) All files from the $IAS_ORACLE_HOME/Apache/Jserv/logs/jvm directory These will
have filenames of the form jvm.jvm.stdoutstderr For example,
OACoreGroup.0.stderr
d) $COMMON_TOP/rgf/instance_hostname/javacache.log

Source: http://leenus2b.wordpress.com

For R12: How To : How to enable Apache, OC4J and OPMN logging in Oracle Applications R12  use Doc id [ID 419839.1]

Forms Servlet mode to Socket mode conversion on Release 12 Apps

How to convert Forms server to Socket mode  from Servlet mode in Oracle Apps R12

Run the following command to enable Forms Socket Mode on Forms/web nodes (Place correct context file name and port value)

$FND_TOP/bin/txkrun.pl -script=ChangeFormsMode -contextfile=$CONTEXT_FILE -mode=socket  -port=9030 -runautoconfig=No


Where port = forms port and contextfile=your environment contextfile

To Start/Stop/check Status of forms servers use following scripts

cd $ADMIN_SCRIPTS_HOME
./adformsrvctl.sh status/stop/start