Error
DataGuard redo apply is broken. No more logs are applied. Alert Log of Standby shows following error.
This happened after server was crashed.
Errors with log +PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Errors in file /ora/app/oracle/diag/rdbms/PRODSBY/PRODSBY2/trace/PRODSBY2_pr00_18575.trc:
ORA-00308: cannot open archived log '+PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751'
ORA-17503: ksfdopn:2 Failed to open file +PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Missing File
As per alertlog below is missing file
+PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Production (PROD)
Login to asmcmd and copy file from prod(PROD) to filesystem
asmcmd -p
cd +PSSFLASH/PROD/ARCHIVELOG/2012_10_04
cp thread_3_seq_19610.2065.795819191 /tmp/thread_3_seq_19610.2065.795819191
Standby Side (PRODSBY)
Stop Recovery if any in progress [ It would have already broken with error ]
SQL > ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
scp <primaryhost>:/tmp/thread_3_seq_19610.2065.795819191 /prodstage01/keepsafe
SQL> ALTER DATABASE REGISTER OR REPLACE PHYSICAL LOGFILE '/prodstage01/keepsafe/thread_3_seq_19610.2065.795819191';
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;
Problem is fixed
Standby Starts Media recovery and will soon catchup
DataGuard redo apply is broken. No more logs are applied. Alert Log of Standby shows following error.
This happened after server was crashed.
Errors with log +PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Errors in file /ora/app/oracle/diag/rdbms/PRODSBY/PRODSBY2/trace/PRODSBY2_pr00_18575.trc:
ORA-00308: cannot open archived log '+PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751'
ORA-17503: ksfdopn:2 Failed to open file +PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Missing File
As per alertlog below is missing file
+PSSFLASH/PRODSBY/archivelog/2012_10_04/thread_3_seq_19610.689.795807751
Production (PROD)
Login to asmcmd and copy file from prod(PROD) to filesystem
asmcmd -p
cd +PSSFLASH/PROD/ARCHIVELOG/2012_10_04
cp thread_3_seq_19610.2065.795819191 /tmp/thread_3_seq_19610.2065.795819191
Standby Side (PRODSBY)
Stop Recovery if any in progress [ It would have already broken with error ]
SQL > ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
scp <primaryhost>:/tmp/thread_3_seq_19610.2065.795819191 /prodstage01/keepsafe
SQL> ALTER DATABASE REGISTER OR REPLACE PHYSICAL LOGFILE '/prodstage01/keepsafe/thread_3_seq_19610.2065.795819191';
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE;
Problem is fixed
Standby Starts Media recovery and will soon catchup