Monday, December 21, 2015

Troubleshooting in-doubt transactions



http://www.oracle11ggotchas.com/articles/ProceduretoROLLBACKFORCEpendingin-doubttransaction.htm


Copied as it is from the above link.

Procedure to ROLLBACK FORCE pending in-doubt transaction


Below is the procedure necessary to force the rollback of a failed distributed transaction, known as an “in-doubt transaction”. Oracle uses a two phase commit (2PC) mechanism to commit changes locally and remotely in a distributed transaction.

The following error is normally associated with an in doubt transaction. Here, a network “glitch” has caused the ORA-03113.

ORA-03113: end-of-file on communication channel

In addition to the ORA-03113 is ORA-02050, stating that the local in-doubt transaction has been rolled back automatically, this can be seen in the local database instance alert log:

Error 3113 trapped in 2PC on transaction 70.31.1376339. Cleaning up.
Error stack returned to user:
ORA-02050: transaction 70.31.1376339 rolled back, some remote DBs may be in-doubt
ORA-03113: end-of-file on communication channel
Thu Jun 09 12:28:32 2011
DISTRIB TRAN REMDB.WORLD.f9784a67.3.9.924681
  is local tran 70.31.1376339 (hex=46.1f.150053)
  insert pending collecting tran, scn=6187118039 (hex=1.70c7edd7)

However, this is not always the case, as the transaction is seen as pending in the “pending two phase commit” view (DBA_2PC_PENDING)

SQL> select LOCAL_TRAN_ID, GLOBAL_TRAN_ID, STATE, FAIL_TIME, RETRY_TIME from dba_2pc_pending;

LOCAL_TRAN_ID GLOBAL_TRAN_ID                  STATE     FAIL_TIME RETRY_TIME
------------- ------------------------------- --------- --------- ----------
70.31.1376339 REMDB.WORLD.f9784a67.3.9.924681 collecting 09-JUN-11 09-JUN-11

If the state of the transaction is “prepared”, it is possible to force rollback the transaction by appending the transaction id to the command as follows (as sysdba):

SQL> ROLLBACK FORCE '70.31.1376339';


If the state of the transaction is “collecting”, you will suffer the following error:

SQL> ROLLBACK FORCE '70.31.1376339';
ROLLBACK FORCE '70.31.1376339'
*
ERROR at line 1:
ORA-02058: no prepared transaction found with ID 70.31.1376339

In this case, you need to execute the following procedure in the DBMS_TRANSACTION package to clear.

SQL> execute dbms_transaction.purge_lost_db_entry('70.31.1376339')

PL/SQL procedure successfully completed.

Rerun the query against DBA_2PC_PENDING to confirm the pending local transaction has gone.

SQL>  select LOCAL_TRAN_ID, GLOBAL_TRAN_ID, STATE, FAIL_TIME, RETRY_TIME from dba_2pc_pending;

no rows selected

Also check DBA_2PC_NEIGHBORS to confirm the pending remote transaction has gone.

SQL> select database,local_tran_id,dbid,sess#,branch from dba_2pc_neighbors;

no rows selected



 

eclipse shortcuts


Below are few eclipse shortcuts that can be used for code navigation








































































































































Ctrl+O     Show code outline / structure
F2     Open class, method, or variable information (tooltip text)
F3     Open Declaration: Jump to Declaration of selected class, method, or parameter
F4     Open Type Hierarchy window for selected item
Ctrl+T     Show / open Quick Type Hierarchy for selected item
Ctrl+Shift+T     Open Type in Hierarchy
Ctrl+Alt+H     Open Call Hierarchy
Ctrl+Shift+U     Find occurrences of expression in current file
Ctrl+move over method     Open Declaration or Implementation










Sunday, December 13, 2015

Cleaning up the EJBTimers DB


WebSphere will use an internal derby database for EJBTimers to persist .
The ejbtimers will survive the server crash as well as server restart as they are persisted in database.
EJB Timer service settings can be changed via the WebSphere console. The description for settings can be easily found in the IBM knowledge center. 
Below is one of the links for WebSphere 8.5.5
EJBTimer can also be easily configured to persist in a different database like oracle. What we need to define is just a different DataSource with an authentication alias. While server is restarted , WebSphere will automatically create the 4 tables required by EJBTimer service. You may use any DataSource available as EJBTimer tables will not interfere with other schema and table’s .You can also define a prefix for the EJBTimer tables in the EJBTimer service settings. Make sure to have the create table permission for authentication alias you are using.
There are also some other important parameters for EJBTimer like poll interval, number of timer threads which can be used to customize the behavior of EJBTimer as per your requirements.
In case, you are using the default EJBTimer DB which is derby. Then, how can you clean the DB. This is an unanswered question in IBM knowledge center.
The procedure is simple which I have found by trial and error methods.
Stop the server. This server is the server for which you are planning to clean the EJBTimer db.
Go to
WAS_INSTALL_ROOT/profiles/Profile Name/databases/EJBTimers/Server Name/EJBTimerDB
and rename the existing directory EJBTimerDB to  EJBTimerDB_bak.
Later you can remove this directory after few days of testing. I am recommending this as this is a solution I found by trial and error and there is no official documentation for it.
Start the server.
You can verify that after server is restarted, a new directory EJBTimerDB is created
Location of an example directory on windows operating system is shown below.
I have tested the proposed solution in Solaris as well as windows operating system.