BPEL Server hangs due to java.lang.OutOfMemoryError
Oracle(R) BPEL Process Manager - Version: 10.1.2.0.2
Linux x86
Symptoms
The BPEL server hangs and a java.lang.OutOfMemoryError is displayed in the domain.log file.
You will see the following stack trace in the domain.log
failed to handle message
javax.ejb.EJBException: Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
at com.evermind.server.ejb.EJBUtils.makeException(EJBUtils.java:886)
at
ICubeDeliveryLocalBean_StatelessSessionBeanWrapper16.handleInvoke(ICubeDeliveryLocalBean_StatelessSe
ssionBeanWrapper16.java:1846)
at
com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessageHandler.handle(InvokeInstanceMe
ssageHandler.java:37)
...
.....
Message handle error.
An exception occurred while attempting to process the message
"com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessage"; the exception is:
Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
ORABPEL-05002
Message handle error.
An exception occurred while attempting to process the message
"com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessage"; the exception is:
Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
Cause
The BPEL OC4J container uses 512 MB of memory by default 512MB, which is not enough if you are running BPEL in a high load environment.
You need to increase the JVM settings. Here is are some recommendations for a Linux Box with 4GB of memory and 2 CPUs. (If you are using a different platform from Linux, these parameters may not apply.)
-Xmn1228m -XX:+AggressiveHeap -Xms2048m -Xmx2048m
-Xmn1228m
Garbage collector (Oracle recommends sizing the Eden space to be 60 to 70 percent of the total heap size.)
-XX:+AggressiveHeap
Recommended if you are using two or more CPUs.
The -XX:+AggressiveHeap option inspects the system resources (size of memory and number of processors) and attempts to set various parameters to be optimal for long-running, memory allocation-intensive jobs.
-Xms2048m
Iniatial heapsize. The heap size controls the amount of memory the JVM uses
-Xmx2048m
Maximum heapsize. It is also neccesary to apply some patches related to the memory usage in BPEL:
5638122 -->MERGE LABEL REQUEST ON TOP OF 10.1.2.0.2 BUGS 5149866 5595347 5205630
5499891 -->CHANGES MADE FOR BETTER MEMORY USE TO AVOID OUTOFMEMORY
References
Patch 5499891 - CHANGES MADE FOR BETTER MEMORY USE TO AVOID OUTOFMEMORY
Patch 5638122 - MERGE LABEL REQUEST ON TOP OF 10.1.2.0.2 FOR BUGS 5149866 5595347 5205630
Linux x86
Symptoms
The BPEL server hangs and a java.lang.OutOfMemoryError is displayed in the domain.log file.
You will see the following stack trace in the domain.log
javax.ejb.EJBException: Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
at com.evermind.server.ejb.EJBUtils.makeException(EJBUtils.java:886)
at
ICubeDeliveryLocalBean_StatelessSessionBeanWrapper16.handleInvoke(ICubeDeliveryLocalBean_StatelessSe
ssionBeanWrapper16.java:1846)
at
com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessageHandler.handle(InvokeInstanceMe
ssageHandler.java:37)
...
.....
Message handle error.
An exception occurred while attempting to process the message
"com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessage"; the exception is:
Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
ORABPEL-05002
Message handle error.
An exception occurred while attempting to process the message
"com.collaxa.cube.engine.dispatch.message.invoke.InvokeInstanceMessage"; the exception is:
Transaction was rolled back: java.lang.OutOfMemoryError; nested exception is:
java.rmi.RemoteException: ; nested exception is:
java.lang.OutOfMemoryError
Cause
The BPEL OC4J container uses 512 MB of memory by default 512MB, which is not enough if you are running BPEL in a high load environment.
You need to increase the JVM settings. Here is are some recommendations for a Linux Box with 4GB of memory and 2 CPUs. (If you are using a different platform from Linux, these parameters may not apply.)
-Xmn1228m -XX:+AggressiveHeap -Xms2048m -Xmx2048m
-Xmn1228m
Garbage collector (Oracle recommends sizing the Eden space to be 60 to 70 percent of the total heap size.)
-XX:+AggressiveHeap
Recommended if you are using two or more CPUs.
The -XX:+AggressiveHeap option inspects the system resources (size of memory and number of processors) and attempts to set various parameters to be optimal for long-running, memory allocation-intensive jobs.
-Xms2048m
Iniatial heapsize. The heap size controls the amount of memory the JVM uses
-Xmx2048m
Maximum heapsize. It is also neccesary to apply some patches related to the memory usage in BPEL:
5638122 -->MERGE LABEL REQUEST ON TOP OF 10.1.2.0.2 BUGS 5149866 5595347 5205630
5499891 -->CHANGES MADE FOR BETTER MEMORY USE TO AVOID OUTOFMEMORY
Solution
To implement the solution, please execute the following steps:
1. Shut down your application server
2. Backup the\opmn\conf\opmn.xml
3. Increase the BPEL JVM settings. Edit the file\opmn\conf\opmn.xml :
3.1 Look for the OC4J_BPEL process type section
Your original opmn.xml should look like the example below (just the OC4J_BPEL section)
Look for the entry
-server -Xms256M -Xmx512M
and replace it with
-server -Xmn1228m -XX:+AggressiveHeap -Xms2048m -Xmx2048m
3.2. After the modifications, your opmn.xml file should look like this
New opmn.xml file (just the OC4J_BPEL section):
4. Apply the following patches:
Bug 5638122 MERGE LABEL REQUEST ON TOP OF 10.1.2.0.2 BUGS 5149866 5595347 5205630
Bug 5499891 CHANGES MADE FOR BETTER MEMORY USE TO AVOID OUTOFMEMORY
4) Restart your application server.
References
Patch 5499891 - CHANGES MADE FOR BETTER MEMORY USE TO AVOID OUTOFMEMORY
Patch 5638122 - MERGE LABEL REQUEST ON TOP OF 10.1.2.0.2 FOR BUGS 5149866 5595347 5205630
Comments