Wednesday, October 21, 2009

Spring事务管理以及与数据库连接的关系

这里写两个碰到过的有意思的问题。

Question 1

Q1是由于oracle允许的最大连接数太小导致并发用户多的时候出现服务器无响应的问题。 Q2是Spring的AOP事务控制的问题,可能出现commit a rollback-only transaction.
先交代一下上下文(因为问题都是在同一个项目总碰到的),项目用的是Sprint+JPA(hibernate), spring的事务声明如下:
<tx:advice id="defaultTxAdvice" transaction-manager="transactionManager">
        <tx:attributes>
                <!-- Keep SequenceService in a isolation transaction -->
                <tx:method name="get*" read-only="true" />
                <tx:method name="verifyPayoutLimit" read-only="true" />
                <tx:method name="enquiry" read-only="true" />
                <!-- By default, A runtime exception will rollback transaction. -->
                <tx:method name="*" rollback-for="ApplicationException" />
        </tx:attributes>
</tx:advice>

<tx:advice id="asynTxAdvice" transaction-manager="transactionManager">
        <tx:attributes>
                <!-- Keep SequenceService in a isolation transaction -->
                <tx:method name="get*" read-only="true" />
                <!-- By default, A runtime exception will rollback transaction. -->
                <tx:method name="*" propagation="REQUIRES_NEW"
                        rollback-for="ApplicationException" />
        </tx:attributes>
</tx:advice>

<aop:config>
        <aop:pointcut id="service"
                expression="execution(* com.mpos.lottery.te..service.*Service.*(..))" />
        <aop:pointcut id="asynService"
                expression="execution(* com.mpos.lottery.te..service.*ServiceAsyn.*(..))" />
        <aop:advisor advice-ref="defaultTxAdvice" pointcut-ref="service" />
        <aop:advisor advice-ref="asynTxAdvice" pointcut-ref="asynService" />
</aop:config>

先说Q1。比如有两个service:TicketService和SequenceServiceAsyn,并且TicketService需要调用SequenceServiceAsyn的服务。预先设置了oracle的最大允许连接数为100(可以通过show parameter processes命令查看),比如这个时候恰好有个100个并发用户访问TicketService,而且TicketService还没有执行到需要调用SequenceAsyn的服务那一步, 这意味着oracle的100个连接会被这100个用户的会话(线程)占用。这个时候,有一个用户的会话的TicketService开始调用SequenceServiceAsyn的服务,从Spring的事务声明来看,这个时候需要启动一个新的事务,所以这个会话就会要求获得一个新的数据库连接,但是连接都已经用完,oracle服务器无法提供更多的连接,所以这个会话开始等待,并且一直尝试建立新连接。 那么其他的会话呢?其他所有的会话都会走到这里,然后尝试从数据库获得一个新的连接,但是没有一个会成功,而他们本身获得的连接只有在事务提交后才会释放。。。最终导致服务器无法响应。当然这个问题也可能用另外一种面貌展示出来,但是问题是一样,要么增加数据库的最大允许连接数,或者控制客户端的请求。

Question 2

然后说Q2。 这里通过一个叫batch validation的交易来说明这个问题,这个交易的控制流是这样的。FacadeService.facade->InstantTicketService.batchValidate->InstantTicketService.valdate->MerchantService.verfiyPayoutLimit. 因为以前关于这个问题已经在代码中进行了描述,这里也不重复了,直接copy过来(chinese-english)。

Why change the name of "MerchantService" to "MerchantManager"? What does it interfere? In spring-service.xml, we state the transaction boundary as below: "execution(* com.mpos.lottery.te..service.*Service.*(..))" "PROPAGATION_REQUIRED,ISOLATION_DEFAULT,-ApplicationException" and transaction will be roll back when catch a ApplicationException or RuntimeException.
When client issue a 'batch validation' request, TE will handle these tickets in request one by one. Even fail to validate one ticket, the other tickets are still can be validated maybe. When handle 'batch validation', Spring will create a transaction(A) when enter FacadeService.facade which is a facade for all services. Then the control flow enter InstantTicketService#batchValidate, Spring will get transaction for this cut-point(due to PROPAGATION is REQUIRED):
  1. [TransactionInterceptor] Getting transaction for [com.mpos.lottery.te.instantgame.service.InstantTicketService.batchValidate]
  2. [TransactionSynchronizationManager] Retrieved value [org.springframework.orm.jpa.EntityManagerHolder@4dd413] for key [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean@1a4a1c9] bound to thread [http-8080-1]
Spring will retrieve the exist transaction(A) which has been created when enter FacadeService.facade and bound to current thread.
Methond 'batchValidate' will invoke MerchantService.veryfPayoutLimit to check if the actual payout amount exceed allowed max payout amount of a merchant. In the same way, when enter MerchantService#verifyPayoutLimit, Spring will get the exist transaction(A) from current thread. Here, if fail to pass this checking, a ApplicationException(code=341) will be thrown out from MerchantService. When quit MerchantService.verifyPayoutLimit, Spring will complete the transaction of this service.
  1. [TransactionInterceptor] Completing transaction for [com.mpos.lottery.te.merchant.service.MerchantService.verifyPayoutLimit] after exception: com.mpos.lottery.te.config.domain.exception.ApplicationException: ...
  2. [RuleBasedTransactionAttribute] Applying rules to determine whether transaction should rollback on com.mpos.lottery.te.config.domain.exception.ApplicationException: ...
  3. [RuleBasedTransactionAttribute] Winning rollback rule is: RollbackRuleAttribute with pattern [ApplicationException]
  4. [JpaTransactionManager] Participating transaction failed - marking existing transaction as rollback-only
  5. [JpaTransactionManager] Setting JPA transaction on EntityManager [org.hibernate.ejb.EntityManagerImpl@10463c3] rollback-only
Now, Spring has marked this transaction(A) as rollback-only, but when InstantTicketService catch this ApplicationException, it will only fail to validate this ticket, and then continue to handle the next ticket. Let's say the validation of next ticket is successful, and eventually no ApplicationException or RuntimeException will be thrown out when quit FacadeService.facade. So Spring will plan to commit this rollback-only transaction(A), then a excetpion will be thrown out by Spring:
     Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: 
         Transaction marked as rollbackOnly

         org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction marked as rollbackOnly

        at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:465)

        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709)

        ......
 
That is why I change 'MerchantService' to 'MerchantManager'. After renaming, Spring will not manage the transaction when enter MerchantManager#verfyPayoutLimit. ANOTHER approach is set the transaction of 'MerchantService#verifyPayoutLimit' as read-only.
NOTE that Spring won't get transaction for InstantTicketService.validate. why? In fact, when Spring start to initialize the whole ApplicationContext or BeanFactory, it will find the AOP definition first, Here let's focus on Service pointcut. Then when initialize the service instance, for example 'MerchantServiceImpl', it find 'MerchantServiceImpl' implements 'MerchantService' which is in Service pointcut, so sprint will return a dynamic proxy which will target to MerchantServiceImpl instance..
  1. [AspectJAwareAdvisorAutoProxyCreator] Creating implicit proxy for bean 'merchantService' with 0 common interceptors and 2 specific interceptors
  2. [JdkDynamicAopProxy] Creating JDK dynamic proxy: target source is SingletonTargetSource for target object [com.mpos.lottery.te.merchant.service.impl.MerchantServiceImpl@c39410]
The mechanism of JDK dynamic proxy:
  1. A proxy instance is extends from java.lang.reflect.Proxy
  2. One proxy instance will associate with only one InvocationHandler instance.
  3. The invocation on proxy instance(MerchantService$Proxy) will be dispatched to InvocationHandler instance, and then InvocationHandler instance dispatch the request to target instance(MerhcantServiceImpl)
  4. InvocationHandler instance will create transaction context before dispatching request, or do some other things
Now come back to the question, why doesn't Spring get a transaction for InstantTicketService.validate? The flow should be below:
  1. Enter FacadeService.facade, in fact client invoke FacadeService$Proxy.facade. This proxy(In fact proxy will dispatch reqeust to InvocationHandler, or interceptor) will check if a transaction has bound with current thread, if not, create one, and bount it to current thread. Then control flow enter FacadeServiceImpl.facade.
  2. Enter InstantTicketService.batchValidate(in fact InstantTicketService$Proxy.batchValidate), this proxy will also check transaction context too. Then control flow enter InstantTicketServiceImpl.batchValidate
  3. Enter InstantTicketServiceImpl.validate, not InstantTicketService$Proxy...Due to invoke validate \ from batchValidate directly, not access to IntantTicketService$Proxy...
  4. Enter MerchantService$Proxy, then InvocationHandler(interceptor), and then MerchantServiceImpl$Proxy
  5. Quit MerchantServiceImpl.verfiyPayoutLimit
As described above, InstantTicketService.validate is invoked from target class InstantTicketServiceImpl.batchValidate directly, Spring doesn't aware of this invocation, so no transaction will be got for batchValidate.
public class MerchantServiceImpl implements MerchantService {
 ...
}