Nuts and Bolts of Transaction ProcessingIntroductionTransaction management is one of the most crucial requirements for enterprise application development. Most of the large enterprise applications in the domains of finance, banking and electronic commerce rely on transaction processing for delivering their business functionality. Given the complexity of today’s business requirements, transaction processing occupies one of the most complex segments of enterprise level distributed applications to build, deploy and maintain. This article walks the reader through the following:
This article is not specific to any product, and so attempts to be generic while describing various issues and concepts. This article does not aim to compare various transaction processing technologies/standards, and offers a study only. What is a Transaction?Enterprise applications often require concurrent access to distributed data shared amongst multiple components, to perform operations on data. Such applications should maintain integrity of data (as defined by the business rules of the application) under the following circumstances:
In such cases, it may be required that a group of operations on (distributed) resources be treated as one unit of work. In a unit of work, all the participating operations should either succeed or fail and recover together. This problem is more complicated when
In either case, it is required that success or failure of a unit of work be maintained by the application. In case of a failure, all the resources should bring back the state of the data to the previous state (i.e., the state prior to the commencement of the unit of work). The concept of a transaction, and a transaction manager (or a transaction processing service) simplifies construction of such enterprise level distributed applications while maintaining integrity of data in a unit of work. A transaction is a unit of work that has the following properties:
These properties, called as ACID properties, guarantee that a transaction is never incomplete, the data is never inconsistent, concurrent transactions are independent, and the effects of a transaction are persistent. For a brief description of what can go wrong in distributed transaction processing, see Fault Tolerance and Recovery in Transaction Processing Systems. Issues in Building Transactional ApplicationsTo elicit the issues involved in building transactional applications, consider an order capture and order process application with the architecture shown in Figure 1.
This application consists of two client components implementing the order capture and order process operations respectively. These two operations constitute a unit of work or transaction. The order capture and order process components access and operate on four databases for products, orders, inventory and shipping information respectively. In this figure, while the dotted arrows indicate read-only data access, the continuous arrows are transactional operations modifying data. The following are the transactional operations in this application:
While implementing these operations as a single transaction, the following issues should be addressed:
Transaction Processing – ArchitectureHaving seen the issues in building transactional applications from scratch, consider the same application built around a transaction processing architecture as shown in Figure 2. Note that, although there are several architectures possible, as will be discussed in a later section, the one shown in Figure 2 represents the essential features.
This architecture introduces a transaction manager and a resource manager for each database (resource). These components abstract most of the transaction specific issues from application components (Order Capture and Order Process), and share the responsibility of implementation of transactions. The various components of this architecture are discussed below. Application Components
Application components are clients for the transactional resources. These are the programs with which the application developer implements business transactions. With the help of the transaction manager, these components create global transactions, propagate the transaction context if necessary, and operate on the transactional resources with in the scope of these transactions. These components are not responsible for implementing semantics for preserving ACID properties of transactions. However, as part of the application logic, these components generally make a decision whether to commit or rollback transactions. Resource Managers
A resource manager is a component that manages persistent and stable data storage system, and participates in the two phase commit and recovery protocols with the transaction manager. A resource manager is typically a driver or a wrapper over a stable storage system, with interfaces for operating on the data (for the application components), and for participating in two phase commit and recovery protocols coordinated by a transaction manager. This component may also, directly or indirectly, register resources with the transaction manager so that the transaction manager can keep track of all the resources participating in a transaction. This process is called as resource enlistment. For implementing the two-phase commit and recovery protocols, the resource manager should implement supplementary mechanisms using which recovery is possible. Resource managers provide two sets of interfaces: one set for the application components to get connections and perform operations on the data, and the other set for the transaction manager to participate in the two-phase commit and recovery protocol. Transaction Manager
The transaction manager is the core component of a transaction processing environment. Its primary responsibilities are to create transactions when requested by application components, allow resource enlistment and delistment, and to conduct the two-phase commit or recovery protocol with the resource managers. A typical transactional application begins a transaction by issuing a request to a transaction manager to initiate a transaction. In response, the transaction manager starts a transaction and associates it with the calling thread. The transaction manager also establishes a transaction context. All application components and/or threads participating in the transaction share the transaction context. The thread that initially issued the request for beginning the transaction, or, if the transaction manager allows, any other thread may eventually terminate the transaction by issuing a commit or rollback request. Before a transaction is terminated, any number of components and/or threads may perform transactional operations on any number of transactional resources known to the transaction manager. If allowed by the transaction manager, a transaction may be suspended or resumed before finally completing the transaction. Once the application issues the commit request, the transaction manager prepares all the resources for a commit operation (by conducting a voting), and based on whether all resources are ready for a commit or not, issues a commit or rollback request to all the resources. The following sections discuss various concepts associated with transaction processing. Transaction Processing – ConceptsTransaction DemarcationA transaction can be specified by what is known as transaction demarcation. Transaction demarcation enables work done by distributed components to be bound by a global transaction. It is a way of marking groups of operations to constitute a transaction. The most common approach to demarcation is to mark the thread executing the operations for transaction processing. This is called as programmatic demarcation. The transaction so established can be suspended by unmarking the thread, and be resumed later by explicitly propagating the transaction context from the point of suspension to the point of resumption. The transaction demarcation ends after a commit or a rollback request to the transaction manager. The commit request directs all the participating resources managers to record the effects of the operations of the transaction permanently. The rollback request makes the resource managers undo the effects of all operations on the transaction. An alternative to programmatic demarcation is declarative demarcation. Component based transaction processing systems such as Microsoft Transaction Server, and application servers based on the Enterprise Java Beans specification support declarative demarcation. In this technique, components are marked as transactional at the deployment time. This has two implications. Firstly, the responsibility of demarcation is shifted from the application to the container hosting the component. For this reason, this technique is also called as container managed demarcation. Secondly, the demarcation is postponed from application build time (static) to the component deployment time (dynamic). Transaction Context and PropagationSince multiple application components and resources participate in a transaction, it is necessary for the transaction manager to establish and maintain the state of the transaction as it occurs. This is usually done in the form of transaction context. Transaction context is an association between the transactional operations on the resources, and the components invoking the operations. During the course of a transaction, all the threads participating in the transaction share the transaction context. Thus the transaction context logically envelops all the operations performed on transactional resources during a transaction. The transaction context is usually maintained transparently by the underlying transaction manager. Resource EnlistmentResource enlistment is the process by which resource managers inform the transaction manager of their participation in a transaction. This process enables the transaction manager to keep track of all the resources participating in a transaction. The transaction manager uses this information to coordinate transactional work performed by the resource managers and to drive two-phase commit and recovery protocol. At the end of a transaction (after a commit or rollback) the transaction manager delists the resources. Thereafter, association between the transaction and the resources does not hold. Two-Phase CommitThis protocol between the transaction manager and all the resources enlisted for a transaction ensures that either all the resource managers commit the transaction or they all abort. In this protocol, when the application requests for committing the transaction, the transaction manager issues a prepare request to all the resource managers involved. Each of these resources may in turn send a reply indicating whether it is ready for commit or not. Only when all the resource managers are ready for a commit, does the transaction manager issue a commit request to all the resource managers. Otherwise, the transaction manager issues a rollback request and the transaction will be rolled back. Transaction Processing – Standards and TechnologiesX/Open Distributed Transaction Processing ModelThe X/Open Distributed Transaction Processing (DTP) model is a distributed transaction processing model proposed by the Open Group, a vendor consortium. This model is a standard among most of the commercial vendors in transaction processing and database domains. This model consists of four components:
This model also specifies the following interfaces:
The X/Open DTP model is well established in the industry. A number of commercial transaction management products, such as TXSeries/Encina (from Tranarc, a wholly owned subsidiary of IBM), Tuxedo and TopEnd (both from BEA Systems), and AT&T GIS support the TX interface. Although Microsoft’s Transaction Server does not support the TX interface, it can interoperate with XA compliant databases, such as Oracle. Similarly, most of the commercial databases such as Oracle, Sybase, Informix and Microsoft SQL Server, and messaging middleware products like IBM’s MQSeries, and Microsoft’s MSMQ Server provide an implementation of the XA interface. OMG Object Transaction ServiceObject Transaction Service (OTS) is a distributed transaction processing service specified by the Object Management Group (OMG). This specification extends the CORBA model and defines a set of interfaces to perform transaction processing across multiple CORBA objects. The OTS model is based on the X/Open DTP model with the following enhancements:
However, the OTS is interoperable with X/Open DTP model. An application using transactional objects could use the TX interface with the transaction manager for transaction demarcation. The OTS architecture consists of the following components:
In addition to the usual transactional semantics, the CORBA OTS provides for the following:
The following are the principal interfaces in the CORBA OTS specification.
The following products offer implementations of the OTS: Integrated Transaction Service (from Inprise), OrbixOTM (from Iona), OTSARjuna (from Arjuna Solutions Limited), and TPBroker (from Hitachi Software). JTA and JTSThe Java Transaction Service and the Java Transaction API are the latest entrants into the enterprise distributed computing arena. As a part of the enterprise Java initiative, Sun Microsystems Inc. proposed these specifications in early 1999.
JTS specifies the implementation of a Java transaction manager. This transaction manager supports the JTA, using which application servers can be built to support transactional Java applications. Internally the JTS implements the Java mapping of the OMG OTS 1.1 specifications. The Java mapping is specified in two packages: org.omg.CosTransactions and org.omg.CosTSPortability. Although the JTS is a Java implementation of the OMG OTS 1.1 specification, the JTA retains the simplicity of the XA and TX functional interfaces of the X/Open DTP model. The JTA specifies an architecture for building transactional application servers and defines a set of interfaces for various components of this architecture. The components are: the application, resource managers, and the application server, as shown in Figure 3. The JTS thus provides a new architecture for transactional application servers and applications, while complying with the OMG OTS 1.1 interfaces internally. This allows the JTA compliant applications to interoperate with other OTS 1.1 complaint applications through the standard IIOP. As shown in Figure 3, in the Java transaction model, the Java application components can conduct transactional operations on JTA compliant resources via the JTS. The JTS acts as a layer over the OTS. The applications can therefore initiate global transactions to include other OTS transaction managers, or participate in global transactions initiated by other OTS compliant transaction managers. For more details on JTS and JTA, see Java Transaction Service. Microsoft Transaction ServerThe Microsoft Transaction Server (MTS) is a component based transaction server for components based on the Microsoft’s Component Object Model (COM). The MTS programming model provides interfaces for building transactional COM components, while the MTS runtime environment provides a means to deploy and manage these components and manage transactions. Using the MTS, work done by multiple COM components can be composed into a single transaction. Unlike other technologies discussed in this section, MTS is a product and is not based on open specifications. Also note that, although the MTS environment offers several other features such as resource pooling, object recycling, access control etc., this section focuses only on the transactional capabilities of MTS, and attempts to map the various transaction management concepts to the MTS environment. MTS ArchitectureThe high level architecture of MTS is shown in Figure 4.
The MTS environment consists of the following:
MTS Objects and Transaction ContextAn MTS object is an instance of an MTS component (a component deployed on MTS, and managed by MTS). For each MTS object, the MTS creates and maintains a context object (ObjectContext) which provides execution context for an MTS object. The context object also maintains information of transaction context. Resource dispensers and the DTC can access this transaction context information for transaction demarcation, resource enlistment, delistment, two-phase commit etc. Note that, in MTS, the transaction context information is maintained with each MTS object, as opposed to a single transaction context object for all the objects participating in a transaction. Transaction OutcomeEach MTS object can participate in determining the outcome of a transaction by calling one of the methods on the ObjectContext object:
Transaction DemarcationMTS allows both programmatic and declarative demarcation of transactions. Declarative demarcation is compulsory for all components deployed on the MTS. In addition, MTS clients can also initiative and end transactions programmatically.
Resource EnlistmentMTS does automatic resource enlistment. When a MTS object requests the resource dispenser for a connection to a resource, the resource dispenser obtains the calling object’s transaction context, and registers the connection with it. Although MTS is available for Microsoft Windows platforms only, MTS can interoperate with resource managers complying to the XA protocol, and such resource managers operating on non-Windows platforms can participate in transactions coordinated by the DTC. For more information on MTS, refer to the MSDN library. For a quick but elaborate compilation of features of MTS vis-a-vis other competing technologies, refer to MTS FAQ. Enterprise Java BeansEnterprise Java Beans (EJB) is a technology specification from Sun Microsystems Inc. that specifies a framework for building component-based distributed applications. Application servers conforming to this technology are beginning to appear from various vendors over the past six months, while the specification is being improved currently by Sun Microsystems Inc. As an application server framework, the EJB servers address transaction processing, resource pooling, security, threading, persistence, remote access, life cycle etc. However, as with the case of MTS, this section focuses only on distributed transactional model of the EJB framework. The EJB framework specifies construction, deployment and invocation of components called as enterprise beans. The EJB specification classifies enterprise beans into two categories: entity beans and session beans. While entity beans abstract persistent domain data, session beans provide for session specific application logic. Both types of beans are maintained by EJB compliant servers in what are called as containers. A container provides the run time environment for an enterprise bean. Figure 5 shows a simplified architecture of transaction management in EJB compliant application servers. This figure shows only the essential interactions between the constituent parts of the architecture.
An enterprise bean is specified by two interfaces: the home interface and the remote interface. The home interface specifies how a bean can created or found. With the help of this interface, a client or another bean can obtain a reference to a bean residing in a container on an EJB server. The remote interface specifies application specific methods that are relevant to entity or session beans. Clients obtain references to home interfaces of enterprise beans via the Java Naming and Directory Interface (JNDI) mechanism. An EJB server should provide a JNDI implementation for any naming and directory server. Using this reference to the home interface, a client can obtain a reference to the remote interface. The client can then access methods specified in the remote interface. The EJB specification specifies the Java Remote Method Invocation (RMI) as the application level protocol for remote method invocation. However, an implementation can use IIOP as the wire-level protocol. In Figure 5, the client first obtains a reference to the home interface, and then a reference to an instance of Bean A via the home interface. The same procedure is applicable for instance of Bean A to obtain a reference and invoke methods on an instance of Bean B. EJB Transaction ModelThe EJB framework does not specify any specific transaction service (such as the JTS) or protocol for transaction management. However, the specification requires that the javax.transaction.UserTransaction interface of the JTS be exposed to enterprise beans. This interface is required for programmatic transaction demarcation as discussed in the next section. Similar to the MTS, the EJB framework provides for declarative demarcation of transactions. The container performs automatic demarcation depending on the transaction attributes specified at the time of deploying an enterprise bean in a container. The following attributes determine how transactions are created.
Transaction DemarcationThe EJB framework supports three types of transaction demarcation.
Resource EnlistmentResource enlistment is automatic with EJB. The EJB containers automatically enlists connections to EJB-aware resource managers whenever a bean obtains a connection. Application SynchronizationThe EJB specification provides the javax.ejb.SessionSynchronization interface for application synchronization. When implemented by a bean, the container calls the afterBegin, beforeCompletion and afterCompletion methods for application synchronization during the two-phase commit process. There are several vendors who currently offer application servers complying to the EJB 1.0 specification. Some of the products are WebLogic from BEA Systems Inc., GemStone/J from GemStone Systems Inc., NetDynamics from Sun Microsystems Inc., Oracle Application Server from Oracle Corporation, and PowerTier for EJB from Persistence Software Inc.. Refer to the EJB Directory for a listing of EJB application servers and EJB-aware database servers. The specification is currently undergoing a major overhaul, and products complying to the new standard are expected by late 1999 or early 2000. In summary, the EJB framework provides features almost similar to the MTS. Both the technologies allow component-based transactional distributed applications, and abstract the process of transaction demarcation from application components. ConclusionTransaction processing has always been complex and critical. However, with the advent of MTS and EJB, transaction processing has caught the interest and attention of both developers and IT organizations simultaneously. This is not without reason. These recent technologies simplify distributed transaction management, and are fueled by three major developments:
In addition, these technologies address the scalability and robustness that are required for today’s enterprise applications. The purpose of this document is to focus on the issues and concepts involved in distributed transaction management. By no means this is a comprehensive enough to cover all the finer details of the underlying technologies. At the same time, this document does not attempt to compare technologies, but attempts to map various concepts to each of the technologies. Only the nuts and the bolts are discussed, not how nuts and bolts are made, and not how machines are built with them. (Written sometime in 1999) |
|