| Oracle® Streams Advanced Queuing User's Guide and Reference Release 10.1 Part Number B10785-01 |
|
|
View PDF |
| Oracle® Streams Advanced Queuing User's Guide and Reference Release 10.1 Part Number B10785-01 |
|
|
View PDF |
This chapter discusses Oracle Streams Advanced Queuing (AQ) and the requirements for complex information handling in an integrated environment.
This chapter contains the following topics:
|
Note: For helpful examples on using Oracle Streams AQ, search for the "Oracle By Example Series" at the OTN Web site: |
When Web-based business applications communicate with each other, producer applications enqueue messages and consumer applications dequeue messages. At the most basic level of queuing, one producer enqueues one or more messages into one queue. Each message is dequeued and processed once by one of the consumers. A message stays in the queue until a consumer dequeues it or the message expires. A producer may stipulate a delay before the message is available to be consumed, and a time after which the message expires. Likewise, a consumer may wait when trying to dequeue a message if no message is available. An agent program or application may act as both a producer and a consumer.
Producers can enqueue messages in any sequence. Messages are not necessarily dequeued in the order in which they are enqueued. Messages can be enqueued without being dequeued.
At a slightly higher level of complexity, many producers enqueue messages into a queue, all of which are processed by one consumer. Or many producers enqueue messages, each message being processed by a different consumer depending on type and correlation identifier.
Oracle Streams AQ provides database-integrated message queuing functionality. It is built on top of Oracle Streams and leverages the functions of Oracle Database so that messages can be stored persistently, propagated between queues on different computers and databases, and transmitted using Oracle Net Services and HTTP(S).
Because Oracle Streams AQ is implemented in database tables, all operational benefits of high availability, scalability, and reliability are also applicable to queue data. Standard database features such as recovery, restart, and security are supported by Oracle Streams AQ. Also queue tables can be imported and exported. You can use database development and management tools such as Oracle Enterprise Manager to monitor queues.
Requests for service must be decoupled from supply of services to increase efficiency and provide the infrastructure for complex scheduling. Oracle Streams AQ exhibits high performance characteristics as measured by the following metrics:
Number of messages enqueued/dequeued each second
Time to evaluate a complex query on a message warehouse
Time to recover/restart the messaging process after a failure
Queuing systems must be scalable. Oracle Streams AQ exhibits high performance when the number of programs using the application increases, when the number of messages increases, and when the size of the message warehouse increases.
Messages that constitute requests for service must be stored persistently and processed exactly once for deferred execution to work correctly in the presence of network, computer, and application failures. Oracle Streams AQ is able to meet requirements in the following situations:
Applications that do not have the resources to handle multiple unprocessed messages arriving simultaneously from external clients or from programs internal to the application.
Communication links between databases that are not available all the time or are reserved for other purposes. If the system falls short in its capacity to deal with these messages immediately, then the application must be able to store the messages until they can be processed.
External clients or internal programs that are not ready to receive messages that have been processed.
Queuing systems need message persistence so they can deal with priorities: messages arriving later can be of higher priority than messages arriving earlier; messages arriving earlier may wait for messages arriving later before actions are executed; the same message may be accessed by different processes; and so on. Priorities also change. Messages in a specific queue can become more important, and so must be processed with less delay or interference from messages in other queues. Similarly, messages sent to some destinations can have a higher priority than others.
Message persistence is needed to preserve message metadata, which can be as important as the payload data. For example, the time that a message is received or dispatched can be crucial for business and legal reasons. With the persistence features of Oracle Streams AQ, you can analyze periods of greatest demand or evaluate the lag between receiving and completing an order.
Oracle Streams AQ provides the message management and communication needed for application integration. In an integrated environment, messages travel between the Oracle Database server and the applications and users, as shown in Figure 1-1.
Using Oracle Net Services, messages are exchanged between a client and the Oracle Database server or between two Oracle Database servers. Oracle Net Services also propagates messages from one Oracle Database queue to another. Or, as shown in Figure 1-1, you can perform Oracle Streams AQ operations over the Internet using HTTP(S). In this case, the client, a user or Internet application, produces structured XML messages. During propagation over the Internet, Oracle Database servers communicate using structured XML also.
|
See Also: Chapter 17, " Internet Access to Oracle Streams AQ" for more information on Internet integration with Oracle Streams AQ |
Application integration also involves the integration of heterogeneous messaging systems. Oracle Streams AQ seamlessly integrates with existing non-Oracle Database messaging systems like IBM Websphere MQ through Messaging Gateway, thus allowing existing Websphere MQ-based applications to be integrated into an Oracle Streams AQ environment.
|
See Also: Chapter 18, " Introducing Oracle Messaging Gateway" for more information on Oracle Streams AQ integration with non-Oracle Database messaging systems |
Figure 1-1 Integrated Application Environment Using Oracle Streams AQ
Client/Server applications usually run in a synchronous manner. Figure 1-2 demonstrates the asynchronous alternative using Oracle Streams AQ. In this example Application B (a server) provides service to Application A (a client) using a request/response queue.
Figure 1-2 Client/Server Communication Using Oracle Streams AQ
Application A enqueues a request into the request queue. Application B dequeues and processes the request. Application B enqueues the result in the response queue, and Application A dequeues it.
The client need not wait to establish a connection with the server, and the server dequeues the message at its own pace. When the server is finished processing the message, there is no need for the client to be waiting to receive the result. A process of double-deferral frees both client and server.
|
Note: The various enqueue and dequeue operations are part of different transactions. |
A message can only be enqueued into one queue at a time. If a producer had to insert the same message into several queues in order to reach different consumers, then this would require management of a very large number of queues. To allow multiple consumers to dequeue the same message, Oracle Streams AQ provides for queue subscribers and message recipients.
To allow for subscriber and recipient lists, the queue must reside in a queue table that is created with the multiple consumer option. Each message remains in the queue until it is consumed by all its intended consumers.
Multiple consumers can be associated with a queue as subscribers. This causes all messages enqueued in the queue to be made available to be consumed by each of the queue subscribers. The subscribers to the queue can be changed dynamically without any change to the messages or message producers. Subscribers to the queue are added and removed by using the Oracle Streams AQ administrative package.
It cannot be known which subscriber will dequeue which message first, second, and so on, because there is no priority among subscribers. More formally, the order of dequeuing by subscribers is undetermined.
Every message will eventually be dequeued by every subscriber.
In Figure 1-3, Application B and Application C each need messages produced by Application A, so a multiconsumer queue is specially configured with Application B and Application C as queue subscribers. Each receives every message placed in the queue.
|
Note: Queue subscribers can be applications or other queues. |
A message producer can submit a list of recipients at the time a message is enqueued. This allows for a unique set of recipients for each message in the queue. The recipient list associated with the message overrides the subscriber list associated with the queue, if there is one. The recipients need not be in the subscriber list. However, recipients can be selected from among the subscribers.
Subscribing to a queue is like subscribing to a magazine: each subscriber is able to dequeue all the messages placed into a specific queue, just as each magazine subscriber has access to all its articles. Being a recipient, on the other hand, is like getting a letter: each recipient is a designated target of a particular message.
Figure 1-4 shows how Oracle Streams AQ can accommodate both kinds of consumers. Application A enqueues messages. Application B and Application C are subscribers. But messages can also be explicitly directed toward recipients like Application D, which may or may not be subscribers to the queue. The list of such recipients for a given message is specified in the enqueue call for that message. It overrides the list of subscribers for that queue.
Figure 1-4 Explicit and Implicit Recipients of Messages
|
Note: Multiple producers can simultaneously enqueue messages aimed at different targeted recipients. |
Figure 1-5 illustrates the use of Oracle Streams AQ for implementing a workflow, also known as a chained application transaction. Application A begins a workflow by enqueuing Message 1. Application B dequeues it, performs whatever activity is required, and enqueues Message 2. Application C dequeues Message 2 and generates Message 3. Application D, the final step in the workflow, dequeues it.
Figure 1-5 Implementing a Workflow using Oracle Streams AQ
|
Note: The contents of the messages 1, 2 and 3 can be the same or different. Even when they are different, messages can contain parts of the of the contents of previous messages. |
The queues are used to buffer the flow of information between different processing stages of the business process. By specifying delay interval and expiration time for a message, a window of execution can be provided for each of the applications.
From a workflow perspective, knowledge of the volume and timing of message flows is a business asset quite apart from the value of the payload data. Oracle Streams AQ helps you gain this knowledge by supporting the optional retention of messages for analysis of historical patterns and prediction of future trends.
A point-to-point message is aimed at a specific target. Senders and receivers decide on a common queue in which to exchange messages. Each message is consumed by only one receiver. Figure 1-6 shows that each application has its own message queue, known as a single-consumer queue.
A publish/subscribe message can be consumed by multiple receivers, as shown in Figure 1-7. Publish/subscribe messaging has a wide dissemination mode called broadcast and a more narrowly aimed mode called multicast.
Broadcasting is like a radio station not knowing exactly who the audience is for a given program. The dequeuers are subscribers to multiconsumer queues. In contrast, multicast is like a magazine publisher who knows who the subscribers are. Multicast is also referred to as point-to-multipoint, because a single publisher sends messages to multiple receivers, called recipients, who may or may not be subscribers to the queues that serve as exchange mechanisms.
Publish/subscribe describes a situation in which a publisher application enqueues messages to a queue anonymously (no recipients specified). The messages are then delivered to subscriber applications based on rules specified by each application. The rules can be defined on message properties, message data content, or both.
Figure 1-8 illustrates the use of Oracle Streams AQ for implementing a publish/subscribe relationship between publisher Application A and subscriber Applications B, C, and D. Application B subscribes with rule "priority=1", application C subscribes with rule "priority > 1", and application D subscribes with rule "priority = 3".
Figure 1-8 Implementing Publish/Subscribe using Oracle Streams AQ
Application A enqueues 3 messages with differing priorities. Application B receives a single message (priority 1), application C receives two messages (priority 2, 3) and application D receives a single message (priority 3). Message recipients are computed dynamically based on message properties and content.
A combination of Oracle Streams AQ features allows publish/subscribe messaging between applications. These features, described later in this guide, include rule-based subscribers, message propagation, the listen feature, and notification capabilities.
Enqueued messages are said to be propagated when they are reproduced on another queue.
This section contains these topics:
In Oracle Streams AQ, message recipients can be either consumers or other queues. If the message recipient is a queue, then message recipients include all subscribers to the queue (one or more of which can be other queues). Thus it is possible to fan out messages to a large number of recipients without requiring them all to dequeue messages from a single queue.
For example, imagine a queue named Source with subscriber queues dispatch1@dest1 and dispatch2@dest2. Queue dispatch1@dest1 has subscriber queues outerreach1@dest3 and outerreach2@dest4, while queue dispatch2@dest2 has subscriber queues outerreach3@dest21 and outerreach4@dest4. Messages enqueued in Source are propagated to all the subscribers of four different queues.
Messages from different queues can be combined into a single queue. This is also known as funneling. For example, if queue composite@endpoint is a subscriber to both funnel1@source1 and funnel2@source2, then subscribers to composite@endpoint get all messages enqueued in those queues as well as messages enqueued directly to composite@endpoint.
Figure 1-9 illustrates applications on different databases communicating using Oracle Streams AQ. Each application has an inbox for handling incoming messages and an outbox for handling outgoing messages. Whenever an application enqueues a message, it goes into its outbox regardless of the message destination. Messages sent locally (on the same node) and messages sent remotely (on a different node) all go in the outbox. Similarly, an application dequeues messages from its inbox no matter where the message originates. Oracle Streams AQ facilitates such interchanges, treating all messages on the same basis.
Applications often use data in different formats. A transformation defines a mapping from one Oracle data type to another. The transformation is represented by a SQL function that takes the source data type as input and returns an object of the target data type.
You can arrange transformations to occur when a message is enqueued, when it is dequeued, or when it is propagated to a remote subscriber.
As Figure 1-10 shows, queuing, routing, and transformation are essential building blocks to an integrated application architecture. The figure shows how data from the Out queue of a CRM application is routed and transformed in the integration hub and then propagated to the In queue of the Web application. The transformation engine maps the message from the format of the Out queue to the format of the In queue.
You can access Oracle Streams AQ over the Internet by using Simple Object Access Protocol (SOAP). Internet Data Access Presentation (IDAP) is the SOAP specification for Oracle Streams AQ operations. IDAP defines the XML message structure for the body of the SOAP request. An IDAP-structured message is transmitted over the Internet using HTTP(S).
This section contains these topics:
Oracle Streams AQ supports messages of three types: RAW, Oracle object, and Java Message Service (JMS). All these message types can be accessed using SOAP and Web Services. If the queue holds messages in RAW, Oracle object, or JMS format, then XML payloads are transformed to the appropriate internal format during enqueue and stored in the queue. During dequeue, when messages are obtained from queues containing messages in any of the preceding formats, they are converted to XML before being sent to the client.
The message payload type depends on the queue type on which the operation is being performed:
The contents of RAW queues are raw bytes. You must supply the hex representation of the message payload in the XML message. For example, <raw>023f4523</raw>.
For Oracle object type queues that are not JMS queues (that is, they are not type AQ$_JMS_*), the type of the payload depends on the type specified while creating the queue table that holds the queue. The XML specified here must map to the SQL type of the payload for the queue table.
Example 1-1 A Queue Type and its XML Equivalent
Assume the queue is defined to be of type EMP_TYP, which has the following structure:
CREATE OR REPLACE TYPE emp_typ AS object (
empno NUMBER(4),
ename VARCHAR2(10),
job VARCHAR2(9),
mgr NUMBER(4),
hiredate DATE,
sal NUMBER(7,2),
comm NUMBER(7,2)
deptno NUMBER(2));
The corresponding XML representation is:
<EMP_TYP>
<EMPNO>1111</EMPNO>
<ENAME>Mary</ENAME>
<MGR>5000</MGR>
<HIREDATE>1996-01-01 0:0:0</HIREDATE>
<SAL>10000</SAL>
<COMM>100.12</COMM>
<DEPTNO>60</DEPTNO>
</EMP_TYP>
For queues with JMS types (that is, those with payloads of type AQ$_JMS_*), there are four XML elements, depending on the JMS type. IDAP supports queues or topics with the following JMS types:
TextMessage
MapMessage
BytesMessage
ObjectMessage
JMS queues with payload type StreamMessage are not supported through IDAP.
Figure 1-11 shows the architecture for performing Oracle Streams AQ operations over HTTP. The major components are:
Oracle Streams AQ client program
Web server/Servlet Runner hosting the Oracle Streams AQ servlet
Oracle Database server
The Oracle Streams AQ client program sends XML messages (conforming to IDAP) to the Oracle Streams AQ servlet, which understands the XML message and performs Oracle Streams AQ operations. Any HTTP client, for example Web browsers, can be used. The Web server/Servlet Runner hosting the Oracle Streams AQ servlet interprets the incoming XML messages. Examples include Apache/Jserv or Tomcat. The Oracle Streams AQ servlet connects to the Oracle Database server and performs operations on the users' queues.
Figure 1-11 Architecture for Performing Oracle Streams AQ Operations Using HTTP
Internet Data Access Presentation (IDAP) uses the Content-Type of text/xml to specify the body of the SOAP request. XML provides the presentation for IDAP request and response messages as follows:
All request and response tags are scoped in the SOAP namespace.
Oracle Streams AQ operations are scoped in the IDAP namespace.
The sender includes namespaces in IDAP elements and attributes in the SOAP body.
The receiver processes IDAP messages that have correct namespaces. For requests with incorrect namespaces, the receiver returns an invalid request error.
The SOAP namespace has the value http://schemas.xmlsoap.org/soap/envelope/
The IDAP namespace has the value http://ns.oracle.com/AQ/schemas/access
You can access Oracle Streams AQ functionality through the following interfaces:
PL/SQL using DBMS_AQ, DBMS_AQADM, and DBMS_AQELM
Visual Basic using Oracle Objects for OLE
Java Message Service (JMS) using the oracle.jms Java package
Internet access using HTTP(S)
|
Note: Theoracle.AQ Java package has been deprecated in Oracle Streams AQ release 10.1. Oracle recommends that you migrate existing Java AQ applications to Oracle JMS and use Oracle JMS to design your future Java AQ applications. |
This section contains these topics:
The following features apply to enqueuing messages:
When enqueuing messages into a queue, you can operate on an array of messages simultaneously, instead of one message at a time. This can improve the performance of enqueue operations. When enqueuing an array of messages into a queue, each message shares the same enqueue options, but each message can have different message properties. You can perform array enqueue operations using PL/SQL or OCI.
You can assign an identifier to each message, thus providing a means to retrieve specific messages at a later time.
A single message can be designed to be consumed by multiple consumers. A queue administrator can specify the list of subscribers who can retrieve messages from a queue. Different queues can have different subscribers, and a consumer program can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby overriding the subscriber list.
You can design a single message for consumption by multiple consumers in a number of different ways. The consumers who are allowed to retrieve the message are specified as explicit recipients of the message by the user or application that enqueues the message. Every explicit recipient is an agent identified by name, address, and protocol.
A queue administrator can also specify a default list of recipients who can retrieve all the messages from a specific queue. These implicit recipients become subscribers to the queue by being specified in the default list. If a message is enqueued without specifying any explicit recipients, then the message is delivered to all the designated subscribers.
A rule-based subscriber is one that has a rule associated with it in the default recipient list. A rule-based subscriber is sent a message with no explicit recipients specified only if the associated rule evaluated to TRUE for the message. Different queues can have different subscribers, and the same recipient can be a subscriber to more than one queue. Further, specific messages in a queue can be directed toward specific recipients who may or may not be subscribers to the queue, thereby overriding the subscriber list.
A recipient can be specified only by its name, in which case the recipient must dequeue the message from the queue in which the message was enqueued. It can be specified by its name and an address with a protocol value of 0. The address should be the name of another queue in the same database or another installation of Oracle Database (identified by the database link), in which case the message is propagated to the specified queue and can be dequeued by a consumer with the specified name. If the recipient's name is NULL, then the message is propagated to the specified queue in the address and can be dequeued by the subscribers of the queue specified in the address. If the protocol field is nonzero, then the name and address are not interpreted by the system and the message can be dequeued by a special consumer.
It is possible to specify the priority of the enqueued message. An enqueued message can also have its exact position in the queue specified. This means that users have three options to specify the order in which messages are consumed: (a) a sort order specifies which properties are used to order all message in a queue; (b) a priority can be assigned to each message; (c) a sequence deviation positions a message in relation to other messages. Further, if several consumers act on the same queue, then a consumer gets the first message that is available for immediate consumption. A message that is in the process of being consumed by another consumer is skipped.
Messages belonging to one queue can be grouped to form a set that can only be consumed by one user at a time. This requires that the queue be created in a queue table that is enabled for message grouping. All messages belonging to a group must be created in the same transaction and all messages created in one transaction belong to the same group. This feature allows users to segment complex messages into simple messages; for example, messages directed to a queue containing invoices can be constructed as a group of messages starting with the header message, followed by messages representing details, followed by the trailer message.
This feature enables applications to communicate with each other without having to be connected to the same database or the same queue. Messages can be propagated from one Oracle Streams AQ to another, irrespective of whether the queues are local or remote. Propagation is accomplished using database links and Oracle Net Services.
Applications can mark the messages they send with a custom identification. Oracle Streams AQ also automatically identifies the queue from which a message was dequeued. This allows applications to track the pathway of a propagated message or a string message within the same database.
Delay interval or expiration intervals can be specified for an enqueued message, thereby providing windows of execution. A message can be marked as available for processing only after a specified time elapses (a delay time) and must be consumed before a specified time limit expires.
A message can be delivered to multiple recipients based on message properties or message content. Users define a rule-based subscription for a given queue as the mechanism to specify interest in receiving messages of interest. Rules can be specified based on message properties and message data (for object and raw payloads). Subscriber rules are then used to evaluate recipients for message delivery.
The asynchronous notification feature allows clients to receive notification of a message of interest. The client can use it to monitor multiple subscriptions. The client need not be connected to the database to receive notifications regarding its subscriptions.
Clients can use the Oracle Call Interface (OCI) function OCISubscriptionRegister or the PL/SQL procedure DBMS_AQ.REGISTER to register interest in messages in a queue.
The following features apply to dequeuing messages:
When dequeuing messages from a queue, you can operate on an array of messages simultaneously, instead of one message at a time. This can improve the performance of dequeue operations. If you are dequeuing from a transactional queue, you can dequeue all the messages for a transaction with a single call, which makes application programming easier.
When dequeuing an array of messages from a queue, each message shares the same dequeue options, but each message can have different message properties. You can perform array enqueue and array dequeue operations using PL/SQL or OCI.
A message can be retrieved by multiple recipients without the need for multiple copies of the same message. Designated recipients can be located locally or at remote sites.
Users have several options to select a message from a queue. They can select the first message or once they have selected a message and established a position, they can retrieve the next. The selection is influenced by the ordering or can be limited by specifying a correlation identifier. Users can also retrieve a specific message using the message identifier.
A dequeue request can either browse or remove a message. If a message is browsed, then it remains available for further processing. If a message is removed, then it is not available more for dequeue requests. Depending on the queue properties, a removed message can be retained in the queue table.
A dequeue request can be applied against an empty queue. To avoid polling for the arrival of a new message, a user can specify if and for how long the request is allowed to wait for the arrival of a message.
A message must be consumed exactly once. If an attempt to dequeue a message fails and the transaction is rolled back, then the message is made available for reprocessing after some user-specified delay elapses. Reprocessing is attempted up to the user-specified limit.
Enqueue and dequeue requests are usually part of a transaction that contains the requests, thereby providing the wanted transactional action. You can, however, specify that a specific request is a transaction by itself, making the result of that request immediately visible to other transactions. This means that messages can be made visible to the external world when the enqueue or dequeue statement is applied or after the transaction is committed.
A message may not be consumed within given constraints, such as within the window of execution or within the limits of the retries. If such a condition arises, then the message is moved to a user-specified exception queue.
The listen call is a blocking call that can be used to wait for messages on multiple queues. It can be used by a gateway application to monitor a set of queues. An application can also use it to wait for messages on a list of subscriptions. If the listen returns successfully, then a dequeue must be used to retrieve the message.
The dequeue mode REMOVE_NODATA can be used to remove a message from a queue without retrieving the payload. Use this mode to delete a message with a large payload whose content is irrelevant.
The following features apply to propagating messages:
|
See Also: "Internet Integration and Internet Data Access Presentation" for information on propagation over the Internet |
Recipients can be local or remote. Because Oracle Database does not support distributed object types, remote enqueuing or dequeuing using a standard database link does not work. However, you can use Oracle Streams AQ message propagation to enqueue to a remote queue. For example, you can connect to database X and enqueue the message in a queue, DROPBOX, located in database X. You can configure Oracle Streams AQ so that all messages enqueued in DROPBOX are automatically propagated to another queue in database Y, regardless of whether database Y is local or remote. Oracle Streams AQ automatically checks if the type of the remote queue in database Y is structurally equivalent to the type of the local queue in database X and propagates the message.
Recipients of propagated messages can be applications or queues. If the recipient is a queue, then the actual recipients are determined by the subscription list associated with the recipient queue. If the queues are remote, then messages are propagated using the specified database link. AQ-to-AQ message propagation is directly supported; propagation between Oracle Streams AQ and other message systems, such as WebSphere MQ and TIB/Rendezvous, is supported through Messaging Gateway.
Propagation handles payloads with LOB attributes.
|
Note: Payloads containing LOBs require users to grant explicitSelect, Insert and Update privileges on the queue table for doing enqueues and dequeues. |
Messages can be scheduled to propagate from a queue to local or remote destinations. Administrators can specify the start time, the propagation window, and a function to determine the next propagation window (for periodic schedules).
Detailed run-time information about propagation is gathered and stored in the DBA_QUEUE_SCHEDULES view for each propagation schedule. This information can be used by queue designers and administrators to fix problems or tune performance. For example, available statistics about the total and average number of message/bytes propagated can be used to tune schedules. Similarly, errors reported by the view can be used to diagnose and fix problems. The view also describes additional information such as the session ID of the session handling the propagation, and the process name of the job queue process handling the propagation.
Oracle Streams AQ allows messages to be enqueued in queues that can then be propagated to different messaging systems by third-party propagators. If the protocol number for a recipient is in the range 128 - 255, then the address of the recipient is not interpreted by Oracle Streams AQ and so the message is not propagated by the Oracle Streams AQ system. Instead, a third-party propagator can then dequeue the message by specifying a reserved consumer name in the dequeue operation. The reserved consumer names are of the form AQ$_P#, where # is the protocol number in the range 128–255. For example, the consumer name AQ$_P128 can be used to dequeue messages for recipients with protocol number 128. The list of recipients for a message with the specific protocol number is returned in the recipient_list message property on dequeue.
Another way for Oracle Streams AQ to propagate messages to and from third-party messaging systems is through Messaging Gateway, an Enterprise Edition feature. Messaging Gateway dequeues messages from an Oracle Streams AQ queue and guarantees delivery to a third-party messaging system such as Websphere MQ (MQSeries). Messaging Gateway can also dequeue messages from third-party messaging systems and enqueue them to an Oracle Streams AQ queue.
This section contains these topics:
Before release 10.1, the Oracle Streams AQ time manager process was called queue monitor (QMNn), a background process controlled by setting the dynamic init.ora parameter AQ_TM_PROCESSES. Beginning with release 10.1, time management and many other background processes are automatically controlled by a coordinator-slave architecture called Queue Monitor Coordinator (QMNC). QMNC dynamically spawns slaves named qXXX depending on the system load. The slaves provide mechanisms for:
Message delay
Message expiration
Retry delay
Garbage collection for the queue table
Because the number of processes is determined automatically and tuned constantly, you are saved the trouble of setting it with AQ_TM_PROCESSES.
Although it is no longer necessary to set init.ora parameter AQ_TM_PROCESSES, it is still supported. If you do set it (up to a maximum of 10), then QMNC still autotunes the number of processes. But you are guaranteed at least the set number of processes for persistent queues. Processes for buffered queues and other Oracle Streams tasks, however, are not affected by this parameter.
|
Note: Oracle strongly recommends that you do NOT setAQ_TM_PROCESSES = 0. If you are using Oracle Streams, setting this parameter to zero (which Oracle Database respects no matter what) can cause serious problems. |
Oracle Internet Directory is a native LDAPv3 directory service built on Oracle Database that centralizes a wide variety of information, including e-mail addresses, telephone numbers, passwords, security certificates, and configuration data for many types of networked devices. You can look up enterprise-wide queuing information—queues, subscriptions, and events—from one location, the Oracle Internet Directory. Refer to the Oracle Internet Directory Administrator's Guide for more information.
You can use Oracle Enterprise Manager to do the following:
Create and manage queues, queue tables, propagation schedules, and transformations
Monitor your Oracle Streams AQ environment using its topology at the database and queue levels, and by viewing queue errors and queue and session statistics
Messages are placed in normal rows in a database table, and so can be queried using standard SQL. This means that you can use SQL to access the message properties, the message history, and the payload. With SQL access you can also do auditing and tracking. All available SQL technology, such as indexes, can be used to optimize access to messages.
|
Note: Oracle Streams AQ does not support data manipulation language (DML) operations on a queue table or an associated index-organized table (IOT), if any. The only supported means of modifying queue tables is through the supplied APIs. Queue tables and IOTs can become inconsistent and therefore effectively ruined, if DML operations are performed on them. |
Basic statistics about queues are available using the GV$AQ view.
You can use object types to structure and manage message payloads. Relational database systems in general have a richer typing system than messaging systems. Because Oracle Database is an object-relational database system, it supports traditional relational and user-defined types. Many powerful features are enabled as a result of having strongly typed content, such as content whose format is defined by an external type system. These include:
Content-based routing
Oracle Streams AQ can examine the content and automatically route the message to another queue based on the content.
Content-based subscription
A publish and subscribe system is built on top of a messaging system so that you can create subscriptions based on content.
Querying
The ability to run queries on the content of the message enables message warehousing.
You can create queues that use the new opaque type, XMLType. These queues can be used to transmit and store messages that are XML documents. Using XMLType, you can do the following:
Store any type of message in a queue
Store documents internally as CLOB objects
Store more than one type of payload in a queue
Query XMLType columns using the operators ExistsNode() and SchemaMatch()
Specify the operators in subscriber rules or dequeue conditions
The systems administrator specifies the retention duration to retain messages after consumption. Oracle Streams AQ stores information about the history of each message, preserving the queue and message properties of delay, expiration, and retention for messages destined for local or remote receivers. The information contains the enqueue and dequeue times and the identification of the transaction that executed each request. This allows users to keep a history of relevant messages. The history can be used for tracking, data warehouse, and data mining operations, as well as specific auditing functions.
If messages are retained, then they can be related to each other. For example, if a message m2 is produced as a result of the consumption of message m1, then m1 is related to m2. This allows users to track sequences of related messages. These sequences represent event journals, which are often constructed by applications. Oracle Streams AQ is designed to let applications create event journals automatically.
When an online order is placed, multiple messages are generated by the various applications involved in processing the order. Oracle Streams AQ offers features to track interrelated messages independent of the applications that generated them. You can determine who enqueued and dequeued messages, who the users are, and who did what operations.
With Oracle Streams AQ tracking features, you can use SQL SELECT and JOIN statements to get order information from AQ$queuetablename and the views ENQ_TRAN_ID, DEQ_TRAN_ID, USER_DATA (the payload), CORR_ID, and MSG_ID. These views contain the following data used for tracking:
Transaction IDs from ENQ_TRAN_ID and DEQ_TRAN_ID, captured during enqueuing and dequeuing.
Correlation IDs from CORR_ID, part of the message properties
USER_DATA message content that can be used for tracking
The owner of an 8.1-compatible queue can grant or revoke queue-level privileges on the queue. Database administrators can grant or revoke new Oracle Streams AQ system-level privileges to any database user. Database administrators can also make any database user an Oracle Streams AQ administrator.
Oracle Streams AQ can deliver nonpersistent messages asynchronously to subscribers. These messages can be event-driven and do not persist beyond the failure of the system (or instance). Oracle Streams AQ supports persistent and nonpersistent messages with a common API.
An application can specify the instance affinity for a queue table. When Oracle Streams AQ is used with Real Application Clusters and multiple instances, this information is used to partition the queue tables between instances for queue-monitor scheduling. The queue table is monitored by the queue monitors of the instance specified by the user. If an instance affinity is not specified, then the queue tables are arbitrarily partitioned among the available instances. There can be pinging between the application accessing the queue table and the queue monitor monitoring it. Specifying the instance affinity does not prevent the application from accessing the queue table and its queues from other instances.
This feature prevents pinging between queue monitors and Oracle Streams AQ propagation jobs running in different instances. If compatibility is set to Oracle8i release 8.1.5 or higher, then an instance affinity (primary and secondary) can be specified for a queue table. When Oracle Streams AQ is used with Real Application Clusters and multiple instances, this information is used to partition the queue tables between instances for queue-monitor scheduling as well as for propagation. At any time, the queue table is affiliated to one instance. In the absence of an explicitly specified affinity, any available instance is made the owner of the queue table. If the owner of the queue table is terminated, then the secondary instance or some available instance takes over the ownership for the queue table.
Oracle Streams AQ maintains the entire history of information about a message along with the message itself. You can look up history information by using the AQ$QueueTableName view. This information serves as the proof of sending and receiving of messages and can be used for nonrepudiation of the sender and nonrepudiation of the receiver.
|
See Also: Chapter 9, " Oracle Streams AQ Administrative Interface: Views" for more information about theAQ$QueueTableName view |
The following information is kept at enqueue for nonrepudiation of the enqueuer:
Oracle Streams AQ agent doing the enqueue
Database user doing the enqueue
Enqueue time
Transaction ID of the transaction doing the enqueue
The following information is kept at dequeue for nonrepudiation of the dequeuer:
Oracle Streams AQ agent doing dequeue
Database user doing dequeue
Dequeue time
Transaction ID of the transaction doing dequeue
After propagation, the Original_Msgid field in the destination queue of propagation corresponds to the message ID of the source message. This field can be used to correlate the propagated messages. This is useful for nonrepudiation of the dequeuer of propagated messages.
Stronger nonrepudiation can be achieved by enqueuing the digital signature of the sender at the time of enqueue with the message and by storing the digital signature of the dequeuer at the time of dequeue.
The following demos can be found in the $ORACLE_HOME/rdbms/demo directory. Refer to aqxmlreadme.txt and aqjmsreadme.txt in the demo directory for more information.
Table 1-1 Oracle Streams AQ Demos
| Demo and Locations | Topic |
|---|---|
aqjmsdemo01.java |
Enqueue text messages and dequeue based on message properties |
aqjmsdemo02.java |
Message Listener demo |
aqjmsdemo03.java |
Message Listener demo |
aqjmsdemo04.java |
Oracle Type Payload - Dequeue on payload content |
aqjmsdemo05.java |
Example of the QueueBrowser |
aqjmsdemo06.java |
Schedule propagation between queues in the database |
aqjmsdmo.sql |
Set up Oracle Streams AQ JMS demos |
aqjmsREADME.txt |
Describes the Oracle Streams AQ Java API and JMS demos |
aqorademo01.java |
Enqueue and dequeue RAW messages |
aqorademo02.java |
Enqueue and dequeue object type messages using the Custom Datum interface |
aqoradmo.sql |
Setup file for Oracle Streams AQ Java API demos |
aqxml01.xml |
AQXmlSend—Enqueue to Oracle object type single- consumer queue with piggyback commit |
aqxml02.xml |
AQXmlReceive—Dequeue from Oracle object type single-consumer queue with piggyback commit |
aqxml03.xml |
AQXmlPublish—Enqueue to Oracle object type (with LOB) multiconsumer queue |
aqxml04.xml |
AQXmlReceive—Dequeue from Oracle object type multi- consumer queue |
aqxml05.xml |
AQXmlCommit—Commit previous operation |
aqxml06.xml |
AQXmlSend—Enqueue to JMS Text single-consumer queue with piggyback commit |
aqxml07.xml |
AQXmlReceive—Dequeue from JMS Text single-consumer queue with piggyback commit |
aqxml08.xml |
AQXmlPublish—Enqueue JMS MAP message with recipient into multiconsumer queue |
aqxml09.xml |
AQXmlReceive—Dequeue JMS MAP message from multiconsumer queue |
aqxml10.xml |
AQXmlRollback—Roll back previous operation |
aqxmlhtp.sql |
HTTP Propagation |
aqxmlREADME.txt |
Describes the Internet access demos |
AQDemoServlet.java |
Servlet to post Oracle Streams AQ XML files (for Jserv) |
AQPropServlet.java |
Servlet for Oracle Streams AQ HTTP propagation |
aqdemo00.sql |
Create users, message types, tables, and so on |
aqdemo01.sql |
Set up queue_tables, queues, and subscribers |
aqdemo02.sql |
Enqueue messages |
aqdemo03.sql |
Install dequeue procedures |
aqdemo04.sql |
Perform blocking dequeue |
aqdemo05.sql |
Perform listen for multiple agents |
aqdemo06.sql |
Clean up users, queue_tables, queues, subscribers (cleanup script) |
aqdemo07.sql |
Enqueue /dequeue to queue of type ADT with XMLType |
aqdemo08.sql |
Notification |
aqdemo09.sql |
Set up queues and subscribers (for OCI array demos also) |
aqdemo10.sql |
Array enqueue 10 messages |
aqdemo11.sql |
Array dequeue 10 messages |
aqdemo12.sql |
Clean up queues and subscribers (for OCI array demos also) |
ociaqdemo00.c |
Enqueue messages |
ociaqdemo01.c |
Perform blocking dequeue |
ociaqdemo02.c |
Perform listen for multiple agents |
ociaqarrayenq.c |
Array enqueue 10 messages |
ociaqarraydeq.c |
Array dequeue 10 messages |
|
 Copyright © 1996, 2003 Oracle Corporation All Rights Reserved. |
|