21 Oracle Messaging Gateway Message Conversion

Overview of the Non-JMS Message Conversion Process

When a message is propagated by MGW, the message is converted from the native format of the source queue to the native format of the destination queue.

A native message usually contains a message header and a message body. The header contains the fixed header fields that all messages in that messaging system have, such as message properties in Oracle Streams AQ and the fixed header in WebSphere MQ. The body contains message contents, such as the Oracle Streams AQ payload, the WebSphere MQ message body, or the entire TIB/Rendezvous message. MGW converts both message header and message body components.

Figure 21-1shows how non-JMS messages are converted in two stages. A message is first converted from the native format of the source queue to the MGW internal message format, and then it is converted from the internal message format to the native format of the destination queue.

Figure 21-1 Non-JMS Message Conversion

Description of the illustration adqmg003.gif

The MGW agent uses an internal message format consisting of a header that is similar to the Oracle Streams AQ message properties and a body that is a representation of an MGW canonical type.

Oracle Messaging Gateway Canonical Types

MGW defines canonical types to support message conversion between Oracle Streams AQ and non-Oracle messaging systems. A canonical type is a message type representation in the form of a PL/SQL Oracle type in Oracle Database. The canonical types are RAW, SYS.MGW_BASIC_MSG_T, and SYS.MGW_TIBRV_MSG_T.

WebSphere MQ propagation supports the canonical types MGW_BASIC_MSG_T and RAW. TIB/Rendezvous propagation supports the canonical types MGW_TIBRV_MSG_T and RAW.

Message Header Conversion

MGW provides default mappings between Oracle Streams AQ message properties and non-Oracle message header fields that have a counterpart in Oracle Streams AQ message properties with the same semantics. Where MGW does not provide a mapping, the message header fields are set to a default value, usually the default value defined by the messaging system.

Handling Arbitrary Payload Types Using Message Transformations

When converting to or from Oracle Streams AQ messages, the MGW agent uses only its canonical types. Arbitrary payload types are supported, however, with the assistance of user-defined Oracle Streams AQ message transformations to convert between an Oracle Streams AQ queue payload and an MGW canonical type.

For MGW to propagate messages from an Oracle Streams AQ queue with an arbitrary ADT payload (outbound propagation), you must provide a mapping to an MGW canonical ADT. The transformation is invoked when the MGW agent dequeues messages from the Oracle Streams AQ queue. Similarly, for MGW to propagate messages to an Oracle Streams AQ queue with an arbitrary ADT payload (inbound propagation), you must provide a mapping from an MGW canonical ADT. The transformation is invoked when the MGW agent enqueues messages to the Oracle Streams AQ queue.

Figure 21-2 Oracle Streams AQ Message Conversion

Description of the illustration adqmg004.gif

The transformation is always executed in the context of the MGW agent, which means that the MGW agent user (the user specified using DBMS_MGWADM.DB_CONNECT_INFO) must have EXECUTE privileges on the transformation function and the Oracle Streams AQ payload type. This can be accomplished by granting the EXECUTE privilege to PUBLIC or by granting the EXECUTE privilege directly to the MGW agent user.

To configure an MGW subscriber with a transformation:

1. Create the transformation function.

2. Grant EXECUTE to the MGW agent user or to PUBLIC on the function and the object types it references.

3. Call DBMS_TRANSFORM.CREATE_TRANSFORMATION to register the transformation.

4. Call DBMS_MGWADM.ADD_SUBSCRIBER to create an MGW subscriber using the transformation, or DBMS_MGWADM.ALTER_SUBSCRIBER to alter an existing subscriber.

The value passed in the transformation parameter for these APIs must be the registered transformation name and not the function name. For example, trans_sampleadt_to_mgw_basic is a stored procedure representing a transformation function with the signature shown in Example 21-1.

FUNCTION trans_sampleadt_to_mgw_basic(in_msg IN mgwuser.sampleADT)
RETURN SYS.MGW_BASIC_MSG_T;

begin
  DBMS_TRANSFORM.CREATE_TRANSFORMATION(
        schema         => 'mgwuser',
        name           => 'sample_adt_to_mgw_basic',
        from_schema    => 'mgwuser',
        from_type      => 'sampleadt',
        to_schema      => 'sys',
        to_type        => 'MGW_BASIC_MSG_T',
        transformation => 'mgwuser.trans_sampleadt_to_mgw_basic(user_data)');
end;

Once created, this transformation can be registered with MGW when creating a subscriber. Example 21-3 creates subscriber sub_aq2mq, for whom messages are propagated from Oracle Streams AQ queue mgwuser.srcq to non-Oracle messaging system queue destq@mqlink using transformation mgwuser.sample_adt_to_mgw_basic.

begin
  DBMS_MGWADM.ADD_SUBSCRIBER(
    subscriber_id    => 'sub_aq2mq',
    propagation_type => dbms_mgwadm.outbound_propagation,
    queue_name       => 'mgwuser.srcq',
    destination      =>'destq@mqlink',
    transformation   => 'mgwuser.sample_adt_to_mgw_basic'),
    exception_queue  => 'mgwuser.excq');
end;

An error that occurs while attempting a user-defined transformation is usually considered a message conversion exception, and the message is moved to the exception queue if it exists.

Handling Logical Change Records

MGW provides facilities to propagate Logical Change Records (LCRs). Routines are provided to help in creating transformations to handle the propagation of both row LCRs and DDL LCRs stored in queues with payload type SYS.ANYDATA. An LCR is propagated as an XML string stored in the appropriate message type.

Because Oracle Streams uses SYS.ANYDATA queues to store LCRs, a SYS.ANYDATA queue is the source for outbound propagation. The transformation must first convert the SYS.ANYDATA object containing an LCR into an XMLType object using the MGW routine DBMS_MGWMSG.LCR_TO_XML. If the SYS.ANYDATA object does not contain an LCR, then this routine raises an error. The XML document string of the LCR is then extracted from the XMLType and placed in the appropriate MGW canonical type (MGW_BASIC_MSG_T or MGW_TIBRV_MSG_T).

Example 21-4 illustrates a simplified transformation used for LCR outbound propagation. The transformation converts a SYS.ANYDATA payload containing an LCR to a SYS.MGW_TIBRV_MSG_T object. The string representing the LCR as an XML document is put in a field named 'ORACLE_LCR'.

create or replace function any2tibrv(adata in sys.anydata)
return SYS.MGW_TIBRV_MSG_T is
    v_xml   XMLType;
    v_text  varchar2(2000);
    v_tibrv sys.mgw_tibrv_msg_t;
begin
    v_xml   := dbms_mgwmsg.lcr_to_xml(adata);
    -- assume the lcr is smaller than 2000 characters long.
    v_text  := v_xml.getStringVal();
    v_tibrv := SYS.MGW_TIBRV_MSG_T.CONSTRUCT;
    v_tibrv.add_string('ORACLE_LCR', 0, v_text);
    return v_tibrv;
end any2tibrv;

For LCR inbound propagation, an MGW canonical type (MGW_BASIC_MSG_T or MGW_TIBRV_MSG_T) is the transformation source type. A string in the format of an XML document representing an LCR must be contained in the canonical type. The transformation function must extract the string from the message, create an XMLType object from it, and convert it to a SYS.ANYDATA object containing an LCR with the MGW routine DBMS_MGWMSG.XML_TO_LCR. If the original XML document does not represent an LCR, then this routine raises an error.

Example 21-5 illustrates a simplified transformation used for LCR inbound propagation. The transformation converts a SYS.MGW_TIBRV_MSG_T object with a field containing an XML string representing an LCR to a SYS.ANYDATA object. The string representing the LCR as an XML document is taken from a field named 'ORACLE_LCR'.

create or replace function tibrv2any(tdata in sys.mgw_tibrv_msg_t)
return sys.anydata is 
    v_field   sys.mgw_tibrv_field_t;
    v_xml     XMLType;
    v_text    varchar2(2000);
    v_any     sys.anydata;
begin
    v_field := tdata.get_field_by_name('ORACLE_LCR');
    -- type checking
    v_text  := v_field.text_value;
    -- assume it is not null
    v_xml  := XMLType.createXML(v_text);
    v_any  := dbms_mgwmsg.xml_to_lcr(v_xml); 
    return v_any;
end tibrv2any;

WebSphere MQ Message Header Mappings

When the MGW canonical type used in an outbound propagation job is RAW, no WebSphere MQ header information is set from the RAW message body. Similarly, for inbound propagation no WebSphere MQ header information is preserved in the RAW message body. MGW canonical type MGW_BASIC_MSG_T, however, has a header that can be used to specify WebSphere MQ header fields for outbound propagation, and preserve WebSphere MQ header fields for inbound propagation.

This section describes the message properties supported for the WebSphere MQ messaging system when using MGW_BASIC_MSG_T as the canonical type. Table 21-1 defines the MGW {name,value} pairs used to describe the WebSphere MQ header properties. The first column refers to valid string values for the MGW_NAME_VALUE_T.NAME field in the MGW_BASIC_MSG_T header. The second column refers to the MGW_NAME_VALUE_T.TYPE value corresponding to the name. (Refer to "Notes on Table 21-1" for explanations of the numbers in parentheses.)

When a message is dequeued from the WebSphere MQ messaging system, the WebSphere MQ driver generates {name,value} pairs based on the dequeued message header and stores them in the header part of the canonical message of the MGW_BASIC_MSG_T type. When a message is enqueued to WebSphere MQ, the WebSphere MQ driver sets the message header and enqueue options from {name,value} pairs for these properties stored in the header part of the MGW_BASIC_MSG_T canonical message.

1. This use is subject to WebSphere MQ restrictions. For example, if MGW_MQ_accountingToken is set for an outgoing message, then WebSphere MQ overrides its value unless MGW_MQ_putMessageOptions is set to the WebSphere MQ constant MQPMD_SET_ALL_CONTEXT.

2. MGW_MQ_putMessageOptions is used as the putMessageOptions argument to the WebSphere MQ Base Java Queue.put() method. It is not part of the WebSphere MQ header information and is therefore not an actual message property.

   The value for the openOptions argument of the WebSphere MQ Base Java MQQueueManager.accessQueue method is specified when the WebSphere MQ queue is registered using the DBMS_MGWADM.REGISTER_FOREIGN_QUEUE call. Dependencies can exist between the two. For instance, for MGW_MQ_putMessageOptions to include MQPMD_SET_ALL_CONTEXT, the MQ_openMessageOptions queue option must include MQOO_SET_CONTEXT.

   The MGW agent adds the value MQPMO_SYNCPOINT to any value that you can specify.

MGW sets default values for two WebSphere MQ message header fields: messageType defaults to MQMT_DATAGRAM and putMessageOptions defaults to MQPMO_SYNCPOINT.

MGW provides two default mappings between Oracle Streams AQ message properties and WebSphere MQ header fields.

One maps the Oracle Streams AQ message property expiration, representing the time-to-live of the message at the time the message becomes available in the queue, to the WebSphere MQ header field expiry, representing the time-to-live of the message. For outbound propagation, the value used for expiry is determined by subtracting the time the message was available in the queue from the expiration, converted to tenths of a second. Oracle Streams AQ value NEVER is mapped to MQEI_UNLIMITED. For inbound propagation, the value of expiration is simply expiry converted to seconds. WebSphere MQ value MQEI_UNLIMITED is mapped to NEVER.

The other default maps Oracle Streams AQ message property priority with the WebSphere MQ header field priority. It is described in Table 21-2.

WebSphere MQ Outbound Propagation

If no message transformation is provided for outbound propagation, then the Oracle Streams AQ source queue payload type must be either SYS.MGW_BASIC_MSG_T or RAW. If a message transformation is specified, then the target ADT of the transformation must be MGW_BASIC_MSG_T, but the source ADT can be any ADT supported by Oracle Streams AQ.

If the Oracle Streams AQ queue payload is RAW, then the resulting WebSphere MQ message has the message body set to the value of the RAW bytes and, by default, the format field set to the value "MGW_Byte".

If the Oracle Streams AQ queue payload or transformation target ADT is MGW_BASIC_MSG_T, then the message is mapped to a WebSphere MQ native message as follows:

• The WebSphere MQ fixed header fields are based on the internal Oracle Streams AQ message properties and the MGW_BASIC_MSG_T.header attribute of the canonical message, as described in "WebSphere MQ Message Header Mappings".

• If the canonical message has a TEXT body, then the WebSphere MQ format header field is set to MQFMT_STRING unless overridden by the header property MGW_MQ_format. The message body is treated as text.

• If the canonical message has a RAW body, then the WebSphere MQ format header field is set to "MGW_Byte" unless overridden by the header property MGW_MQ_format, and the message body is treated as raw bytes.

• If the canonical message has both a TEXT and RAW body, then message conversion fails.

• If the canonical message has neither a TEXT nor RAW body, then no message body is set, and the WebSphere MQ format header field is MQFMT_NONE.

• If the canonical message has a TEXT body with both small and large values set (MGW_BASIC_MSG_T.TEXT_BODY.small_value and MGW_BASIC_MSG_T.TEXT_BODY.large_value not empty), then message conversion fails.

• If the canonical message has a RAW body with both small and large values set (MGW_BASIC_MSG_T.RAW_BODY.small_value and MGW_BASIC_MSG_T.RAW_BODY.large_value not empty), then message conversion fails.

WebSphere MQ Inbound Propagation

If no message transformation is provided for inbound propagation, then the Oracle Streams AQ destination queue payload type must be either SYS.MGW_BASIC_MSG_T or RAW. If a message transformation is specified, then the source ADT of the transformation must be MGW_BASIC_MSG_T, but the destination ADT can be any ADT supported by Oracle Streams AQ.

If the Oracle Streams AQ queue payload is RAW and the incoming WebSphere MQ message has a format of MQFMT_STRING, then message conversion fails. Otherwise the message body is considered as raw bytes and enqueued directly to the destination queue. If the number of bytes is greater than 32KB, then message conversion fails. The actual limit is 32512 bytes rather than 32767 bytes.

If the Oracle Streams AQ queue payload or transformation source ADT is MGW_BASIC_MSG_T, then the WebSphere MQ message is mapped to a MGW_BASIC_MSG_T message as follows:

• Specific WebSphere MQ header fields are mapped to Oracle Streams AQ message properties as previously described.

• The MGW_BASIC_MSG_T.header attribute of the canonical message is set to {name,value} pairs based on the WebSphere MQ header fields, as described in Table 21-1. These values preserve the original content of the WebSphere MQ message header.

• If the WebSphere MQ format header field is MQFMT_STRING, then the WebSphere MQ message body is treated as text, and its value is mapped to MGW_BASIC_MSG_T.text_body. For any other format value, the message body is treated as raw bytes, and its value is mapped to MGW_BASIC_MSG_T.raw_body.

TIB/Rendezvous Outbound Propagation

If no propagation transformation is provided for outbound propagation, then the Oracle Streams AQ source queue payload type must be either SYS.MGW_TIBRV_MSG_T or RAW. If a propagation transformation is specified, then the target ADT of the transformation must be SYS.MGW_TIBRV_MSG_T, but the source ADT can be any ADT supported by Oracle Streams AQ.

If the Oracle Streams AQ queue payload or transformation target ADT is SYS.MGW_TIBRV_MSG_T, then every field in the source message is converted to a TIB/Rendezvous message field of the resulting TIB/Rendezvous message. If the reply_subject attribute is not NULL, then the reply subject supplementary information is set. The send_subject field is ignored. If the subscriber option AQ_MsgProperties is specified with a value of TRUE, then the MGW agent generates a field for each Oracle Streams AQ message property in the TIB/Rendezvous message. Table 21-4 shows the field name strings and the corresponding values used in the TIB/Rendezvous message.

If the Oracle Streams AQ queue payload is RAW, then the resulting message contains a field named MGW_RAW_MSG with value TibrvMsg.OPAQUE. The field ID is set to 0.

TIB/Rendezvous Inbound Propagation

If no propagation transformation is provided for inbound propagation, then the Oracle Streams AQ destination queue payload type must be either RAW or SYS.MGW_TIBRV_MSG_T. If a propagation transformation is specified, then the target ADT of the transformation can be any ADT supported by Oracle Streams AQ, but the source ADT of the transformation must be SYS.MGW_TIBRV_MSG_T.

If the Oracle Streams AQ queue payload or transformation source ADT is SYS.MGW_TIBRV_MSG_T, then:

• Every field in the source TIB/Rendezvous message is converted to a field of the resulting message of the SYS.MGW_TIBRV_MSG_T type.

• The MGW agent extracts the send subject name from the source TIB/Rendezvous message and sets the send_subject attribute in SYS.MGW_TIBRV_MSG_T. The send subject name is usually the same as the subject name of the registered propagation source queue, but it might be different when wildcards are used.

• The MGW agent extracts the reply subject name from the source TIB/Rendezvous message, if it exists, and sets the reply_subject attribute in SYS.MGW_TIBRV_MSG_T.

• If the source TIB/Rendezvous message contains more than three large text fields (greater than 4000 bytes of text) or more than three large bytes fields (greater than 2000 bytes), then message conversion fails.

If the Oracle Streams AQ queue payload is RAW, then:

• The Oracle Streams AQ message payload is the field data if the source TIB/Rendezvous message has a field named MGW_RAW_MSG of type TibrvMsg.OPAQUE or TibrvMsg.XML. The field name and ID are ignored. If no such field exists or has an unexpected type, then a message conversion failure occurs.

• A message conversion failure occurs if the RAW data size is greater than 32KB. This is due to a restriction on the data size allowed for a bind variable. Also, the actual limit is 32512 rather than 32767.

If the subscriber option AQ_MsgProperties is specified with a value of TRUE, then the MGW agent searches for fields in the original TIB/Rendezvous messages with reserved field names. Table 21-4 shows the field name strings and the corresponding values used in the TIB/Rendezvous message.

If such fields exist, then the MGW agent uses the field value to set the corresponding Oracle Streams AQ message properties, instead of using the default values. If there is more than one such field with the same name, then only the first one is used. Such fields are removed from the resulting payload only if the Oracle Streams AQ queue payload is RAW. If a field with the reserved name does not have the expected datatype, then it causes a message conversion failure.

JMS Outbound Propagation

When dequeuing a message from an Oracle Streams AQ queue, Oracle JMS converts instances of the ADTs shown in Table 21-5 into JMS messages. In addition it can convert instances of SYS.ANYDATA into JMS messages, depending on the content.

A queue with payload type SYS.ANYDATA can hold messages that do not map to a JMS message. MGW fails to dequeue such a message. An error is logged and propagation of messages from that queue does not continue until the message is removed.

JMS Inbound Propagation

Every message successfully dequeued using WebSphere MQ JMS is a JMS message. No message conversion is necessary prior to enqueuing using Oracle JMS. However, if the payload ADT of the propagation destination does not accept the type of the inbound message, then an exception is logged and an attempt is made to place the message in an exception queue. An example of such type mismatches is a JMS_TextMessage ADT and a queue payload type SYS.AQ$_JMS_BYTES_MESSAGE.