22 SQL for Modeling

How Data is Processed in a SQL Model

Figure 22-2 shows the flow of processing within a simple MODEL clause. In this case, we will follow data through a MODEL clause that includes three rules. One of the rules updates an existing value, while the other two create new values for a forecast. The figure shows that the rows of data retrieved by a query are fed into the MODEL clause and rearranged into an array. Once the array is defined, rules are applied one by one to the data. Finally, the data, including both its updated values and newly created values, is rearranged into row form and presented as the results of the query.

Figure 22-2 Model Flow Processing

Description of the illustration dwhsg120.gif

Why Use SQL Modeling?

Oracle modeling enables you to perform sophisticated calculations on your data. A typical case is when you want to apply business rules to data and then generate reports. Because Oracle Database integrates modeling calculations into the database, performance and manageability are enhanced significantly. Consider the following query:

SELECT SUBSTR(country, 1, 20) country, 
      SUBSTR(product, 1, 15) product, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL
  PARTITION BY (country) DIMENSION BY (product, year)
  MEASURES (sales sales)  
  RULES
  (sales['Bounce', 2002] = sales['Bounce', 2001] + sales['Bounce', 2000],
   sales['Y Box', 2002] = sales['Y Box', 2001],
   sales['All_Products', 2002] = sales['Bounce', 2002] + sales['Y Box', 2002])
ORDER BY country, product, year;

This query partitions the data in sales_view (which is illustrated in "Base Schema") on country so that the model computation, as defined by the three rules, is performed on each country. This model calculates the sales of Bounce in 2002 as the sum of its sales in 2000 and 2001, and sets the sales for Y Box in 2002 to the same value as they were in 2001. Also, it introduces a new product category All_Products (sales_view does not have the product All_Products) for year 2002 to be the sum of sales of Bounce and Y Box for that year. The output of this query is as follows, where bold text indicates new values:

COUNTRY              PRODUCT               YEAR      SALES
-------------------- --------------- ---------- ----------
Italy                Bounce                1999    2474.78
Italy                Bounce                2000    4333.69
Italy                Bounce                2001     4846.3
Italy                Bounce                2002    9179.99
...
Italy                Y Box                 1999   15215.16
Italy                Y Box                 2000   29322.89
Italy                Y Box                 2001   81207.55
Italy                Y Box                 2002   81207.55
...
Italy                All_Products          2002   90387.54
...
Japan                Bounce                1999     2961.3
Japan                Bounce                2000    5133.53
Japan                Bounce                2001     6303.6
Japan                Bounce                2002   11437.13
...
Japan                Y Box                 1999   22161.91
Japan                Y Box                 2000   45690.66
Japan                Y Box                 2001   89634.83
Japan                Y Box                 2002   89634.83
...
Japan                All_Products          2002  101071.96
...

Note that, while the sales values for Bounce and Y Box exist in the input, the values for All_Products are derived.

SQL Modeling Capabilities

Oracle Database provides the following capabilities with the MODEL clause:

• Symbolic cell addressing

  Measure columns in individual rows are treated like cells in a multi-dimensional array and can be referenced and updated using symbolic references to dimension values. For example, in a fact table ft(country, year, sales), you can designate country and year to be dimension columns and sales to be the measure and reference sales for a given country and year as sales[country='Spain', year=1999]. This gives you the sales value for Spain in 1999. You can also use a shorthand form sales['Spain', 1999] to mean the same thing. There are a few semantic differences between these notations, though. See "Cell Referencing" for further details.

• Symbolic array computation

  You can specify a series of formulas, called rules, to operate on the data. Rules can invoke functions on individual cells or on a set or range of cells. An example involving individual cells is the following:

  sales[country='Spain',year=2001] = sales['Spain',2000]+ sales['Spain',1999]
  
  

  This sets the sales in Spain for the year 2001 to the sum of sales in Spain for 1999 and 2000. An example involving a range of cells is the following:

  sales[country='Spain',year=2001] = 
     MAX(sales)['Spain',year BETWEEN 1997 AND 2000]
  
  

  This sets the sales in Spain for the year 2001 equal to the maximum sales in Spain between 1997 and 2000.

• The UPSERT and UPDATE options

  Using the UPSERT option, which is the default, you can create cell values that do not exist in the input data. If the cell referenced exists in the data, it is updated. Otherwise, it is inserted. The UPDATE option, on the other hand, would not insert any new cells.

  You can specify these options globally, in which case they apply to all rules, or per each rule. If you specify an option at the rule level, it overrides the global option. Consider the following rules:

  UPDATE sales['Spain', 1999] = 3567.99,
  UPSERT sales['Spain', 2001] = sales['Spain', 2000]+ sales['Spain', 1999]
  
  

  The first rule updates the cell for sales in Spain for 1999. The second rule updates the cell for sales in Spain for 2001 if it exists, otherwise, it creates a new cell.

• Wildcard specification of dimensions

  You can use ANY and IS ANY to specify all values in a dimension. As an example, consider the following statement:

  sales[ANY, 2001] = sales['Japan', 2000]
  
  

  This rule sets the 2001 sales of all countries equal to the sales value of Japan for the year 2000. All values for the dimension, including nulls, satisfy the ANY specification. You can specify the same in symbolic form using an IS ANY predicate as in the following:

  sales[country IS ANY, 2001] = sales['Japan', 2000]
  
  

• Accessing dimension values using the CV function

  You can use the CV function on the right side of a rule to access the value of a dimension column of the cell referenced on the left side of a rule. It enables you to combine multiple rules performing similar computation into a single rule, thus resulting in concise specification. For example, you can combine the following rules:

  sales[country='Spain', year=2002] = 1.2 * sales['Spain', 2001],
  sales[country='Italy', year=2002] = 1.2 * sales['Italy', 2001],
  sales[country='Japan', year=2002] = 1.2 * sales['Japan', 2001]
  
  

  They can be combined into one single rule:

  sales[country IN ('Spain', 'Italy', 'Japan'), year=2002] = 1.2 * 
     sales[CV(country), 2001]
  
  

  Observe that the CV function passes the value for the country dimension from the left to the right side of the rule.

• Ordered computation

  For rules updating a set of cells, the result may depend on the ordering of dimension values. You can force a particular order for the dimension values by specifying an ORDER BY in the rule. An example is the following rule:

  sales[country IS ANY, year BETWEEN 2000 AND 2003] ORDER BY year =   1.05 * sales[CV(country), CV(year)-1]
  
  

  This ensures that the years are referenced in increasing chronological order.

• Automatic rule ordering

  Rules in the MODEL clause can be automatically ordered based on dependencies among the cells using the AUTOMATIC ORDER keywords. For example, in the following assignments, the last two rules will be processed before the first rule because the first depends on the second and third:

  RULES AUTOMATIC ORDER
  {sales[c='Spain', y=2001] = sales[c='Spain', y=2000] 
    + sales[c='Spain', y=1999]
  sales[c='Spain', y=2000] = 50000,
  sales[c='Spain', y=1999] = 40000}
  
  

• Iterative rule evaluation

  You can specify iterative rule evaluation, in which case the rules are evaluated iteratively until the termination condition is satisfied. Consider the following specification:

  MODEL DIMENSION BY (x) MEASURES (s)
    RULES ITERATE (4) (s[x=1] = s[x=1]/2)
  
  

  This statement specifies that the formula s[x=1] = s[x=1]/2 evaluation be repeated four times. The number of iterations is specified in the ITERATE option of the MODEL clause. It is also possible to specify a termination condition by using an UNTIL clause.

  Iterative rule evaluation is an important tool for modeling recursive relationships between entities in a business application. For example, a loan amount might depend on the interest rate where the interest rate in turn depends on the amount of the loan.

• Reference models

  Rules can reference cells from different multi-dimensional arrays. All but one of the multi-dimensional arrays used in the model are read-only and are called reference models. Rules can update or insert cells in only one multi-dimensional array, which is called the main model. The use of reference models enables you to relate models with different dimensionality. For example, assume that, in addition to the fact table ft(country, year, sales), you have a table with currency conversion ratios cr(country, ratio) with country as the dimension column and ratio as the measure. Each row in this table gives the conversion ratio of that country's currency to that of US dollar. These two tables could be used in rules such as the following:

  dollar_sales['Spain',2001] = sales['Spain',2000] * ratio['Spain']
  
  

• Scalable computation

  You can partition data and evaluate rules within each partition independent of other partitions. This enables parallelization of model computation based on partitions. For example, consider the following model:

  MODEL PARTITION BY (c) DIMENSION BY (y) MEASURES (s)
    (sales[y=2001] = AVG(s)[y BETWEEN 1990 AND 2000]
  
  

  The data is partitioned by country and, within each partition, you can compute the sales in 2001 to be the average of sales in the years between 1990 and 2000. Partitions can be processed in parallel and this results in a scalable execution of the model.

Base Schema

This chapter's examples are based on the following view sales_view, which is derived from the sh sample schema.

CREATE VIEW sales_view AS
SELECT country_name country, prod_name product, calendar_year year,
  SUM(amount_sold) sales, COUNT(amount_sold) cnt,
  MAX(calendar_year) KEEP (DENSE_RANK FIRST ORDER BY SUM(amount_sold) DESC)
  OVER (PARTITION BY country_name, prod_name) best_year,
  MAX(calendar_year) KEEP (DENSE_RANK LAST ORDER BY SUM(amount_sold) DESC)
  OVER (PARTITION BY country_name, prod_name) worst_year
FROM sales, times, customers, countries, products
WHERE sales.time_id = times.time_id AND sales.prod_id = products.prod_id AND
  sales.cust_id =customers.cust_id AND customers.country_id=countries.country_id
GROUP BY country_name, prod_name, calendar_year;

This query computes SUM and COUNT aggregates on the sales data grouped by country, product, and year. It will report for each product sold in a country, the year when the sales were the highest for that product in that country. This is called the best_year of the product. Similarly, worst_year gives the year when the sales were the lowest.

MODEL Clause Syntax

The MODEL clause enables you to define multi-dimensional calculations on the data in the SQL query block. In multi-dimensional applications, a fact table consists of columns that uniquely identify a row with the rest serving as dependent measures or attributes. The MODEL clause lets you specify the PARTITION, DIMENSION, and MEASURE columns that define the multi-dimensional array, the rules that operate on this multi-dimensional array, and the processing options.

The MODEL clause contains a list of updates representing array computation within a partition and is a part of a SQL query block. Its structure is as follows:

MODEL
[<global reference options>]
[<reference models>]
[MAIN <main-name>]
  [PARTITION BY (<cols>)]
  DIMENSION BY (<cols>)
  MEASURES (<cols>)
  [<reference options>]
  [RULES]  <rule options>
  (<rule>, <rule>,.., <rule>)

  <global reference options> ::= <reference options> <ret-opt>
   <ret-opt> ::= RETURN {ALL|UPDATED} ROWS
  <reference options> ::=
  [IGNORE NAV | [KEEP NAV]
  [UNIQUE DIMENSION | UNIQUE SINGLE REFERENCE]
  <rule options> ::=
  [UPSERT | UPDATE]
  [AUTOMATIC ORDER | SEQUENTIAL ORDER]
  [ITERATE (<number>)  [UNTIL <condition>]]
  <reference models> ::= REFERENCE ON <ref-name> ON (<query>)
  DIMENSION BY (<cols>) MEASURES (<cols>) <reference options>

Each rule represents an assignment. Its left side references a cell or a set of cells and the right side can contain expressions involving constants, host variables, individual cells or aggregates over ranges of cells. For example, consider Example 22-1.

SELECT SUBSTR(country,1,20) country, SUBSTR(product,1,15) product, year, sales
FROM sales_view
WHERE country in ('Italy', 'Japan')
MODEL
  RETURN UPDATED ROWS
  MAIN simple_model
  PARTITION BY (country)
  DIMENSION BY (product, year)
  MEASURES (sales)
  RULES
   (sales['Bounce', 2001] = 1000,
    sales['Bounce', 2002] = sales['Bounce', 2001] + sales['Bounce', 2000],
    sales['Y Box', 2002] = sales['Y Box', 2001])
ORDER BY country, product, year;

COUNTRY              PRODUCT               YEAR      SALES
-------------------- --------------- ---------- ----------
Italy                Bounce                2001       1000
Italy                Bounce                2002    5333.69
Italy                Y Box                 2002   81207.55
Japan                Bounce                2001       1000
Japan                Bounce                2002    6133.53
Japan                Y Box                 2002   89634.83

SELECT SUBSTR(country,1,20) country, SUBSTR(product,1,15) product, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan');

COUNTRY              PRODUCT               YEAR      SALES
-------------------- --------------- ---------- ----------
Italy                Bounce                1999    2474.78
Italy                Bounce                2000    4333.69
Italy                Bounce                2001     4846.3
...

SELECT country, p product, year, sales, profits
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL
  RETURN UPDATED ROWS
  PARTITION BY (SUBSTR(country,1,20) AS country)
  DIMENSION BY (product AS p, year)
  MEASURES (sales, 0 AS profits)
  RULES
   (profits['Bounce', 2001] = sales['Bounce', 2001] * 0.25,
    sales['Bounce', 2002] = sales['Bounce', 2001] + sales['Bounce', 2000],
    profits['Bounce', 2002] = sales['Bounce', 2002] * 0.35)
ORDER BY country, year;

COUNTRY   PRODUCT    YEAR   SALES         PROFITS
-------   ---------  ----   --------     --------
Italy     Bounce     2001     4846.3     1211.575
Italy     Bounce     2002    9179.99    3212.9965
Japan     Bounce     2001     6303.6       1575.9
Japan     Bounce     2002   11437.13    4002.9955

See Oracle Database SQL Reference for more information regarding MODEL clause syntax.

Keywords in SQL Modeling

This section defines keywords used in SQL modeling.

Cell Referencing

In the MODEL clause, a relation can be viewed as a multi-dimensional array of cells. A cell of this multi-dimensional array contains the measure values and is indexed using DIMENSION BY keys, within each partition defined by the PARTITION BY keys. For example, consider the following:

SELECT country, product, year, sales, best_year, best_year
FROM sales_view
MODEL 
  PARTITION BY (country) DIMENSION BY (product, year) 
MEASURES (sales, best_year)
(<rules> ..)
ORDER BY country, product, year;

This partitions the data by country and defines within each partition, a two-dimensional array on product and year. The cells of this array contain two measures: sales and best_year.

Accessing the measure value of a cell by specifying the DIMENSION BY keys constitutes a cell reference. An example of a cell reference is sales[product= 'Bounce', year=2000].

Here, we are accessing the sales value of a cell referenced by product Bounce and the year 2000. In a cell reference, you can specify DIMENSION BY keys either symbolically as in the preceding cell reference or positionally as in sales['Bounce', 2000].

sales['Bounce', 2001]

sales['Bounce', year=2001]

sales[year>=2001],
sales[product='Bounce', year < 2001], AND
sales[product LIKE '%Finding Fido%', year IN (1999, 2000, 2001)]

Rules

Model computation is expressed in rules that manipulate the cells of the multi-dimensional array defined by PARTITION BY, DIMENSION BY, and MEASURES clauses. A rule is an assignment statement whose left side represents a cell or a range of cells and whose right side is an expression involving constants, bind variables, individual cells or an aggregate function on a range of cells. Rules can use wild cards and looping constructs for maximum expressiveness. An example of a rule is the following:

sales['Bounce', 2003] = 1.2 * sales['Bounce', 2002]

This rule says that, for the product Bounce, the sales for 2003 are 20% more than that of 2002.

Note that this rule has single cell references on both left and right side and is relatively simple. Complex rules can be written with multi-cell references, aggregates, and nested cell references. These are discussed in the following sections.

sales[product='Finding Fido', year=2003] = 100000
sales['Bounce', 2003] = 1.2 * sales['Bounce', 2002]
sales[product='Finding Fido', year=2004] = 0.8 * sales['Standard Mouse Pad',
  year=2003] + sales['Finding Fido', 2003]

sales['Bounce', 2003] = 100 + MAX(sales)['Bounce', year BETWEEN 1998 AND 2002]

sales[product='Finding Fido', year=2003] = 1.3 * 
  PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY sales) [product IN ('Finding
  Fido','Standard Mouse Pad','Boat'), year < 2003]

SELECT country, product, year, sales
FROM sales_view
WHERE country in ('Poland', 'France')
MODEL
  PARTITION BY (country)
  DIMENSION BY (product, year)
  MEASURES (sales sales, year y)
  RULES UPSERT
  (sales['Finding Fido', 2003] = 
     REGR_SLOPE(sales, y)['Finding Fido', year < 2002] * 
     sales['Finding Fido', 2002] + sales['Finding Fido', 2002]);

sales['Bounce', 2003] = 
   AVG(sales * weight)['Bounce', year BETWEEN 1998 AND 2002]

sales['Standard Mouse Pad', year > 2000] = 
   0.2 * sales['Finding Fido', year=2000]

sales[product='Standard Mouse Pad', year>2000] =   sales[CV(product), CV(year)] + 0.2 * sales['Finding Fido', 2000]

sales['Standard Mouse Pad', 2001] + 0.2 * sales['Finding Fido', 2000]

sales['Standard Mouse Pad', 2002] + 0.2 * sales['Finding Fido', 2000]

sales[product='Standard Mouse Pad', year>2000] =
   sales[CV(), CV()] + 0.2 * sales['Finding Fido', 2000]

sales[product IN ('Finding Fido','Standard Mouse Pad','Bounce'), year 
  BETWEEN 2002 AND 2004] = 2 * sales[CV(product), CV(year)-10]

sales[product IS ANY, 2003] = 1.1 * sales[CV(product), 2002]

sales[ANY, 2003] = 1.1 * sales[CV(), 2002]

sales[product='Bounce', year = best_year['Bounce', 2003]]

sales['Standard Mouse Pad', 2003] = (sales[CV(), best_year['Finding Fido',
  CV(year)]] + sales[CV(), worst_year['Finding Fido', CV(year)]]) / 2

Order of Evaluation of Rules

By default, rules are evaluated in the order they appear in the MODEL clause. You can specify an optional keyword SEQUENTIAL ORDER in the MODEL clause to make such an evaluation order explicit. SQL models with sequential rule order of evaluation are called sequential order models. For example, the following RULES specification makes Oracle evaluate rules in the specified sequence:

RULES SEQUENTIAL ORDER
 (sales['Bounce', 2001] = 
    sales['Bounce', 2000] + sales['Bounce', 1999],   --Rule R1
  sales['Bounce', 2000] = 50000,                     --Rule R2
  sales['Bounce', 1999] = 40000)                     --Rule R3

Alternatively, the option AUTOMATIC ORDER enables Oracle to determine the order of evaluation of rules automatically. Oracle examines the cell references within rules and constructs a dependency graph based on dependencies among rules. If cells referenced on the left side of rule R1 are referenced on the right side of another rule R2, then R2 is considered to depend on R1. In other words, rule R1 should be evaluated before rule R2. If you specify AUTOMATIC ORDER in the preceding example as in:

RULES AUTOMATIC ORDER
 (sales['Bounce', 2001] = sales['Bounce', 2000] + sales['Bounce', 1999],
  sales['Bounce', 2000] = 50000,
  sales['Bounce', 1999] = 40000)

Rules 2 and 3 are evaluated, in some arbitrary order, before rule 1. This is because rule 1 depends on rules 2 and 3 and hence need to be evaluated after rules 2 and 3. The order of evaluation among second and third rules can be arbitrary as they do not depend on one another. The order of evaluation among rules independent of one another can be arbitrary. A dependency graph is analyzed to come up with the rule evaluation order. SQL models with automatic order of evaluation, as in the preceding fragment, are called automatic order models.

In an automatic order model, multiple assignments to the same cell are not allowed. In other words, measure of a cell can be assigned only once. Oracle will return an error in such cases as results would be non-deterministic. For example, the following rule specification will generate an error as sales['Bounce', 2001] is assigned more than once:

RULES AUTOMATIC ORDER
 (sales['Bounce', 2001] = sales['Bounce', 2000] + sales['Bounce', 1999],
  sales['Bounce', 2001] = 50000,
  sales['Bounce', 2001] = 40000)

The rules assigning the sales of product Bounce for 2001 do not depend on one another and hence, no particular evaluation order can be fixed among them. This leads to non-deterministic results as the evaluation order is arbitrary - sales['Bounce', 2001] can be 40000 or 50000 or sum of Bounce sales for years 1999 and 2000. Oracle prevents this by disallowing multiple assignments when AUTOMATIC ORDER is specified. However, multiple assignments are fine in sequential order models. If SEQUENTIAL ORDER was specified instead of AUTOMATIC ORDER in the preceding example, the result of sales['Bounce', 2001] would be 40000.

Differences Between Update and Upsert

Rules in the MODEL clause have UPSERT semantics by default. If the cell referenced on the left side of a rule exists, then its measure is updated with the value of the right side expression. Otherwise, if a cell reference is positional, a new cell is created (that is, inserted in to the multi-dimensional array) with the measure value equal to the value of the right side expression. If a cell reference is not positional, it will not insert cells. Note that if any column in a cell is symbolic, inserts are not possible. For example, consider the following rule:

sales['Bounce', 2003] = sales['Bounce', 2001] + sales ['Bounce', 2002]

The cell for product Bounce and year 2003, if it exists, gets updated with the sum of Bounce sales for years 2001 and 2002, otherwise, it gets created. An optional UPSERT keyword can be specified in the MODEL clause to make this upsert semantic explicit.

Alternatively, the UPDATE option forces strict update mode. In this mode, the rule is ignored if the cell it references on the left side does not exist.

You can specify an UPDATE or UPSERT option at the global level in the RULES clause in which case all rules operate in the respective mode. These options can be specified at a local level with each rule and in which case, they override the global behavior. For example, in the following specification:

RULES UPDATE 
(UPDATE s['Bounce',2001] = sales['Bounce',2000] + sales['Bounce',1999],
 UPSERT s['Y Box', 2001] = sales['Y Box', 2000] + sales['Y Box', 1999],
   sales['Mouse Pad', 2001] = sales['Mouse Pad', 2000] +
   sales['Mouse Pad',1999])

The UPDATE option is specified at global level so, first and third rules operate in update mode. The second rule operates in upsert mode as an UPSERT keyword is specified with that rule. Note that no option was specified for the third rule and hence it inherits the update behavior from the global option.

Using UPSERT would create a new cell corresponding to the one referenced on the left side of the rule when the cell is missing, and the cell reference contains only positional references qualified by constants.

Assuming we do not have cells for years greater than 2003, consider the following rule:

UPSERT sales['Bounce', year = 2004] = 1.1 * sales['Bounce', 2002]

This would not create any new cell because of the symbolic reference year = 2004. However, consider the following:

UPSERT sales['Bounce', 2004] = 1.1 * sales['Bounce', 2002]

This would create a new cell for product Bounce for year 2004. On a related note, new cells will not be created if any of the positional reference is ANY. This is because ANY is predicate that qualifies all dimensional values including NULL. If there is positional reference ANY for a dimension d, then it can be considered as a predicate (d IS NOT NULL OR d IS NULL).

Treatment of NULLs and Missing Cells

Applications using models would not only have to deal with non-deterministic values for a cell measure in the form of NULL, but also with non-determinism in the form of missing cells. A cell, referenced by a single cell reference, that is missing in the data is called a missing cell. The MODEL clause provides a default treatment for nulls and missing cells that is consistent with the ANSI SQL standard and also provides options to treat them in other useful ways according to business logic, for example, to treat nulls as zero for arithmetic operations.

By default, NULL cell measure values are treated the same way as nulls are treated elsewhere in SQL. For example, in the following rule:

sales['Bounce', 2001] = sales['Bounce', 1999] + sales['Bounce', 2000]

The right side expression would evaluate to NULL if Bounce sales for one of the years 1999 and 2000 is NULL. Similarly, aggregate functions in rules would treat NULL values in the same way as their regular behavior where NULL values are ignored during aggregation.

Missing cells are treated as cells with NULL measure values. For example, in the preceding rule, if the cell for Bounce and 2000 is missing, then it is treated as a NULL value and the right side expression would evaluate to NULL.

PRESENTV(sales['Bounce', 2000], 1.1*sales['Bounce', 2000], 100)

PRESENTNNV(sales['Bounce', 2000], 1.1*sales['Bounce', 2000], 100)

CASE WHEN sales['Bounce', 2000] IS PRESENT AND sales['Bounce', 2000] IS NOT NULL
THEN
1.1 * sales['Bounce', 2000]
ELSE
100
END

RULES
 (UPSERT sales['Bounce', 2003] =
  PRESENTV(sales['Bounce', 2003], sales['Bounce', 2003], 1000),
  UPSERT profit['Bounce', 2003] = 
  PRESENTV(profit['Bounce', 2003], profit['Bounce', 2003], 500))

SELECT product, year, sales
FROM sales_view
WHERE country = 'Poland'
MODEL
  DIMENSION BY (product, year) MEASURES (sales sales) IGNORE NAV
  RULES UPSERT
  (sales['Bounce', 2003] = sales['Bounce', 2002] + sales['Bounce', 2001]);

Reference Models

In addition to the multi-dimensional array on which rules operate, which is called the main model, one or more read-only multi-dimensional arrays, called reference models, can be created and referenced in the MODEL clause to act as look-up tables. Like the main model, a reference model is defined over a query block and has DIMENSION BY and MEASURES clauses to indicate its dimensions and measures respectively. A reference model is created by the following subclause:

REFERENCE model_name ON (query) DIMENSION BY (cols) MEASURES (cols) 
   [reference options]

Like the main model, a multi-dimensional array for the reference model is built before evaluating the rules. But, unlike the main model, reference models are read-only in that their cells cannot be updated and no new cells can be inserted after they are built. Thus, the rules in the main model can access cells of a reference model, but they cannot update or insert new cells into the reference model. References to the cells of a reference model can only appear on the right side of rules. You can view reference models as look-up tables on which the rules of the main model perform look-ups to obtain cell values. The following is an example using a currency conversion table as a reference model:

CREATE TABLE dollar_conv_tbl(country VARCHAR2(30), exchange_rate NUMBER);
INSERT INTO dollar_conv_tbl VALUES('Poland', 0.25);
INSERT INTO dollar_conv_tbl VALUES('France', 0.14);
...

Now, to convert the projected sales of Poland and France for 2003 to the US dollar, you can use the dollar conversion table as a reference model as in the following:

SELECT country, year, sales, dollar_sales
FROM sales_view 
GROUP BY country, year
MODEL
  REFERENCE conv_ref ON (SELECT country, exchange_rate FROM dollar_conv_tbl)
  DIMENSION BY (country) MEASURES (exchange_rate) IGNORE NAV
MAIN conversion
DIMENSION BY (country, year)
   MEASURES (SUM(sales) sales, SUM(sales) dollar_sales) IGNORE NAV
RULES
(dollar_sales['France', 2003] = sales[CV(country), 2002] * 1.02 * 
   conv_ref.exchange_rate['France'], 
   dollar_sales['Poland', 2003] = 
      sales['Poland', 2002] * 1.05 * exchange_rate['Poland']);

Observe in this example that:

• A one dimensional reference model named conv_ref is created on rows from the table dollar_conv_tbl and that its measure exchange_rate has been referenced in the rules of the main model.

• The main model (called conversion) has two dimensions, country and year, whereas the reference model conv_ref has one dimension, country.

• Different styles of accessing the exchange_rate measure of the reference model. For France, it is rather explicit with model_name.measure_name notation conv_ref.exchange_rate, whereas for Poland, it is a simple measure_name reference exchange_rate. The former notation needs to be used to resolve any ambiguities in column names across main and reference models.

Growth rates, in this example, are hard coded in the rules. The growth rate for France is 2% and that of Poland is 5%. But they could come from a separate table and you can have a reference model defined on top of that. Assume that you have a growth_rate(country, year, rate) table defined as the following:

CREATE TABLE growth_rate_tbl(country VARCHAR2(30), 
   year NUMBER, growth_rate NUMBER);
INSERT INTO growth_rate_tbl VALUES('Poland', 2002, 2.5);
INSERT INTO growth_rate_tbl VALUES('Poland', 2003, 5);
...
INSERT INTO growth_rate_tbl VALUES('France', 2002, 3);
INSERT INTO growth_rate_tbl VALUES('France', 2003, 2.5);

Then the following query computes the projected sales in dollars for 2003 for all countries:

SELECT country, year, sales, dollar_sales
FROM sales_view 
GROUP BY country, year 
MODEL
  REFERENCE conv_ref ON
          (SELECT country, exchange_rate FROM dollar_conv_tbl)
           DIMENSION BY (country c) MEASURES (exchange_rate) IGNORE NAV
  REFERENCE growth_ref ON 
          (SELECT country, year, growth_rate FROM growth_rate_tbl)
           DIMENSION BY (country c, year y) MEASURES (growth_rate) IGNORE NAV
  MAIN projection  
   DIMENSION BY (country, year) MEASURES (SUM(sales) sales, 0 dollar_sales)
  IGNORE NAV 
  RULES  
  (dollar_sales[ANY, 2003] = sales[CV(country), 2002] * 
   growth_rate[CV(country), CV(year)] * 
   exchange_rate[CV(country)]);

This query shows the capability of the MODEL clause in dealing with and relating objects of different dimensionality. Reference model conv_ref has one dimension while the reference model growth_ref and the main model have two dimensions. Dimensions in the single cell references on reference models are specified using the CV function thus relating the cells in main model with the reference model. This specification, in effect, is performing a relational join between main and reference models.

Reference models also help you convert keys to sequence numbers, perform computations using sequence numbers (for example, where a prior period would be used in a subtraction operation), and then convert sequence numbers back to keys. For example, consider a view that assigns sequence numbers to years:

CREATE or REPLACE VIEW year_2_seq (i, year) AS 
SELECT ROW_NUMBER() OVER (ORDER BY calendar_year), calendar_year 
FROM (SELECT DISTINCT calendar_year FROM TIMES); 

This view can define two lookup tables: integer-to-year i2y, which maps sequence numbers to integers, and year-to-integer y2i, which performs the reverse mapping. The references y2i.i[year] and y2i.i[year] - 1 return sequence numbers of the current and previous years respectively and the reference i2y.y[y2i.i[year]-1] returns the year key value of the previous year. The following query demonstrates such a usage of reference models:

SELECT country, product, year, sales, prior_period
FROM sales_view
MODEL
  REFERENCE y2i ON (SELECT year, i FROM year_2_seq) DIMENSION BY (year y) 
   MEASURES (i)
  REFERENCE i2y ON (SELECT year, i FROM year_2_seq) DIMENSION BY (i) 
   MEASURES (year y)
  MAIN projection2 PARTITION BY (country)
  DIMENSION BY (product, year) 
  MEASURES (sales, CAST(NULL AS NUMBER) prior_period)
(prior_period[ANY, ANY] = sales[CV(product), i2y.y[y2i.i[CV(year)]-1]])
ORDER BY country, product, year;

Nesting of reference model cell references is evident in the preceding example. Cell reference on the reference model y2i is nested inside the cell reference on i2y which, in turn, is nested in the cell reference on the main SQL model. There is no limitation on the levels of nesting you can have on reference model cell references. However, you can only have two levels of nesting on the main SQL model cell references.

Finally, the following are restrictions on the specification and usage of reference models:

• Reference models cannot have a PARTITION BY clause.

• The query block on which the reference model is defined cannot be correlated to an outer query.

• Reference models must be named and their names should be unique.

• All references to the cells of a reference model should be single cell references.

FOR Loops

The MODEL clause provides a FOR construct that can be used inside rules to express computations more compactly. It can be used on both the left and right side of a rule. For example, consider the following computation, which estimates the sales of several products for 2004 to be 10% higher than their sales for 2003:

RULES UPSERT
(sales['Bounce', 2004] = 1.1 * sales['Bounce', 2003],
 sales['Standard Mouse Pad', 2004] = 1.1 * sales['Standard Mouse Pad', 2003],
...
 sales['Y Box', 2004] = 1.1 * sales['Y Box', 2003])

The UPSERT option is used in this computation so that cells for these products and 2004 will be inserted if they are not previously present in the multi-dimensional array. This is rather bulky as you have to have as many rules as there are products. Using the FOR construct, this computation can be represented compactly and with exactly the same semantics as in:

RULES UPSERT 
(sales[FOR product IN ('Bounce', 'Standard Mouse Pad', ..., 'Y Box'), 2004] = 
   1.1 * sales[CV(product), 2003])

If you write a specification similar to this, but without the FOR keyword as in the following:

RULES UPSERT 
(sales[product IN ('Bounce', 'Standard Mouse Pad', ..., 'Y Box'), 2004] = 
   1.1 * sales[CV(product), 2003])

You would get UPDATE semantics even though you have specified UPSERT. In other words, existing cells will be updated but no new cells will be created by this specification. This is because of the symbolic multi-cell reference on product that is treated as a predicate. You can view a FOR construct as a macro that generates multiple rules with positional references from a single rule, thus preserving the UPSERT semantics. Conceptually, the following rule:

sales[FOR product IN ('Bounce', 'Standard Mouse Pad', ..., 'Y Box'),
      FOR year IN (2004, 2005)] = 1.1 * sales[CV(product),
    CV(year)-1]

Can be treated as an ordered collection of the following rules:

sales['Bounce', 2004] = 1.1 * sales[CV(product), CV(year)-1],
sales['Bounce', 2005] = 1.1 * sales[CV(product), CV(year)-1],
sales['Standard Mouse Pad', 2004] = 1.1 * 
  sales[CV(product), CV(year)-1],
sales['Standard Mouse Pad', 2005] = 1.1 * sales[CV(product),
 CV(year)-1],
...
sales['Y Box', 2004] = 1.1 * sales[CV(product), CV(year)-1],
sales['Y Box', 2005] = 1.1 * sales[CV(product), CV(year)-1]

The FOR construct in the preceding examples is of type FOR dimension IN (list of values). Values in the list should either be constants or expressions involving constants. In this example, there are separate FOR constructs on product and year. It is also possible to specify all dimensions using one FOR construct. Consider for example, we want only to estimate sales for Bounce in 2004, Standard Mouse Pad in 2005 and Y Box in 2004 and 2005. This can be formulated as the following:

sales[FOR (product, year) IN (('Bounce', 2004), ('Standard Mouse Pad', 2005),
  ('Y Box', 2004), ('Y Box', 2005))] = 
     1.1 * sales[CV(product), CV(year)-1]

This FOR construct should be of form FOR (d1, ..., dn) IN ((d1_val1, ..., dn_val1), ..., (d1_valm, ..., dn_valm)] when there are n dimensions d1, ..., dn and m values in the list.

In some cases, the list of values for a dimension in FOR can be stored in a table or they can be the result of a subquery. Oracle Database provides a flavor of FOR construct as in FOR dimension in (subquery) to handle these cases. For example, assume that the products of interest are stored in a table interesting_products, then the following rule estimates their sales in 2004 and 2005:

sales[FOR product IN (SELECT product_name FROM interesting_products)
    FOR year IN (2004, 2005)] = 1.1 * sales[CV(product), CV(year)-1]

As another example, consider the scenario where you want to introduce a new country, called new_country, with sales that mimic those of Poland. This is accomplished by issuing the following statement:

SELECT country, product, year, s
FROM sales_view
MODEL
  DIMENSION BY (country, product, year)
  MEASURES (sales s) IGNORE NAV
  RULES UPSERT
(s[FOR (country, product, year) IN (SELECT DISTINCT 'new_country', product, year
FROM sales_view
WHERE country = 'Poland')] = s['Poland',CV(),CV()])
ORDER BY country, year, product;

Note the multi-column IN-list produced by evaluating the subquery in this specification. The subquery used to obtain the IN-list cannot be correlated to outer query blocks and it should return fewer than 10000 rows. Otherwise, Oracle returns an error. Model has a 10,000 rule limit and each combination of values created in a cell reference creates a rule. Therefore, FOR constructs should be designed so that the combination of values they generate does not exceed 10,000.

If the FOR construct is used to densify sparse data on multiple dimensions, it is possible to encounter the 10,000 rule limit. In those cases, densification can be done outside the MODEL clause using a partitioned outer join. See Chapter 21, " SQL for Analysis and Reporting" for further information.

If you know that the values of interest come from a discrete domain, you can use FOR construct FOR dimension FROM value1 TO value2 [INCREMENT | DECREMENT] value3. This specification results in values between value1 and value2 by starting from value1 and incrementing (or decrementing) by value3. The values value1, value2, and value3 should be constants or expressions involving constants. For example, the following rule:

sales['Bounce', FOR year FROM 2001 TO 2005 INCREMENT 1] = 
  sales['Bounce', year=CV(year)-1] * 1.2

This is semantically equivalent to the following rules in order:

sales['Bounce', 2001] = sales['Bounce', 2000] * 1.2,
sales['Bounce', 2002] = sales['Bounce', 2001] * 1.2,
...
sales['Bounce', 2005] = sales['Bounce', 2004] * 1.2

This kind of FOR construct can be used for dimensions of numeric, date and datetime datatypes. The increment/decrement expression value3 should be numeric for numeric dimensions and can be numeric or interval for dimensions of date or datetime types. Also, value3 should be positive. Oracle will return an error if you use FOR year FROM 2005 TO 2001 INCREMENT -1. You should use either FOR year FROM 2005 TO 2001 DECREMENT 1 or FOR year FROM 2001 TO 2005 INCREMENT 1. Oracle will also report an error if the domain (or the range) is empty, as in FOR year FROM 2005 TO 2001 INCREMENT 1.

To generate string values, you can use the FOR construct FOR dimension LIKE string FROM value1 TO value2 [INCREMENT | DECREMENT] value3. The string string should contain only one % character. This specification results in string by replacing % with values between value1 and value2 with appropriate increment/decrement value value3. For example, the following rule:

sales[FOR product LIKE 'product-%' FROM 1 TO 3 INCREMENT 1, 2003] = 
sales[CV(product), 2002] * 1.2

This equivalent to the following:

sales['product-1', 2003] = sales['product-1', 2002] * 1.2,
sales['product-2', 2003] = sales['product-2', 2002] * 1.2,
sales['product-3', 2003] = sales['product-3', 2002] * 1.2

For this kind of FOR construct, value1, value2, and value3 should all be of numeric type.

sales['product-1', 2003] = sales['product-1', 2002] * 1.2,
sales['product-2', 2003] = sales['product-2', 2002] * 1.2,
sales['product-3', 2003] = sales['product-3', 2002] * 1.2

In SEQUENTIAL ORDER models, rules represented by a FOR construct are evaluated in the order they are generated. On the contrary, rule evaluation order would be dependency based if AUTOMATIC ORDER is specified. For example, the evaluation order for the rules represented by the rule:

sales['Bounce', FOR year FROM 2004 TO 2001 DECREMENT 1] = 
  1.1 * sales['Bounce', CV(year)-1]

For SEQUENTIAL ORDER models would be:

sales['Bounce', 2004] = 1.1 * sales['Bounce', 2003],
sales['Bounce', 2003] = 1.1 * sales['Bounce', 2002],
sales['Bounce', 2002] = 1.1 * sales['Bounce', 2001],
sales['Bounce', 2001] = 1.1 * sales['Bounce', 2000]

While for AUTOMATIC ORDER models, it would be:

sales['Bounce', 2001] = 1.1 * sales['Bounce', 2000],
sales['Bounce', 2002] = 1.1 * sales['Bounce', 2001],
sales['Bounce', 2003] = 1.1 * sales['Bounce', 2002],
sales['Bounce', 2004] = 1.1 * sales['Bounce', 2003]

Iterative Models

Using the ITERATE option of the MODEL clause, you can evaluate rules iteratively for a certain number of times, which you can specify as an argument to the ITERATE clause. ITERATE can be specified only for SEQUENTIAL ORDER models and such models are referred to as iterative models. For example, consider the following:

SELECT x, s FROM DUAL
MODEL 
  DIMENSION BY (1 AS x) MEASURES (1024 AS s)
  RULES UPDATE ITERATE (4)
(s[1] = s[1]/2);

In Oracle, the table DUAL has only one row. Hence this model defines a 1-dimensional array, dimensioned by x with a measure s, with a single element s[1] = 1024. The rule s[1] = s[1]/2 evaluation will be repeated four times. The result of this query will be a single row with values 1 and 64 for columns x and s respectively. The number of iterations arguments for the ITERATE clause should be a positive integer constant. Optionally, you can specify an early termination condition to stop rule evaluation before reaching the maximum iteration. This condition is specified in the UNTIL subclause of ITERATE and is checked at the end of an iteration. So, you will have at least one iteration when ITERATE is specified. The syntax of the ITERATE clause is:

ITERATE (number_of_iterations) [ UNTIL (condition) ]

Iterative evaluation will stop either after finishing the specified number of iterations or when the termination condition evaluates to TRUE, whichever comes first.

In some cases, you may want the termination condition to be based on the change, across iterations, in value of a cell. Oracle provides a mechanism to specify such conditions in that it enables you to access cell values as they existed before and after the current iteration in the UNTIL condition. Oracle's PREVIOUS function takes a single cell reference as an argument and returns the measure value of the cell as it existed after the previous iteration. You can also access the current iteration number by using the system variable ITERATION_NUMBER, which starts at value 0 and is incremented after each iteration. By using PREVIOUS and ITERATION_NUMBER, you can construct complex termination conditions.

Consider the following iterative model that specifies iteration over rules till the change in the value of s[1] across successive iterations falls below 1, up to a maximum of 1000 times:

SELECT x, s, iterations FROM DUAL
MODEL 
  DIMENSION BY (1 AS x) MEASURES (1024 AS s, 0 AS iterations)
  RULES ITERATE (1000) UNTIL ABS(PREVIOUS(s[1]) - s[1]) < 1
 (s[1] = s[1]/2, iterations[1] = ITERATION_NUMBER);

The absolute value function (ABS) can be helpful for termination conditions because you may not know if the most recent value is positive or negative. Rules in this model will be iterated over 11 times as after 11th iteration the value of s[1] would be 0.5. This query results in a single row with values 1, 0.5, 10 for x, s and iterations respectively.

You can use the PREVIOUS function only in the UNTIL condition. However, ITERATION_NUMBER can be anywhere in the main model. In the following example, ITERATION_NUMBER is used in cell references:

SELECT country, product, year, sales
FROM sales_view
MODEL
  PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales sales)
  IGNORE NAV
  RULES ITERATE(3)
(sales['Bounce', 2002 + ITERATION_NUMBER] = sales['Bounce', 1999 
  + ITERATION_NUMBER]);

This statement achieves an array copy of sales of Bounce from cells in the array 1999-2001 to 2002-2005.

Rule Dependency in AUTOMATIC ORDER Models

Oracle Database determines the order of evaluation of rules in an AUTOMATIC ORDER model based on their dependencies. A rule will be evaluated only after the rules it depends on are evaluated. The algorithm chosen to evaluate the rules is based on the dependency graph analysis and whether rules in your model have circular (or cyclical) dependencies. A cyclic dependency can be of the form "rule A depends on B and rule B depends on A" or of the self-cyclic "rule depending on itself" form. An example of the former is:

sales['Bounce', 2002] = 1.5 * sales['Y Box', 2002],
sales['Y Box', 2002] = 100000 / sales['Bounce', 2002

An example of the latter is:

sales['Bounce', 2002] = 25000 / sales['Bounce', 2002]

However, there is no self-cycle in the following rule as different measures are being accessed on the left and right side:

projected_sales['Bounce', 2002] = 25000 / sales['Bounce', 2002]

When the analysis of an AUTOMATIC ORDER model finds that the rules have no circular dependencies (that is, the dependency graph is acyclic), Oracle Database will evaluate the rules in their dependency order. For example, in the following AUTOMATIC ORDER model:

MODEL DIMENSION BY (prod, year) MEASURES (sale sales) IGNORE NAV
  RULES AUTOMATIC ORDER
 (sales['SUV', 2001] = 10000,
  sales['Standard Mouse Pad', 2001] = sales['Finding Fido', 2001] 
    * 0.10 + sales['Boat', 2001] * 0.50,
  sales['Boat', 2001] = sales['Finding Fido', 2001] 
    * 0.25 + sales['SUV', 2001]* 0.75,
  sales['Finding Fido', 2001] = 20000)

Rule 2 depends on rules 3 and 4, while rule 3 depends on rules 1 and 4, and rules 1 and 4 do not depend on any rule. Oracle, in this case, will find that the rule dependencies are acyclic and will evaluate rules in one of the possible evaluation orders (1, 4, 3, 2) or (4, 1, 3, 2). This type of rule evaluation is called an ACYCLIC algorithm.

In some cases, Oracle Database may not be able to ascertain that your model is acyclic even though there is no cyclical dependency among the rules. This can happen if you have complex expressions in your cell references. Oracle Database assumes that the rules are cyclic and employs a CYCLIC algorithm that evaluates the model iteratively based on the rules and data. Iteration will stop as soon as convergence is reached and the results will be returned. Convergence is defined as the state in which further executions of the model will not change values of any of the cell in the model. Convergence is certain to be reached when there are no cyclical dependencies.

If your AUTOMATIC ORDER model has rules with cyclical dependencies, Oracle will employ the earlier mentioned CYCLIC algorithm. Results are produced if convergence can be reached within the number of iterations Oracle is going to try the algorithm. Otherwise, Oracle will report a cycle detection error. You can circumvent this problem by manually ordering the rules and specifying SEQUENTIAL ORDER.

Ordered Rules

An ordered rule is one that has ORDER BY specified on the left side. It accesses cells in the order prescribed by ORDER BY and applies the right side computation. When you have a positional ANY or symbolic references on the left side of a rule but without the ORDER BY clause, Oracle might return an error saying that the rule's results depend on the order in which cells are accessed and hence are non-deterministic. Consider the following SEQUENTIAL ORDER model:

SELECT t, s
FROM sales, times
WHERE sales.time_id = times.time_id
GROUP BY calendar_year
MODEL 
  DIMENSION BY (calendar_year t) MEASURES (SUM(amount_sold) s)
  RULES SEQUENTIAL ORDER
  (s[ANY] = s[CV(t)-1]);

This query attempts to set, for all years t, sales s value for a year to the sales value of the prior year. Unfortunately, the result of this rule depends on the order in which the cells are accessed. If cells are accessed in the ascending order of year, the result would be that of column 3 in Table 22-1. If they are accessed in descending order, the result would be that of column 4.

If you want the cells to be considered in descending order and get the result given in column 4, you should specify:

SELECT t, s
FROM sales, times
WHERE sales.time_id = times.time_id
GROUP BY calendar_year
MODEL 
  DIMENSION BY (calendar_year t) MEASURES (SUM(amount_sold) s)
  RULES SEQUENTIAL ORDER
  (s[ANY] ORDER BY t DESC = s[CV(t)-1]);

In general, you can use any ORDER BY specification as long as it produces a unique order among cells that qualify the left side cell reference. Expressions in the ORDER BY of a rule can involve constants, measures and dimension keys and you can specify the ordering options [ASC | DESC] [NULLS FIRST | NULLS LAST] to get the order you want.

You can also specify ORDER BY for rules in an AUTOMATIC ORDER model to make Oracle consider cells in a particular order during rule evaluation. Rules are never considered self-cyclic if they have ORDER BY. For example, to make the following AUTOMATIC ORDER model with a self-cyclic formula:

MODEL
  DIMENSION BY (calendar_year t) MEASURES (SUM(amount_sold) s)
  RULES AUTOMATIC ORDER
  (s[ANY] = s[CV(t)-1])

acyclic, you need to provide the order in which cells need to be accessed for evaluation using ORDER BY. For example, you can say:

s[ANY] ORDER BY t = s[CV(t) - 1]

Then Oracle will pick an ACYCLIC algorithm (which is certain to produce the result) for formula evaluation.

Unique Dimensions Versus Unique Single References

The MODEL clause, in its default behavior, requires the PARTITION BY and DIMENSION BY keys to uniquely identify each row in the input to the model. Oracle verifies that and returns an error if the data is not unique. Uniqueness of the input rowset on the PARTITION BY and DIMENSION BY keys guarantees that any single cell reference accesses one and only one cell in the model. You can specify an optional UNIQUE DIMENSION keyword in the MODEL clause to make this behavior explicit. For example, the following query:

SELECT country, product, sales
FROM sales_view
WHERE country IN ('France', 'Poland')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (product) MEASURES (sales sales)
  IGNORE NAV RULES UPSERT
(sales['Bounce'] = sales['All Products'] * 0.24); 

This would return a uniqueness violation error as the rowset input to model is not unique on country and product:

ERROR at line 2:ORA-32638: Non unique addressing in MODEL dimensions

However, the following query does not return such an error:

SELECT country, product, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (product, year) MEASURES (sales sales)
  RULES UPSERT
(sales['Bounce', 2003] = sales['All Products', 2002] * 0.24);

Input to the MODEL clause in this case is unique on country, product, and year as shown in:

COUNTRY   PRODUCT                         YEAR   SALES
-------   -----------------------------   ----   --------
Italy     1.44MB External 3.5" Diskette   1998    3141.84
Italy     1.44MB External 3.5" Diskette   1999    3086.87
Italy     1.44MB External 3.5" Diskette   2000    3440.37
Italy     1.44MB External 3.5" Diskette   2001     855.23
...

If you want to relax this uniqueness checking, you can specify UNIQUE SINGLE REFERENCE keyword. This can save processing time. In this case, the MODEL clause checks the uniqueness of only the single cell references appearing on the right side of rules. So the query that returned the uniqueness violation error would be successful if you specify UNIQUE SINGLE REFERENCE instead of UNIQUE DIMENSION.

Another difference between UNIQUE DIMENSION and UNIQUE SINGLE REFERENCE semantics is the number of cells that can be updated by a rule with a single cell reference on left side. In the case of UNIQUE DIMENSION, such a rule can update at most one row as only one cell would qualify the single cell reference on the left side. This is because the input rowset would be unique on PARTITION BY and DIMENSION BY keys. With UNIQUE SINGLE REFERENCE, all cells that qualify the left side single cell reference would be updated by the rule.

Rules and Restrictions when Using SQL for Modeling

The following rules and restrictions apply when using the MODEL clause:

• The only columns that can be updated are the columns specified in the MEASURES subclause of the main SQL model. Measures of reference models cannot be updated.

• The MODEL clause is evaluated after all clauses in the query block except SELECT DISTINCT, and ORDER BY clause are evaluated. These clauses and expressions in the SELECT list are evaluated after the MODEL clause.

• If your query has a MODEL clause, then the query's SELECT and ORDER BY lists cannot contain aggregates or analytic functions. If needed, these can be specified in PARTITION BY, DIMENSION BY, and MEASURES lists and need to be aliased. Aliases can then be used in the SELECT or ORDER BY clauses. In the following example, the analytic function RANK is specified and aliased in the MEASURES list of the MODEL clause, and its alias is used in the SELECT list so that the outer query can order resulting rows based on their ranks.

  SELECT country, product, year, s, RNK
  FROM (SELECT country, product, year, s, rnk
        FROM sales_view
        MODEL
          PARTITION BY (country) DIMENSION BY (product, year)
          MEASURES (sales s, year y, RANK() OVER (ORDER BY sales) rnk)
          RULES UPSERT
            (s['Bounce Increase 90-99', 2001] =
               REGR_SLOPE(s, y) ['Bounce', year BETWEEN 1990 AND 2000],
             s['Bounce', 2001] = s['Bounce', 2000] * 
            (1+s['Bounce increase 90-99', 2001])))
  WHERE product <> 'Bounce Increase 90-99'
  ORDER BY country, year, rnk, product;
  
  

• When there is a multi-cell reference on the right hand side of a rule, you need to apply a function to aggregate the measure values of multiple cells referenced into a single value. You can use any kind of aggregate function for this purpose: regular, OLAP aggregate (inverse percentile, hypothetical rank and distribution), or user-defined aggregate.

• You cannot use analytic functions (functions that use the OVER clause) in rules.

• Only rules with positional single cell references on the left side have UPSERT semantics. All other rules have UPDATE semantics, even when you specify the UPSERT option for them.

• Negative increments are not allowed in FOR loops. Also, no empty FOR loops are allowed. FOR d FROM 2005 TO 2001 INCREMENT -1 is illegal. You should use FOR d FROM 2005 TO 2001 DECREMENT 1 instead. FOR d FROM 2005 TO 2001 INCREMENT 1 is illegal as it designates an empty loop.

• You cannot use nested query expressions (subqueries) in rules except in the FOR construct. For example, it would be illegal to issue the following:

  SELECT *
  FROM sales_view WHERE country = 'Poland'
  MODEL DIMENSION BY (product, year)
    MEASURES (sales sales)
    RULES UPSERT
    (sales['Bounce', 2003] = sales['Bounce', 2002] + 
     (SELECT SUM(sales) FROM sales_view));
  
  

  This is because the rule has a subquery on its right side. Instead, you can rewrite the preceding query in the following legal way:

  SELECT *
  FROM sales_view WHERE country = 'Poland'
  MODEL DIMENSION BY (product, year)
    MEASURES (sales sales, (SELECT SUM(sales) FROM sales_view) AS grand_total)
    RULES UPSERT
    (sales['Bounce', 2003] =sales['Bounce', 2002] + 
     grand_total['Bounce', 2002]);
  
  

• You can also use subqueries in the FOR construct specified on the left side of a rule. However, they:

  • Cannot be correlated

  • Must return fewer than 10000 rows

  • Cannot be a query defined in the WITH clause

  • Will make the cursor unsharable

• Nested cell references must be single cell references. Aggregates on nested cell references are not supported. So, it would be illegal to say s['Bounce', MAX(best_year)['Bounce', ANY]].

• Only one level of nesting is supported for nested cell references on the main model. So, for example, s['Bounce', best_year['Bounce', 2001]] is legal, but s['Bounce', best_year['Bounce', best_year['Bounce', 2001]]] is not.

• Nested cell references appearing on the left side of rules in an AUTOMATIC ORDER model should not be updated in any rule of the model. This restriction ensures that the rule dependency relationships do not arbitrarily change (and hence cause non-deterministic results) due to updates to reference measures.

  There is no such restriction on nested cell references in a SEQUENTIAL ORDER model. Also, this restriction is not applicable on nested references appearing on the right side of rules in both SEQUENTIAL or AUTOMATIC ORDER models.

• Reference models have the following restrictions:

  • The query defining the reference model cannot be correlated to any outer query. It can, however, be a query with subqueries, views and so on.

  • Reference models cannot have a PARTITION BY clause.

  • Reference models cannot be updated.

Parallel Execution

MODEL clause computation is scalable in terms of the number of processors you have. Scalability is achieved by performing the MODEL computation in parallel across the partitions defined by the PARTITION BY clause. Data is distributed among processing elements (also called parallel query slaves) based on the PARTITION BY key values such that all rows with the same values for the PARTITION BY keys will go to the same slave. Note that the internal processing of partitions will not create a one-to-one match of logical and internally processed partitions. This way, each slave can finish MODEL clause computation independent of other slaves. The data partitioning can be hash based or range based. Consider the following MODEL clause:

MODEL 
  PARTITION BY (country) DIMENSION BY (product, time) MEASURES (sales)
  RULES UPDATE
  (sales['Bounce', 2002] = 1.2 * sales['Bounce', 2001],
   sales['Car', 2002] = 0.8 * sales['Car', 2001])

Here input data will be partitioned among slaves based on the PARTITION BY key country and this partitioning can be hash or range based. Each slave will evaluate the rules on the data it receives.

Parallelism of the model computation is governed or limited by the way you specify the MODEL clause. If your MODEL clause has no PARTITION BY keys, then the computation cannot be parallelized (with exceptions mentioned in the following). If PARTITION BY keys have very low cardinality, then the degree of parallelism will be limited. In such cases, Oracle identifies the DIMENSION BY keys that can used for partitioning. For example, consider a MODEL clause equivalent to the preceding one, but without PARTITION BY keys as in the following:

MODEL 
  DIMENSION BY (country, product, time) MEASURES (sales)
  RULES UPDATE
  (sales[ANY, 'Bounce', 2002] = 1.2 * sales[CV(country), 'Bounce', 2001],
   sales[ANY, 'Car', 2002] = 0.8 * sales[CV(country), 'Car', 2001])

In this case, Oracle Database will identify that it can use the DIMENSION BY key country for partitioning and uses region as the basis of internal partitioning. It partitions the data among slaves on country and thus effects parallel execution.

Aggregate Computation

The MODEL clause processes aggregates in two different ways: first, the regular fashion in which data in the partition is scanned and aggregated and second, an efficient window style aggregation. The first type as illustrated in the following introduces a new dimension member ALL_2002_products and computes its value to be the sum of year 2002 sales for all products:

MODEL PARTITION BY (country) DIMENSION BY (product, time) MEASURES (sale sales)
RULES UPSERT 
 (sales['ALL_2002_products', 2002] = SUM(sales)[ANY, 2002])

To evaluate the aggregate sum in this case, each partition will be scanned to find the cells for 2002 for all products and they will be aggregated. If the left side of the rule were to reference multiple cells, then Oracle will have to compute the right side aggregate by scanning the partition for each cell referenced on the left. For example, consider the following example:

MODEL PARTITION BY (country) DIMENSION BY (product, time)
  MEASURES (sale sales, 0 avg_exclusive)
  RULES UPDATE
  (avg_exclusive[ANY, 2002] =  AVG(sales)[product <> CV(product), CV(time)])

This rule calculates a measure called avg_exclusive for every product in 2002. The measure avg_exclusive is defined as the average sales of all products excluding the current product. In this case, Oracle scans the data in a partition for every product in 2002 to calculate the aggregate, and this may be expensive

Oracle Database will optimize the evaluation of such aggregates in some scenarios with window-style computation as used in analytic functions. These scenarios involve rules with multi-cell references on their left side and computing window computations such as moving averages, cumulative sums and so on. Consider the following example:

MODEL PARTITION BY (country) DIMENSION BY (product, time)
  MEASURES (sale sales, 0 mavg)
  RULES UPDATE
  (mavg[product IN ('Bounce', 'Y Box', 'Mouse Pad'), ANY] =
   AVG(sales)[CV(product), time BETWEEN CV(time) 
   AND CV(time) - 2])

It computes the moving average of sales for products Bounce, Y Box, and Mouse Pad over a three year period. It would be very inefficient to evaluate the aggregate by scanning the partition for every cell referenced on the left side. Oracle identifies the computation as being in window-style and evaluates it efficiently. It sorts the input on product, time and then scans the data once to compute the moving average. You can view this rule as an analytic function being applied on the sales data for products Bounce, Y Box, and Mouse Pad:

AVG(sales) OVER (PARTITION BY product ORDER BY time
RANGE BETWEEN 2 PRECEDING AND CURRENT ROW)

This computation style is called WINDOW (IN MODEL) SORT. This style of aggregation is applicable when the rule has a multi-cell reference on its left side with no ORDER BY, has a simple aggregate (SUM, COUNT, MIN, MAX, STDEV, and VAR) on its right side, only one dimension on the right side has a boolean predicate (<, <=, >, >=, BETWEEN), and all other dimensions on the right are qualified with CV.

Using EXPLAIN PLAN to Understand Model Queries

You will see a line in the main explain plan output showing the model and the algorithm used. Reference models are tagged with the keyword REFERENCE in the plan output. Also, Oracle annotates the plan with WINDOW (IN MODEL) SORT if any of the rules qualify for window-style aggregate computation.

By examining an explain plan, you can find out the algorithm chosen to evaluate your model. If your model has SEQUENTIAL ORDER semantics, then ORDERED is displayed. For AUTOMATIC ORDER models, Oracle displays ACYCLIC or CYCLIC based on whether it chooses ACYCLIC or CYCLIC algorithm for evaluation. In addition, the plan output will have an annotation FAST in case of ORDERED and ACYCLIC algorithms if all left side cell references are single cell references and aggregates, if any, on the right side of rules are simple arithmetic non-distinct aggregates like SUM, COUNT, AVG, and so on. Rule evaluation in this case would be highly efficient and hence the annotation FAST. Thus, the output you will see in the explain plan would be MODEL {ORDERED [FAST] | ACYCLIC [FAST] | CYCLIC}.

EXPLAIN PLAN FOR
SELECT country, prod, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (prod, year) MEASURES (sale sales)
  RULES UPSERT
  (sales['Bounce', 2003] = AVG(sales)[ANY, 2002] * 1.24,
   sales['Y Box', 2003] = sales['Bounce', 2003] * 0.25);

EXPLAIN PLAN FOR
SELECT country, prod, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (prod, year) MEASURES (sale sales)
  RULES UPSERT
  (sales['Bounce', 2003] = AVG(sales)[ANY, 2002] * 1.24
   sales[prod <> 'Bounce', 2003] = sales['Bounce', 2003] * 0.25);

EXPLAIN PLAN FOR
SELECT country, prod, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (prod, year) MEASURES (sale sales)
  RULES UPSERT AUTOMATIC ORDER
  (sales['Y Box', 2003] = sales['Bounce', 2003] * 0.25,
   sales['Bounce', 2003] = sales['Bounce', 2002] / SUM(sales)[ANY, 2002] * 2 *   sales['All Products', 2003],
   sales['All Products', 2003] = 200000);

SELECT country, prod, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (prod, year) MEASURES (sale sales)
  RULES UPSERT AUTOMATIC ORDER
  (sales['Y Box', 2003] = sales['Bounce', 2003] * 0.25,
   sales['Bounce',2003] = PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY
     sales)[ANY,2002] / SUM(sales)[ANY, 2002] * 2 * sales['All Products', 2003],
   sales['All Products', 2003] = 200000);

EXPLAIN PLAN FOR
SELECT country, prod, year, sales
FROM sales_view
WHERE country IN ('Italy', 'Japan')
MODEL UNIQUE DIMENSION
  PARTITION BY (country) DIMENSION BY (prod, year) MEASURES (sale sales)
  IGNORE NAV RULES UPSERT AUTOMATIC ORDER
  (sales['All Products', 2003] = 200000,
   sales['Y Box', 2003] = sales['Bounce', 2003] * 0.25,
   sales['Bounce', 2003] = sales['Y Box', 2003] + 
    (sales['Bounce', 2002] / SUM(sales)[ANY, 2002] * 2 * 
     sales['All Products', 2003]));