| Oracle® Data Mining Concepts 11g Release 1 (11.1) Part Number B28129-01 |
|
|
View PDF |
| Oracle® Data Mining Concepts 11g Release 1 (11.1) Part Number B28129-01 |
|
|
View PDF |
This chapter describes Naive Bayes, one of the classification algorithms supported by Oracle Data Mining.
This chapter contains the following topics:
The Naive Bayes algorithm is based on conditional probabilities. It uses Bayes' Theorem, a formula that calculates a probability by counting the frequency of values and combinations of values in the historical data.
Bayes' Theorem finds the probability of an event occurring given the probability of another event that has already occurred. If B represents the dependent event and A represents the prior event, Bayes' theorem can be stated as follows.
Bayes' Theorem:
Prob(B given A) = Prob(A and B)/Prob(A)To calculate the probability of B given A, the algorithm counts the number of cases where A and B occur together and divides it by the number of cases where A occurs alone.
Example 15-1 Use Bayes' Theorem to Predict an Increase in Spending
Suppose you want to determine the likelihood that a customer under 21 will increase spending. In this case, the prior condition (A) would be Òunder 21,Ó and the dependent condition (B) would be Òincrease spending."
If there are 100 customers in the training data and 25 of them are customers under 21 who have increased spending, then:
Prob(A and B) = 25%
If 75 of the 100 customers are under 21, then:
Prob(A) = 75%
Bayes' Theorem would predict that 33% of customers under 21 are likely to increase spending (25/75).
The cases where both conditions occur together are referred to as pairwise. In Example 15-1, 25% of all cases are pairwise.
The cases where only the prior event occurs are referred to as singleton. In Example 15-1, 75% of all cases are singleton.
A visual representation of the conditional relationships used in Bayes' Theorem is shown in Figure 15-1.
Figure 15-1 Conditional Probabilities in Bayes' Theorem
For purposes of illustration, Example 15-1 and Figure 15-1 show a dependent event based on a single independent event. In reality, the Naive Bayes algorithm must usually take many independent events into account. In Example 15-1, factors such as income, education, gender, and store location might be considered in addition to age.
Naive Bayes makes the assumption that each predictor is conditionally independent of the others. For a given target value, the distribution of each predictor is independent of the other predictors. In practice, this assumption of independence, even when violated, does not degrade the model's predictive accuracy significantly, and makes the difference between a fast, computationally feasible algorithm and an intractable one.
Sometimes the distribution of a given predictor is clearly not representative of the larger population. For example, there might be only a few customers under 21 in the training data, but in fact there are many customers in this age group in the wider customer base. To compensate for this, you can specify prior probabilities when training the model. See Priors .
The Naive Bayes algorithm affords fast, highly scalable model building and scoring. It scales linearly with the number of predictors and rows. The build process for Naive Bayes is parallelized. (Scoring can be parallelized irrespective of the algorithm.)
Naive Bayes can be used for both binary and multiclass classification problems.
Naive Bayes calculates a probability by dividing the percentage of pairwise occurrences by the percentage of singleton occurrences. If these percentages are very small for a given predictor, they probably will not contribute to the effectiveness of the model. Occurrences below a certain threshold can usually be ignored.
Two build settings are available for adjusting the probability thresholds: NABS_PAIRWISE_THRESHOLD and NABS_SINGLETON_THRESHOLD. The default thresholds work well for most models, so you will not generally need to adjust them.
NABS_PAIRWISE_THRESHOLD —The minimum percentage of pairwise occurrences required for including a predictor in the model.
NABS_SINGLETON_THRESHOLD — The minimum percentage of singleton occurrences required for including a predictor in the model.
See:
Oracle Database PL/SQL Packages and Types Reference for details about algorithm settings for Naive Bayes.Naive Bayes relies on counting techniques to calculate probabilities. Columns should be binned to reduce the cardinality as appropriate.
Numerical data can be binned into ranges of values (for example, low, medium, and high), and categorical data can be binned into meta-classes (for example, regions instead of cities).
You can use the Automatic Data Preparation (ADP) feature of Oracle Data Mining to perform the binning. ADP uses a technique called supervised binning for Naive Bayes models. This technique uses decision trees to create the optimal bin boundaries.
When NB uses equi-width binning, outliers cause most of the data to concentrate in a few bins, sometimes a single bin. As a result, the discriminating power of these algorithms can be significantly reduced.
