Professional Documents
Culture Documents
www.vosesoftware.com
1|Page
Introduction
This technical review compares the three most common risk analysis Excel add-in software products:
Product
Crystal Ball:
ModelRisk:
@RISK:
Summary of conclusions
The best product choice depends on the factors you consider most important:
Approach
The primary data source for this review is this1 independent and up-to-date overview of Excel risk
analysis add-ins on Wikipedia. This review only includes the Wikipedia information where there is a
difference between the software products. Aside from which technical capabilities each product has,
user-experience is also extremely important and a strong measure of the quality of the software as a
whole so, where possible, links have been provided to official videos produced by each developer so you
can compare the approaches. Please do take a look at them. This review is split into the following
sections:
Functionality
Technical specifications
Simulation controls
Reporting results
Help file, technical support and training
Implementation within Excel
Precision of the algorithms used in the software
Pricing
www.vosesoftware.com
2|Page
Functionality
The following chart is plotted from the Wikipedia article which gave a numerical summary of the
number of features each product offered2
The three types of random variables that you tend to see in risk analysis modeling are:
Distributions
Correlation structures
Time series
Distributions - there are something like 150 distribution types that have a practical use and are not
simply a rehashed or renamed version of another one. You could say that 90% of users only need to use
10% of the types of distribution that are known, but which 10% varies enormously between the types of
problems that the users focus on. You can view videos of how the different products fit distributions to
data here: Crystal Ball3, ModelRisk, @RISK.
Correlation structures (copulas) are a very important and much overlooked component of risk modeling.
Failure to incorporate appropriate correlation will usually result in an underestimation of the risk. All of
the products reviewed effectively use a Gaussian copula which allows simulating a correlation structure
between any number of variables4. ModelRisk also provides the Clayton, Gumbel, Frank and Student
The table was removed from general view because of Wikipedias policy of not including original research (you can track the discussion that
resulted in its removal on this page).
3
Official video not found
4
In fact, @RISK and Crystal Ball uses rank order correlation, a simulation technique from 1982 that closely resembles a Gaussian copula. See
Iman, R. L. and Conover, W. J., (1982). 'A Distribution-Free Approach to Inducing Rank Order Correlation Among Input Variables', Commun
Statist-Simula Computa 11(3) 311-334.
www.vosesoftware.com
3|Page
copulas. Bivariate versions can be rotated and all can be statistically fit to data. ModelRisk also offers an
empirical copula that will statistically reproduce unusual data patterns across any number of variables.
Correlating two Uniform distributions using the Gaussian copula with, for example, a 90% correlation
would produce a scatter pattern that looks like this:
But the different copulas will produce markedly different patterns for the same 90% correlation, for
example (reading from the top left: Clayton, inverted Clayton, Frank, Gumbel, inverted Gumbel,
Student):
For example, the Clayton copula (top left plot) is often used by banks for loan defaults because these are
strongly correlated in times of recession, but not in better times. The selection of correlation shape can
have a very significant effect on the tails of results when the correlation level is moderate (e.g. 10% to
50%). You can view the way ModelRisk fits copulas to data here, and for unusual patterns here.
Time series models are used to forecast variables across several periods, like exchange rates, inflation,
sales volumes. They incorporate at a minimum the trend and the level of randomness around that trend.
More sophisticated models incorporate characteristics like sudden jumps, cycles of high and low periods
of volatility, etc. The more sophisticated models are usually fit to data using software. You can view
videos of how the different products fit time series to data here: Crystal Ball Suite, ModelRisk, @RISK.
www.vosesoftware.com
4|Page
Technical specifications
This section compares the key differences in the modeling tools that each product offers.
Technical specification
Distribution fitting
Correlation fitting
Time series fitting
Optimizer included
Database connectivity
VBA calls to functions
C++ calls to functions
Six sigma supported
UDF error analysis
Extreme value modeling
Expert elicitation tools
Data previsualizer
ODE and numerical integration
Assumption library
Converters for
@RISK
@RISK
@RISK
Standard Professional Industrial
No
No
No
No
No
No
No
Yes
No
No
Yes
No
No
No
CB
Yes
No
No
No
No
Yes
Yes
Yes
No
No
Yes
No
No
Yes
CB
Yes
No
Yes
Yes
No
Yes
Yes
Yes
No
No
Yes
No
No
Yes
CB
Crystal
Ball
Yes
No
Yes
No
No
Yes
No
Yes
No
No
No
No
No
No
No
Yes
No
Yes
Yes
No
Yes
No
Yes
No
No
No
No
No
No
No
No
No
No
No
No
Yes
Yes
No
Yes
No
No
No
No
No
CB, @RISK
Yes
Yes
Yes
Yes
No
Yes
Yes
No
Yes
Yes
Yes
Yes
No
No
CB, @RISK
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
CB, @RISK
Source: Wikipedia
Notes
Distribution fitting a fundamental requirement unless your risk analysis model is only based on subjective estimates of uncertainty
Correlation fitting another fundamental requirement unless your risk analysis model is only based on subjective estimates of
uncertainty, or all your variables are unconnected (independent)
Time series fitting a fundamental requirement if you want to make forecasts based on historic data
Optimizer allows you to determine the optimal values of variables you control based on some target(s) like maximizing the mean
profit. All three products use the industry leader OptQuest optimizer, and @RISK includes other optimizers
Database connectivity useful if you have databases with large datasets that you want to fit distributions, correlation structures or
time series and that will update as new data are added to the database
VBA calls useful if you want to automatically run simulations, or functions that will internally perform simulations or probability
calculations, etc.
C++ calls for very advanced modelers who want to build their own specialist, high performance simulation programs
Six Sigma functionality a moderately useful feature for Six Sigma specialists that analyzes simulation results using Six Sigma
metrics
Probability calculations a very useful capability for advanced probability modelers because, amongst other reasons, one can
combine calculation and simulation together to create more precise and faster models.
User Defined Function error analysis an error message appears in the spreadsheet cell. For example:
Extreme vale modeling very useful in certain circumstances - many risk analysis issues are driven by extremes, like predicting the
size of the largest wave or wind gust, the lowest temperature, the largest insurance claim, the longest blackout, etc.
Expert elicitation tools to help the modeler get the most precise uncertain estimates from subject matter experts. If this is useful
to you, you should investigate each tools capabilities because there is a great deal of variation in what they offer
Data previsualizer allows you to review and explore the data before you start fitting distributions, time series, or correlations.
Again, a large variation in what is offered, so worth investigating the differences if this is important
Differentiation and integration a very advanced capability for either (i) building a system of ordinary differential equations with
uncertain inputs; or (ii) performing one-dimensional numerical integrations. Useful for scientists, engineers and finance/insurance
people mostly
www.vosesoftware.com
5|Page
Assumption library allows the user to store key assumptions that are used in various models. Again, a very wide range of
capabilities offered, so worth investigating the differences if this is important
Converter used if one changes to another software product, and wishes to keep using models developed in the old product (CB =
Crystal Ball)
Simulation controls
This section compares the controls that are available to run simulations.
@RISK
@RISK
@RISK Crystal ModelRisk ModelRisk ModelRisk
Standard Professional Industrial Ball
Standard Professional Industrial
Simulation controls
Lock / unlock random variables
Apply specific sample in model
Random number generator
Spreadsheet interpreter
Sampling method
Yes
Yes
Several
No
MC, LHS
Yes
Yes
Several
No
MC, LHS
Yes
No
Yes
No
Several
MCG
No
Yes
MC, LHS MC, LHS
Yes
Yes
Twister
No
MC
Yes
Yes
Twister
No
MC
Yes
Yes
Twister
No
MC
Source: Wikipedia
Notes:
Lock/unlock sampling switches off the Monte Carlo simulation, replacing random values with a predictable value like the median.
Useful in model auditing as one can compare values between different versions of a model
Apply sample to model allows the user to review the results data and force Excel to show a particular selected sample. This lets
you check, for example, why a particularly high or peculiar value was produced by the model. It also demonstrates to an auditor that
specific scenarios appear within the results, which is especially useful for banks and insurance companies
Random number generator advances in random number generators have made this essentially irrelevant. Nearly every product
uses the Mersenne Twister by default.
Spreadsheet interpreter this creates a compiled version of the Excel model and then runs this instead of within Excel. The result is
a much faster simulation time. Mostly useful if one has a large model that needs optimizing. The drawback is that not all Excel
models can be compiled.
Sampling method MCS = Monte Carlo simulation, LHS = Latin Hypercube sampling. We describe in a LinkedIn blog why LHS is no
longer important here.
Reporting results
Risk analysis is a decision-making tool. That means that it is vital to be able to share the results of a risk
analysis effectively. The simulation results from a risk analysis model can be complex for a decisionmaker to understand, so a wide variety of customizable graphical reports is essential.
Reporting results
@RISK
@RISK
@RISK Crystal ModelRisk ModelRisk ModelRisk
Standard Professional Industrial Ball Standard Professional Industrial
No
Yes
Yes
No
Yes
Yes
No
Yes
Yes
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
The greatest difference between the products is how they organize the storage and presentation of
results, and particularly how these results can be shared with people who are not licensed users of their
products, which is explained below.
www.vosesoftware.com
6|Page
Crystal Ball
Crystal Ball opens a new window for each graph after clicking a ribbon button. Examples:
Results files can only be opened by another Crystal Ball user, though the charts can be copied as bitmaps
into Word, PowerPoint, etc.
ModelRisk
Results are shown in a separate application called the Results Viewer which opens at the end of the
simulation. The user can create and save an electronic report and that file can then be distributed to
others, without the need to share the model itself. The file can be opened by anyone who has installed
the ModelRisk Results Viewer, which is available free of charge from Vose Softwares web site here.
Aside from saving on licenses, the main advantages of this approach is that the report is electronic and
completely customizable labels and color schemes can be edited by the reviewer, new graphs can be
created, plots can be interrogated for say the 95th percentile instead of the 90th, and the whole
report can be resaved if desired. Plots can also be copied into Word, PowerPoint and other applications.
www.vosesoftware.com
7|Page
@RISK
@RISK opens a new window for each graph after clicking a ribbon button. Examples:
Results files can only be opened by another @RISK user, though the charts can be copied as bitmaps into
Word, PowerPoint, etc.
@RISK
Standard
@RISK
Professional
@RISK
Industrial
Crystal
Ball
Yes
Yes
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
No
Yes
Yes
Yes
No
ZH, EN, FR, ZH, EN, FR, DE, ZH, EN, FR,
EN, JA, ES
DE, JA, PT, ES
JA, PT, ES
DE, JA, PT, ES
www.vosesoftware.com
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
EN
EN
EN
8|Page
Notes
UDFs linked to help file this means that when you open a dialog for a function, there is a link that will take you directly to the
relevant help file topic. Excel makes this quite easy for developers to do, and automatically adds a hyperlink to the dialog box, so it is
hard to see why this feature should not be implemented
=VoseTriangle(0,40,100)
=RiskTriang(0,40,100)
The use of UDFs makes it possible to trace through the logic of the model using Excels Formula Auditing
tools as illustrated below:
www.vosesoftware.com
9|Page
A fundamental Excel rule is that there is a hierarchy of calculation in any formula. For example, in this
formula:
=SUM(SQRT(A1), A2)
Excel will evaluate A1 and A2, then evaluate SQRT(A1), and finally evaluate the SUM.
These rules are critical to having a predictable calculation outcome. However, this presents problems for
developers in creating functions that perform some operation on a distribution. For example, consider
the following @RISK function:
=RiskCompound(5, RiskLognorm(10,1))
The function sums 5 independent samples from a Lognormal(10,1) distribution. However, if we follow
Excels rules, the RiskLognorm function which is a random sampling function - must be evaluated
before the RiskCompound function. So, wed get something like this as an intermediary calculation:
=RiskCompound(5, 1.032)
in which case the RiskCompound would just be adding five 1.032 values together. To avoid this, @RISK
suppresses the evaluation of the RiskLognorm function. This causes problems of predicting how the
function behaves. For example, it is not immediately evident how the RiskCompound function should
behave in these circumstances:
=RiskCompound(5, RiskUniform(RiskUniform(2,3),4))
=RiskCompound(5, A2), where A2: =RiskNormal(A1,2)
ModelRisks approach is to use different functions when defining a distribution, called Objects, not the
same function used for sampling. Their equivalent function works as follows:
= VoseAggregateMC(5, VoseLognormalObject(10,1))
The LognormalObject function is used to define the distribution we are intending to use. Object
functions allow ModelRisk to comply with Excels evaluation rules. They are used in a variety of
ModelRisks tools.
www.vosesoftware.com
10 | P a g e
Pricing
All software vendors offer different editions of their products. In the following table, the products are
sorted according to price in US$ for a single perpetual (i.e. not time limited) license including the first
years maintenance and support. These prices appear on the following web sites, accessed on 10 July
2014: Crystal Ball5, ModelRisk6, @RISK7.
Price comparison
Perpetual license + 1st year support
Cost of additional year's support
# CPUs supported
Single click function view
Distribution fitting
Links to Microsoft Project
Aggregate modeling
Markov Chain simulation
Multivariate time series
Dynamic sorting
Optimizer
Stochastic data sharing
Specialist financial tools
Specialist insurance tools
Bayesian model averaging
ModelRisk
Standard
@RISK
Standard
Crystal Ball
@RISK
ModelRisk
Professional Professional
$895
$161
$1,195
?
$1,213
$218
$1,595
?
All
Yes
No
No
No
No
No
No
No
No
No
No
No
2
No
No
No
No
No
No
No
No
No
No
No
No
?
No
Yes
No
No
No
No
No
No
No
No
No
No
2
No
Yes
Yes
Yes
No
No
No
No
No
No
No
No
@RISK
Industrial
ModelRisk
Industrial
Crystal Ball
Suite
$1,645
$296
$2,195
?
$2,295
$413
$2,433
$438
All
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
No
No
No
No
All
No
Yes
Yes
Yes
No
No
No
Yes
No
No
No
No
All
Yes
Yes
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
?
No
Yes
No
No
No
No
No
Yes
No
No
No
No
Notes
Some additional factors have been included that are not already described in previous sections. The choice is somewhat subjective
but focused on features useful to a modeler.
https://shop.oracle.com/pls/ostore/product?p1=oraclecrystalball&p2=&p3=&p4=&sc=ocom_crystalball
www.vosesoftware.com/purchasing.php
7
www.palisade.com
6
www.vosesoftware.com
11 | P a g e