On The Application of A Multivariate Metric For Multiple Set Points in Validation Exercises of RANS Solvers

May 16 - 20, 2016
On the Application of a Multivariate

Metric for Multiple Set Points in
Validation Exercises of
RANS solvers
L. Eça (IST-Lisbon, MARIN Academy)

R. Lopes (IST-Lisbon)
H.Abreu (IST-Lisbon)
F.Pereira (MARIN Academy, IST-Lisbon)
G.Vaz (MARIN)
May 16 - 20, 2016

May 16 - 20, 2016
Contents
1. Introduction
2. ASME V&V 20 Validation Metrics
3. Multivariate Metric
4. Examples
5. Conclusions
May 16 - 20, 2016

May 16 - 20, 2016
1. Introduction
• Validation exercises of mathematical models
solved by CFD are often performed for
functional quantitions as for example force
coefficients or local quantities
• For such cases, ASME V&V 20 proposed a
Validation procedure that estimates an interval
that contains the modelling error 95 out of 100
times
May 16 - 20, 2016

May 16 - 20, 2016
1. Introduction
• There are several cases where modelling errors
are evaluated at multiple set points, as for
example lift coefficient of an airfoil for a range of
angles of attack, airfoil surface pressure
distribution or velocity field in the wake of a ship
• For such cases, ASME V&V 20 is proposing
a “Multivariate Metric” using results from
Multiple Validation Set Points
May 16 - 20, 2016

May 16 - 20, 2016
1. Introduction
• This presentation includes several examples of
application of the Multivariate Metric
- Flow over a flat plate
- Flow around the NACA 0012 airfoil
- Flow around the Eppler 387 airfoil
- Flow around a circular cylinder
- Flow around the KVLCC2 tanker
• All calculations are performed with

(www.refresco.org)
May 16 - 20, 2016

May 16 - 20, 2016
2. ASME V&V 20 Validation Metric

• ASME V&V 20-2009 Validation procedure is
based on the comparison of two quantities
- Comparison error E =S−D
- Validation uncertainty U val = U D2 + U input
2 2
+ U num
UD Experimental uncertainty
U num Numerical uncertainty
U input Parameter uncertainty
May 16 - 20, 2016

May 16 - 20, 2016
2. ASME V&V 20 Validation Metric

• Estimated interval that contains the modelling
errorδ model is given by
E − U val ≤ δ model ≤ E + U val
• It is not a pass/fail method! For example,
E = 2% 
 ⇒ 1% ≤ δ model ≤ 3%
U val = 1%
E = 4% 
 ⇒ −6% ≤ δ model ≤ 14%
U val = 10%
May 16 - 20, 2016

May 16 - 20, 2016
T −1
• Vector of weighted differences r = r Vval r
based on the vector of differences at
n set points
 S1 − D1 
r= 
 
 Sn − Dn 
and covariance matrix
Vval = VD + Vinput + Vnum
May 16 - 20, 2016

May 16 - 20, 2016
• Covariance matrix depends on numerical
uncertainties U num
2
0 ... 0 
 1

Vnum = 0 ...U numi ...0 
2
 
0 ...
2
U numn  U D21 0 ... 0 
 
experimental uncertainties VD = 0 ...U Di ...0 
2
 2

0 ... U Dn 
and parameter uncertainties that are assumed

to be zero in the present exercises Vinput ≅ 0
May 16 - 20, 2016

May 16 - 20, 2016
• If the uncertainties in the n estimated δ model are
represented by normal distributions, the
2
expected value of r is obtained from a χ
distribution that leads to a reference value
given by rref = n + 2n
• If r / rref > 1 , the modelling errors are in general
larger than the validation uncertainty
• Otherwise, modelling errors are globally within
the Validation uncertainty
May 16 - 20, 2016
May 16 - 20, 2016
4. Flow over a Flat Plate

• Reynolds-Averaged Navier-Stokes (RANS)
equations (time-averaging)
• Three versions of k-ω eddy-viscosity model with
and without transition modelling (SST, W98,
W06)
• Quantity of interest is the skin friction coefficient
Cf τw
Cf =
1 2 ρV∞2
May 16 - 20, 2016

May 16 - 20, 2016

• Experimental data from the ERCOFTAC Classic
Database (T3A, T3AM, T3B) and the Stanford
Conference 1968
• 2% uncertainty assumed from variability in the
different techniques used to measure
• Experimental data for the inlet turbulence
intensity for the ERCOFTAC cases and forced
transition for the Stanford Conference data
May 16 - 20, 2016

May 16 - 20, 2016

• Reynolds number based on plate length L and
incoming velocity V∞ equal to Re=107
C p =0
y ∂φ
1.5L =0
∂x
U x =1
U y =0 0 .25L
k = ki
ε = εi x
0 . 25L L 0 .25L
May 16 - 20, 2016

May 16 - 20, 2016

• 9 geometrically similar grids ranging from
1537×193 to 385×49 (1≤hi/h1≤4)
0.2
y/L
L
0.1
0 0 0.5 1
x/L
• Calculations converged to maximum normalized
residual of all transport equations below 10-7
May 16 - 20, 2016

May 16 - 20, 2016

8 Wieghardt SST W98 W06T
T3AM, I=0.874%
7 T3A, I=3.3%
T3B, I= 6.5%
6
5
3
10
Cf×1
4
3
2
1
0 4 5 6 7
10 10 10 10
Rex
May 16 - 20, 2016

May 16 - 20, 2016

SST W06
150 Wieghardt
T3AM, I=0.874%
T3A, I=3.3%
T3B, I=6.5%
Cf)ref)%
W98
100
E(Cf / (C
50
4 5 6 7
10 10 10 10
Rex
May 16 - 20, 2016

May 16 - 20, 2016

100 SST
W98
W06
80
60
r/rreef
40
20
0
Wieghardt T3AM T3A T3B
May 16 - 20, 2016

May 16 - 20, 2016

20
SST
Wieghardt data W98
W06
10
Cf)ref)%
E(Cf / (C
-10
-20 5 6 7
10 10 10
Rex
May 16 - 20, 2016

May 16 - 20, 2016

8 8
T3AM, I=0.874% 8 T3AM, I=0.874% T3AM, I=0.874%
T3A, I=3.3% T3A, I=3.3% T3A, I=3.3%
7 T3B, I=6.5% 7 T3B, I=6.5% 7 T3B, I=6.5%
νti= 0.1ν νti= ν νti= 10ν νti= 100ν νti= 0.1ν νti= ν νti= 10ν νti= 100ν νti= 0.1ν νti= ν νti= 10ν νti= 100ν
6 6 6
5 5 5
3
3
3
xti1
Cf×10
Cf×10
Cf×10
4 4 4
3 3 3
2 2 2
LCTM W98T W06T
1 1 1
0 0 0
104 105 106 107 104 105 106 107 104 105 106 107
Rex Rex Rex
8 T3AM, I=0.874% 8 T3AM, I=0.874% 8 T3AM, I=0.874%
T3A, I=3.3% T3A, I=3.3% T3A, I=3.3%
7 T3B, I=6.5% 7 T3B, I=6.5% 7 T3B, I=6.5%
6 6 6
5 5 5
3
3
xti 2
Cf×10
Cf×10
Cf×10
4 4 4
3 3 3
2 2 2
LCTM W98T W06T
1 1 1
0 0 0
104 105 106 107 104 105 106 107 104 105 106 107
Rex Rex Rex
8 T3AM, I=0.874% 8 T3AM, I=0.874% 8 T3AM, I=0.874%

T3A, I=3.3% T3A, I=3.3% T3A, I=3.3%
7 T3B, I=6.5% 7 T3B, I=6.5% 7 T3B, I=6.5%
6 6 6
xti 3 5 5 5
3
3
Cf×10
Cf×10
Cf×10
4 4 4
3 3 3
2 2 2
LCTM W98T W06T
1 1 1
0 0 0
104 105 106 107 104 105 106 107 104 105 106 107
Rex Rex Rex
SST+LCTM W98T W06T

May 16 - 20, 2016
May 16 - 20, 2016

100
SST+LCTM
W98T
W06T
80
60
r/rreef
40
20
0
T3AM T3A T3B
May 16 - 20, 2016

May 16 - 20, 2016

8 T3AM, I=0.874%
T3A, I=3.3%
7 T3B, I= 6.5%
LCTM W98T W06T
6
5
3
10
Cf×1
4
3
2
1
0 4 5 6 7
10 10 10 10
Rex
May 16 - 20, 2016

May 16 - 20, 2016

SST W06
150 T3AM, I=0.874%
T3A, I=3.3%
T3B, I=6.5%
W98
Cf)ref)%
100
E(Cf / (C
50
4 5 6 7
10 10 10 10
Rex
May 16 - 20, 2016

May 16 - 20, 2016
4. Flow around the NACA 0012

transition modelling (SST+LCTM, W98T, W06T)
• Quantities of interest are lift C L and C D drag
coefficients and surface pressure distribution C p
at 0, 2, 4, 6 and 8 degrees of angle of attack α
May 16 - 20, 2016

May 16 - 20, 2016

• Experimental data from two different sources,
but only one used to determine E
• Estimated uncertainties from the information
available in the reports: 1% for C L; 3% for C D and
10% for surface pressure coefficient C p
• Inlet oturbulence conditions tuned to match C Dat
α =0
• Two variants of previous conditions tested
May 16 - 20, 2016

May 16 - 20, 2016

• Reynolds number based on chord c and
incoming velocity V∞ equal to Re=2.88×106
Pressure imposed at one point of the inlet boundary Free Slip
10
5 Streamwise
Everything derivatives
y/c
specified 0 equal to
except zero
pressure -5
-10
-10 -5 0 5 10 15 20
x/c
Free Slip
May 16 - 20, 2016

May 16 - 20, 2016

232320 to 929280 or 90750 to 363000 cells
(1≤hi/h1≤2)
• Calculations converged
to maximum normalized
residual of all transport
equations (including
transition model LCTM variables) below 5×10-6
May 16 - 20, 2016

May 16 - 20, 2016

14 Exp
13 SST+LCTM
W98T
12 W06T
3
11
CD×100
10
9
8
7
6
0 2 4o 6 8
α
May 16 - 20, 2016

May 16 - 20, 2016

1
Exp
SST+LCTM
0.8
W98T
W06T
0.6
CL
0.4
0.2
0
0 2 4o 6 8
α
May 16 - 20, 2016

May 16 - 20, 2016

0.8
α = 2o
0.6 Exp
SST+LCTM
W98T
0.4 W06T
-Cp
0.2
-0.2
0 0.2 0.4 0.6 0.8 1

x/c
May 16 - 20, 2016

May 16 - 20, 2016

4
α = 8o
Exp
3 SST+LCTM
W98T
W06T
2
-Cp
0 0.2 0.4 0.6 0.8 1

x/c
May 16 - 20, 2016

May 16 - 20, 2016

8
7 SST+LCTM
W98T
W06T
6
5
r/rreef
0
CD CL α = 0o α = 2o α = 4 o α = 6 o α = 8o
May 16 - 20, 2016

May 16 - 20, 2016
4. Flow around the Eppler 387

transition modelling (SST+LCTM, W98T, W06T)
• Quantities of interest are lift C L and C D drag
coefficients and surface pressure distribution C p
at -2, -1, 0, 1, 2, 3, 4, 6 and 7 degrees of angle
of attack α
May 16 - 20, 2016

May 16 - 20, 2016

• Experimental data from two different sources,
but only one used to determine E
• Estimated uncertainties from the information
available in the 2 reports are very different:
2% for C L ; 0.12% for C D and 0.25% for C p
5.5% for C L and 5.73% for C p
• Two variants of inlet turbulence conditions
tested
May 16 - 20, 2016

May 16 - 20, 2016

• Reynolds number based on chord c and
incoming velocity V∞ equal to Re=3×105
Pressure imposed at one point of the inlet boundary
Free Slip
10
5 Streamwise
Everything derivatives
y/c
specified 0 equal to
except zero
pressure -5
-10
-10 -5 0 5 10 15 20
x/c
Free Slip
May 16 - 20, 2016

May 16 - 20, 2016

232320 to 929280 or 90750 to 363000 cells
(1≤hi/h1≤2)
equations (including
transition model LCTM variables) below 5×10-6
May 16 - 20, 2016

May 16 - 20, 2016

16 Exp
SST+LCTM
14 W98T
W06T
3
CD×100
12
10
6
-2 0 2 o 4 6 8
α
May 16 - 20, 2016

May 16 - 20, 2016

1.2
0.8
CL
0.6 Exp
Exp
SST+LCTM
0.4
W98T
W06T
0.2
0
-2 0 2 o 4 6 8
α
May 16 - 20, 2016

May 16 - 20, 2016

1
α = 0o Exp
Exp
SST+LCTM
W98T
0.5 W06T
-Cp
-0.5
-1
0 0.2 0.4 0.6 0.8 1
x/c
May 16 - 20, 2016

May 16 - 20, 2016

1
α = 3o Exp
Exp
SST+LCTM
W98T
0.5 W06T
-Cp
-0.5
-1
0 0.2 0.4 0.6 0.8 1
x/c
May 16 - 20, 2016

May 16 - 20, 2016

SST+LCTM
W98T
40 α = 0o W06T
α = 2o
r/rreef
20 α = 4o
α = 7o
0
CD C L α = −1o α = 1o α = 3o α = 6o
May 16 - 20, 2016

May 16 - 20, 2016
4. Flow around a circular cylinder

equations (ensemble-averaging)
• Three two-equation eddy-viscosity models
tested (SST, KSKL, W98T)
• Quantities of interest are the average drag
coefficient(C D )avg and the average pressure C p and
skin friction C f coefficients on the cylinder surface
May 16 - 20, 2016

May 16 - 20, 2016

• Experimental data taken from Achenback E.,
“Distribution of Local Pressure and Skin Friction
around a Circular Cylinder in Cross-flow up to
Re = 5×106”, Journal of Fluid Mechanics, Vol.
34,No 64,1968, pp. 625-639”
• Estimated uncertainties from symbols size and
asymmetry of the solution
May 16 - 20, 2016

May 16 - 20, 2016

• Reynolds numbers based on the diameter D
and incoming velocity V∞ equal to 105, 2.6×105,
8.5×105 and 3.6×106
Pressure imposed at one point of the inlet boundary Free Slip
Everything Streamwise
specified derivatives
except equal to
pressure zero
Free Slip
May 16 - 20, 2016

May 16 - 20, 2016

8.5×104 to 2.5×105 cells
(1≤hi/h1≤1.71)
equations at each time
step below 10-6
May 16 - 20, 2016

May 16 - 20, 2016
May 16 - 20, 2016

May 16 - 20, 2016
May 16 - 20, 2016

May 16 - 20, 2016
May 16 - 20, 2016

May 16 - 20, 2016

8
SST
3.6 ×106
7 KSKL
W98 2×105
6
5 105 8.5 × 105

105
r/rreef
3
2× 1058.5 × 105
2
1 3.6 ×10 6
0
(CD )avg C p Cp Cp Cp Cf Cf Cf Cf
May 16 - 20, 2016

May 16 - 20, 2016
4. Flow around the KVLCC2 tanker

• 14 turbulence models tested including one and
two-equation eddy-viscosity models and Explicit
Algebraic Reynolds-Stress models
• Quantities of interest are the resistance
coefficient C T and the flow field at the propeller
plane
May 16 - 20, 2016

May 16 - 20, 2016

• Experimental data from towing tank and wind
tunnel tests (different boundary conditions at
free surface and different blockage)
• Estimated uncertainties reported in the papers
May 16 - 20, 2016

May 16 - 20, 2016

• Reynolds number based on ship length LPP and
incoming velocity V∞ equal to Re=4.6×106
Symmetry
Pressure imposed
Everything
specified
except
pressure
Everything
specified
except
pressure
Free Slip
May 16 - 20, 2016

May 16 - 20, 2016

• 5 nearly geometrically similar grids ranging from
5×106 to 2×107 cells (1≤hi/h1≤2)
equations below 10-8
for all turbulence models
tested
May 16 - 20, 2016

May 16 - 20, 2016
Flow field at the propeller

plane
May 16 - 20, 2016
May 16 - 20, 2016
Flow field at the propeller plane
May 16 - 20, 2016

May 16 - 20, 2016
May 16 - 20, 2016

May 16 - 20, 2016
May 16 - 20, 2016

May 16 - 20, 2016
4. Conclusions
• The multivariate metric proposed by the ASME
V&V 20 is a useful tool to assess the modelling
accuracy of multiple set points
• It has a direct application in the comparison of
the modelling accuracy of different turbulence
models
• It should be “handled with care”, i.e. as for the
V&V 20 2009 procedure, it must be analysed
taking into account the level of Validation
uncertainty
May 16 - 20, 2016

On The Application of A Multivariate Metric For Multiple Set Points in Validation Exercises of RANS Solvers

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

On The Application of A Multivariate Metric For Multiple Set Points in Validation Exercises of RANS Solvers

Uploaded by

Copyright:

Available Formats

May 16 - 20, 2016