Environmental Modelling & Software 74 (2015) 173e182

Contents lists available at ScienceDirect

Environmental Modelling & Software

journal homepage: www.elsevier.com/locate/envsoft

Establishment and optimization of an evaluation index system for

browneld redevelopment projects: An empirical study
Yuming Zhu a, Keith W. Hipel b, Ginger Y. Ke c, *, Ye Chen d
a

School of Management, Northwestern Polytechnical University, Xi'an, Shaanxi 710072, China

Department of Systems Design Engineering, University of Waterloo, Waterloo, Ontario N2L 3G1, Canada
c
Faculty of Business Administration, Memorial University of Newfoundland, St. John's, Newfoundland and Labrador A1B 3X5, Canada
d
College of Economics and Management, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu 210016, China
b

a r t i c l e i n f o

a b s t r a c t

Article history:
Received 7 May 2015
Received in revised form
21 September 2015
Accepted 21 September 2015
Available online xxx

Browneld redevelopment has recently become the focus of attention of governments, communities,
environmental advocates, scientists, and researchers around the world. The purpose of this study is to
provide a framework for establishing and optimizing an evaluation index for browneld redevelopment
projects (BRPs). This framework involves three steps: the initial design, testing and optimization, and
verication. With the help of two standard statistical software packages, the reliability and validity of the
initialized index system are established, and then the optimization of the initial index system is carried
out by means of Factor Analysis. The effectiveness of the optimization of the index system is veried
through Structural Equation Modeling. Furthermore, an illustration example is used to show how to
apply the established index system in the real world.
2015 Elsevier Ltd. All rights reserved.

Keywords:
Browneld redevelopment project
Evaluation index system
Establishment and optimization
Factor analysis
Structural equation modeling

Software availability

Data availability

Name: SPSS 16.0 and AMOS 8.0.

Developer: IBM Software Group.
Contact address: you can get the local contact information from
the website: http://www-01.ibm.com/software/analytics/spss/.
First available: rst version in 1968 after being developed by
Norman H. Nie and C. Hadlai Hull.
Operation System: Windows 8, Windows 7, Windows Vista
Windows XP, Mac OS X
The latest versions: IBM SPSS Statistics e v22.0 (released on
August 2013).
Add-on: AMOS 22.0 which allows modeling of structural equations and covariance structures, path analysis, and has the more
basic capabilities such as linear regression analysis, ANOVA and
ANCOVA.
Availability: http://www14.software.ibm.com/webapp/downlo
ad/byproduct.jsp?pgelibmhzn1&cm_remasthead-_-supdl-_-dltrials
Cost: 30-day free trial.

Data: collected by questionnaire survey.

Number of questionnaires sent out: 500.
Number of questionnaires returned: 340.
Number of valid questionnaires returned: 335.
Target of questionnaire survey: major stakeholders of browneld redevelopment, which included relevant government divisions, browneld owners, browneld developers, the nancial
sector, public representatives, and others.
Time of survey: from January 2009 to June 2009.
Location of survey: Cities in China: Xi'an, Beijing, Shanghai,
Shenzhen, Jinan.

* Corresponding author.
E-mail address: gke@mun.ca (G.Y. Ke).
http://dx.doi.org/10.1016/j.envsoft.2015.09.012
1364-8152/ 2015 Elsevier Ltd. All rights reserved.

1. Introduction
After decades of effort, the concept of sustainable development
has been accepted and adopted as policy by governments, for-prot
organizations, non-prot organizations, and individuals (WBCSD,
2014; UNDSD, 2009). The World Business Council on Sustainable
Development (WBCSD) (2014) dened sustainable development as
forms of progress that meet the needs of the present without
compromising the ability of future generations to meet their

174

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

needs. Browneld redevelopment continues a concrete application of the concept of sustainable development (Wedding and
Crawford-Brown, 2007). Sardinha et al. (2013) proposed a sustainability redevelopment framework that illustrates how the
integration of different perspectives and approaches can lead to a
locally adapted sustainable development overview that can support
the redevelopment planning of a browneld in a rural setting.
Many countries have given their own denitions of brownelds
according to specic characteristics (Alker et al., 2000). Among
these denitions, the most commonly cited is the one from the US
Environmental Protection Agency (USEPA), which denes brownelds as abandoned, idle, or underutilized commercial or industrial properties where an active potential for redevelopment is
restrained by known or suspected environmental contamination
caused by past actions (USEPA, 2005). Brownelds exist in very
large numbers and pose serious environmental and health risks in
industrialized countries around the globe. For example, the United
States is believed to contain between 500,000 and 1,000,000
browneld sites and Germany about 362,000 (NRTEE, 2003). Canada may have up to 30,000 brownelds, including the sites of
almost-forgotten industrial enterprises such as coal gasication
plants, locations where toxic substances were used or stored, and
former gas stations and mining operations (De Sousa, 2001).
Undoubtedly, restoration and redevelopment of brownelds can
provide a range of economic, social, and environmental benets,
including restoration of environmental quality and improvement of
quality of life for citizens, elimination of health threats, provision of
land for housing or commercial purposes, creation of employment
opportunities, expansion of the tax base for all levels of government, and reduction in the pressure on urban centers to expand
into greenelds (NRTEE, 2003). Thus, brownelds have recently
become the focus of attention for governments, communities,
environmental advocates, scientists, and researchers around the
world. Considerable research has addressed browneld redevelopment issues including development of remediation technologies,
environmental assessment, risk assessment and management,
nancial arrangements, and community and public involvement
(Brebbia, 2006).
Many methodologies from the elds of social science and
management science have been utilized to handle challenging
environmental management problems. Chen et al. (2009) used the
rough-set approach to classify cities facing brownelds redevelopment issues. BenDor et al. (2011) presented a system dynamics
model of the redevelopment process that illustrates how delays
compound before realizing nancial benets from investment in
core urban areas. Chrysochoou et al. (2012) presented an indexing
scheme to screen browneld sites in wide areas which help
develop initial planning strategies for fund allocation and rede berl et al. (2013) developed an assessment method
velopment. Do
based on effectiveness-cost analysis to support decision making in
contaminated site management and to implement the principles
of sustainability into the selection of remediation options in
Austria. Ruelle et al. (2013) used a community investigation
approach to study the relationship of landscape quality and
dler et al. (2011) used an intebrownelds redevelopment. Scha
grated assessment model to design sustainable and economically
dler et al. (2012)
attractive browneld revitalization options. Scha
used a decision support method to build a framework of assessment methods and models which would support an efcient early
judgment about whether and how brownelds could be assigned
dler et al. (2013)
a sustainable and marketable land use. Scha
proved that a spatially explicit algorithmic evaluation of sustainability indicators might signicantly improve the applicability,
comprehensiveness and reliability of the indicator-based evaluation of sustainability with a case study. Among all the

methodologies that have been applied to environmental issues,

the project evaluation method is a systematic method for collecting, analyzing, and using information to answer questions
about projects, policies and programs, particularly about their
effectiveness and efciency (Rossi et al., 2004). Based on a
contextual analysis of the market segments, Guarini and Battisti
(2014) outlined a methodological approach to assess the nancial sustainability of redevelopment projects on browneld sites.
The approach has been tested on a restructuring proposal for the
Corviale (Rome) building-city housing development. All of these
studies are concrete examples of how to use certain evaluation
index systems to assess the value of the BRP, so as to make a sound
decision. However, all of these studies focused on the evaluation
method itself, yet paid very little attention on how to build an
evaluation index system, which is the foundation of any project
evaluation. Our research lls this gap by emphasizing on the
process of establishing and optimizing an evaluation index system
specically for BRPs.
As the rst and main step of project evaluation, the establishment of an evaluation index system (i.e., a set of structured indices
or criteria), plays a very important role and also paves the way for
further project evaluation tasks. Researchers have proposed several
index systems from different perspectives. For example, Syms
(1999) identied six groups of decision-making factors which are
relevant to the redevelopment of browneld land; De Sousa (2000)
proposed an index system based on three aspects: environmental
benets, social benets and economic benets; Wedding and
Crawford-Brown (2007) proposed an index system to assess the
success of redevelopment in meeting sustainability goals, including
multi-stakeholder perspectives, green building elements, and sitelevel details. Despite the previous effort, all existing studies were
qualitative and thus failed to provide objective and conclusive
judgement for subsequent applications. Herein, we propose a
framework of an evaluation index system for BRPs which integrates
both qualitative and quantitative analyses. To the best of our
knowledge, this is the rst time that such an index system is
empirically developed and applied to the browneld redevelopment area. Fig. 1 depicts the structure of this framework, from
which we can see the generality in the process of deriving this
framework. Therefore the corresponding index system can be
applied to most of the BRPs in general, by serving as a basis for
designing further index systems, or as a starting point for various
project evaluation techniques.
The rest of the paper is organized as follows: the initial evaluation index system for BRPs is established in Section 2 (top row in
Fig. 1); the testing and optimization of the index system is given in
Section 3 (the second row in Fig. 1); the effectiveness of the optimized index system is veried in Section 4 (the third row in Fig. 1);
an example is given in Section 5 (bottom row in Fig. 1) to show how
to use the established evaluation index system to assess the proposed redevelopment plans. Finally, conclusions and future
research are presented in Section 6.
2. The initial establishment of the evaluation index system
for browneld redevelopment projects
In this section, we start from the concept of sustainable development (Section 2.1), identify the relevant stakeholder (Section
2.2), and then establish an initial evaluation index system for BRPs
(Section 2.3).
2.1. Evaluating the browneld redevelopment projects from the
perspective of sustainability
After the browneld has been properly remediated and

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

175

Fig. 1. The framework of establishment and optimization of an evaluation index system for BRPs.

redeveloped for useful purposes, the reused land can bring economic, social, and environmental benets. For example, if apartment buildings are constructed on a formal browneld property,
the landlord will collect rent from the residents, the people living in
the apartment will spend money in the community, and the city
where the apartment is located will have a larger tax base and
enhance its reputation with respect to environmental stewardship.
All these activities make the redevelopment sustainable.
A BRP is very different from development projects on clean land.
First, it is critical to remediate the project site and ensure that it
complies with environmental standards in consonance with its
subsequent development. Second, there are special requirements
for the BRP to prevent environment contamination. The environmental index must be considered within an integrated evaluation
process to make sure that all stakeholders of the BRP benet from
the redevelopment, and thus, to achieve the goal of sustainable
development. Traditional evaluation methods emphasize the economic evaluation, which usually uses a nancial index, such as Net
Present Value (NPV), Return on Investment (ROI), or Payback Period
(PP), to measure the project, but rarely take the environmental and
ecological indices into consideration. This omission may signicantly increase the possibility of having more environmental
pollution, ecological deterioration and resources exhaustion.
Therefore, the evaluation of a BRP should not only be viewed from
the economic perspective, but also be integrated with environmental, social, national economic, nancial, and risk evaluation.
Details of the aforementioned evaluation concepts are described as
follows.
(1) Environmental Evaluation refers to the analysis of the BRP
with respect to environmental restoration. The BRP environmental evaluation should focus on the improvement of
soil quality and the increasing percentage of green land.
(2) Social Evaluation. The sustainable development viewpoint
makes people conscious of the importance of harmoniously
developing human projects with nature. The evaluation of
the BRP should take both the economic benets and the goal
of fairness among stakeholders into consideration.
(3) Economic Evaluation refers to evaluating the economic
benets based on the principle of rational resource allocation, which may be indirectly reaped from the BRP. The
economic evaluation of a BRP is mainly concerned with the

inuence of the BRP on the local tax base, employment rates,

and land value of neighborhoods.
(4) Financial Evaluation assesses direct gains from BRP, such as
the rate of return on investment and the investment recovery
period of the BRP.
(5) Risk Evaluation. In the course of implementation, an engineering project will deal with all kinds of the uncertainty.
Risk evaluation is an effective way to reduce uncertainty and
enhance project management. Generally, a BRP has a long life
span and therefore usually faces high uncertainty. So the risk
evaluation is crucial in this case.

2.2. Stakeholder analysis of a BRP

Stakeholder theory constitutes an interpretation of organizational management and business ethics that addresses morals and
values in managing an organization (Freeman, 2003). The Stanford
Research Institute (SRI) rst proposed the concept of stakeholder in
1963 (Freeman, 1984). At that time, researchers from different
areas, such as business administration (Freeman, 1984; Bowie,
1988; Clarkson, 1995), natural resources management (Grimble
and Wellard, 1997; Gass et al., 1997), health policy (Varvasovszky
and Brugha, 2000), and project management (ODA, 1995; PMI,
2000), gave their own denitions of stakeholders according to
their particular characteristics.
In the area of browneld redevelopment, stakeholders are the
involved parties with conicting objectives, which may cause a BRP
to fail. Therefore, it is of great importance to comprehensively
consider each stakeholder's benet, and hence to achieve a preferable result for all the stakeholders. The stakeholders of BRPs
mainly include:
(1) Government bodies. The problems of externalities and information asymmetry in the process of browneld redevelopment cannot be resolved by the market economy system
on its own, so the government's supervision and control are
needed. The government oversees the overall performance of
the BRP. The governmental benets lie in improving the
quality of life, thereby raising the employment rate,
increasing nancial income, and enhancing the government's image.

176

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

(2) Investors and developers. The investor provides capital for

the project, and the developer is the executor of the project.
It is the participation of investors and developers that
changes wasted land into something of value and promotes
the development of the economy and society.
(3) Communities. The communities include the owner of the
browneld and the residents of the urban areas, which
respectively care about the ownership transfer with corresponding compensation and their individual interests such
as health issues. They are both stakeholders in BRPs and the
ultimate consumers of the ecological environment.
(4) Non-Governmental Organizations. Engaging in public
welfare, Non-Governmental Organizations (NGOs) participate in such elds as environment protection, medical health
and education. In a BRP, NGOs are concerned with enhancing
the social, health and environmental benets.

2.3. The initial integrated evaluation index system for browneld

redevelopment projects
In light of the evaluation concepts and relevant stakeholders
from the previous two sections (Sections 2.1 and 2.2), we rst
determine the four dimensions that the index system, i.e., criteria,
can be categorized to.
(1) The Social and Economic dimension was used to evaluate
the effect on economic growth and the vulnerable groups of
the stakeholder, mainly from the perspective of enhancing
the government's image, raising the employment rate,
increasing nancial income, and improving the quality of
life.
(2) The Financial and Accounting dimension was used to
evaluate the cash ow, development potential and protability of the BRP, mainly from the perspective of net
present value, return on investment, payback period, as
well as the total cost of browneld remediation and
construction.
(3) The Environmental and Health dimension was used to
appraise the environmental improvement of the area around
the browneld, mainly from the perspective of enhancing
the soil quality and increasing the green cover percentage,
and so on.
(4) The Prospective value dimension was used to evaluate the
geographical position of the browneld, as well as the difculties in the remediation of the redevelopment and the
nancial backing from the government, largely from the
perspective of location, scale, and infrastructure support of
the browneld.
Next, a structured meeting based on the procedure of Delphi
Method (Linstone and Turoff, 1975) was conducted to derive the
initial index system. More specically, a panel of experts, consisting
of environmental scientists, social scientists, nancial experts,
managerial experts, and urban-planning experts, was hired to
participate in the group decision making process. First, the experts
ll in a pre-dened survey independently according to their own
knowledge base and the references provided by the facilitator,
including Syms (1999), De Sousa (2000), and Wedding and
Crawford-Brown (2007). Then the facilitator collected all the surveys and prepared an anonymous summary, which was sent out to
the experts again for revisions or additional opinions. After three
iterations, a consensus among all the experts was achieved, which
identied the initial set of factors affecting the browneld redevelopment projects. This initial system and the corresponding

measurement are shown in Table 1. Please note that the last column
shows the reliability test results, which are discussed in the next
section.

3. Testing and optimization of the evaluation index system

for browneld redevelopment projects
3.1. Reliability and validity analysis of initial proposed evaluation
index system for BRPs
The reliability of the proposed index system was tested by a
reliability coefcient, which was calculated based on the data
collected from a well-designed questionnaire consisting of all the
criteria listed in Table 1. Stakeholders are requested to evaluate the
criteria according to a nine-level approach, which is different from
the typical ve-level Likert-scale item (Likert, 1932). More specifically, the nine levels are: 1-Strongly Disagree; 2-Somewhere between Strongly Disagree and Disagree; 3-Disagree; 4-Somewhere
between Disagree and Neither Agree or Disagree; 5-Neither Agree
or Disagree; 6-Somewhere between Neither Agree or Disagree and
Agree; 7-Agree; 8-Somewhere between Agree and Strongly Agree;
and 9-Strongly Agree.
The questionnaire was distributed to 500 key stakeholders in
ve major cities of China: Xi'an, Beijing, Shanghai, Shenzhen, and
Jinan. Out of the 340 questionnaires that were returned, 335 were
valid, giving a validity rate of 98.53%.
The questionnaire data were then analyzed using Cronbach's
alpha (Cronbach, 1951), one of the most commonly used reliability
coefcients. The coefcient is calculated as:

0
a

k B
B
B1 
k 1@

1
Pk

variC
C
C
vari A

i1

(1)

where k is the number of components (K-items or testlets) and i is

the component for the current sample of people. As a correlation
coefcient, Cronbach's alpha ranges in value from 0 to 1. Normally,
the reliability of the questionnaire is regarded as questionable
when the value falls within the range of 0.60e0.65. A value within
0.65e0.70 implies minimum acceptability. A value within
0.70e0.80 means fairly good. A value within 0.80e0.90 or above
indicates very good (Li and Ma, 2007). By using the software SPSS
16.0, we calculated the values of Cronbach's alpha for all four dimensions. The results are listed in the last column of Table 1. We
can see that the value of Cronbach's alpha for every dimension is
higher than 0.850, which indicates that our proposed index system
has a very high reliability.
Validity refers to the degree to which evidence and theory
support the interpretations of scores veried by using appropriate
tests (AERA, 1999). In this study, our major concern is the construct
validity, which refers to the extent to which our constructed surveys and questionnaire actually explain and measure the evaluation index system. We herein use Factor Analysis, a statistical
method, to examine the validity, and then to optimize the initially
proposed evaluation index system for BRPs.
The Factor Analysis starts with KMO (Kaiser-Meyer-Olkin)
(Kaiser, 1970; Cerny and Kaiser, 1977; Dziuban and Shirkey, 1974)
and Bartlett's test (Snedecor and Cochran, 1983), which measures
the strength of the relationships among criteria. More particularly, KMO measures the sampling adequacy, and Bartlett's test
tests the null hypothesis that the correlation matrix is an identify
matrix. Given the results of the test (Table 2), one can see that
the KMO value very close to 1 (i.e., the factor analysis should

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

177

Table 1
Initially established evaluation index system for BRP and its reliability coefcients.
Initial evaluation
index system for BRP

Dimensions

Criteria

Measurement

Reliability coefcients

Social and Economic

Improve the image of local community and government

Matchup with city planning
Improve living quality of local residents
Improve local security status
Increase local employment rates
Increase land value of neighborhood
Increase local tax base
Ease the pressure on green land development
Protect and recycle the land/soil resource
Improve their mediation technologies
Net present value (NPV)
Return on investment (ROI)
Payback period (PP)
Total cost of browneld remediation and construction
Ratio of browneld remediation cost to the total cost
Lower the health risks of local residents
Improve soil quality
Enhance the quality of groundwater
Improve air quality
Increase green cover percentage
Size of the browneld
Location of the browneld
Transportation convenience of the browneld
Status of infrastructure facilities of the browneld
Inuence from other nearby contamination hazards
Technological difculties and time requirement of
browneld remediation
Inuence from the policy and legislation

Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Quantitative
Quantitative
Quantitative
Quantitative
Quantitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative
Qualitative

No. of Items 10
Cronbach's Alpha 0.859

Financial and accounting

Environmental and health

Prospective Value

No. of Items 5
Cronbach's Alpha 0.856

No. of Items 5
Cronbach's Alpha 0.898

No. of Items 7
Cronbach's Alpha 0.853

Qualitative

Table 2
Results of KMO and Bartlett's test.
Kaiser-Meyer-Olkin measure of sampling adequacy
Bartlett's test of sphericity

yield distinct and reliable factors), and the Bartlett's test of

sphericity is signicant (i.e., the signicance level is small
enough to reject the null hypothesis). Thus, we can say that our
procedure is valid, and the data collected from the questionnaires
are appropriate for proceeding with Factor Analysis.

3.2. Optimization of the initially proposed evaluation index system

for BRPs
To facilitate better applications of the index system, the number
of criteria is reduced from 27 to 24 by using Factor Analysis
(Gorsuch, 1983), the most commonly used data reduction technique. Details of this technique are discussed as follows.
Suppose that there are n variables, consisting of x1, x2, ,xn.
According to the requirements of Factor Analysis, all of these variables have been standardized. Suppose that n variables can be a
linear combination of k factors, f1, f2, ,fk. Therefore:

8
x a11 f1 a12 f2 ::: a1k fk 1
>
>
< 1
x2 a21 f1 a22 f2 ::: a2k fk 2
:::
>
>
:
xn an1 f1 an2 f2 ::: ank fk n

0.929
4.805E3
351
0.000

Approx. chi-square
Degree of freedom
Signicant level

The method that the authors used for factor extraction (optimization) was principal components analysis (PCA), which is used
as a tool in exploratory data analysis and involves the calculation of
the eigenvalue decomposition of a data covariance matrix or singular value decomposition of a data matrix, after mean centering
the data for each attribute. The results of a PCA are discussed in
terms of component scores and loadings.
Please note that the eigenvalue associated with each factor
shows the variance explained by that particular factor. For example
in Table 3, the eigenvalue of factor 1 is 10.622, which explains
39.340% of total variance. According to the Kaiser rule, the factors
which satisfy eigenvalue >1 are extracted (Bandalos and BoehmKaufman, 2008). From Table 3, only six factors can meet the criteria,
and thus these six factors were extracted, which can explain more
than 66 per cent of total variance. All the remaining factors are not
signicant.

Table 3
Component scores and loadings.

(2)

where aij are the factor loadings for the ith subject, for j 1,2 ,k;
i 1,2 ,n; and i are independently distributed error terms with a
mean of zero and nite variance, which may not be the same for all
i.

Extraction sums of squared loadings

Rotation sums of squared loadings

Total

% Of Variance

Cumulative %

Total

% Of Variance

Cumulative %

10.622
2.491
1.529
1.154
1.022
1.018

39.340
9.227
5.663
4.273
3.785
3.770

39.340
48.568
54.231
58.504
62.290
66.060

4.548
3.250
3.142
2.592
2.328
1.977

16.844
12.036
11.637
9.599
8.623
7.321

16.844
28.879
40.517
50.115
58.739
66.060

178

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

Table 4
Component matrix.
Component

Increase of green cover percentage

Return on investment (ROI)
Total cost of Browneld remediation and construction
Improvement of remediation technologies
Improvement of air quality
Transportation convenience of browneld area
Ratio of Browneld remediation cost to total cost
Improvement of soil quality
Easing the pressure on green land development
Payback period (PP)
Lowering the health risk of local residents
Technological difculties and time requirement of browneld remediation
Net present value (NPV)
Protecting and recycling the land/soil resource
Status of Infrastructure facilities of browneld area
Matchup with city planning
Improvement of living quality of local residents
Increase local employment rate
Increase local tax base
Location of browneld
Increase land value of neighborhood
Inuence from other contamination hazards nearby
Size of browneld
Improvement of image of local community and government
Improvement of local security status
Inuence from policy and legislation
Improvement of the quality of groundwater

0.716
0.678
0.675
0.674
0.674
0.669
0.666
0.659
0.650
0.649
0.644
0.642
0.639
0.634
0.628
0.618
0.615
0.610
0.604
0.604
0.604
0.585
0.584
0.554
0.528
0.495
0.588

0.373
0.342
0.219
0.385
0.448
0.353
0.247
0.532
0.309
0.427
0.156
0.164
0.275
0.360
0.280
0.135
0.394
0.160
0.337
0.102
0.263
0.591

0.145
0.235

0.390
0.233
0.378
0.104
0.175
0.478
0.453
0.235
0.125
0.299
0.254
0.494
0.371
0.116

0.101
0.113
0.199

0.140
0.295
0.302

0.146
0.345
0.114
0.113
0.312
0.187
0.244

0.159
0.117
0.296

0.137

0.168
0.243
0.360
0.190

0.128
0.338
0.210

0.382
0.357
0.131
0.179
0.101

0.136
0.252

0.202
0.157
0.176
0.101

0.420
0.237
0.145
0.245

0.397
0.378
0.197
0.195
0.240
0.128

0.301
0.149
0.281
0.180

0.384
0.158
0.139
0.378
0.116
0.223
0.177

Extraction Method: Principal Component Analysis.

Result: 6 components extracted.

The component matrix (Table 4) helps to explain the degree of

correlation between the component and the factor. Pure variables
have loadings of 0.3 or greater on only one factor. Complex variables may have high loadings on more than one factor, which
makes interpretation of the results more difcult. In this case,
rotation would be necessary.
Varimax rotation, where the factor axes are kept at right angles
to each other, is most frequently chosen in principal component
analysis and Factor Analysis. It maximizes the sum of the variances
of the squared loadings (Kaiser, 1958). Ordinarily, rotation reduces
the number of complex variables and improves interpretation.
However, in this analysis, the rotated solution still includes several
complex variables (see Table 5). As can be seen in the table, Factor
1 comprises 7 items with factor loadings ranging from 0.477 to
0.856. Factor 2 consists of 5 items with factor loadings going from
0.594 to 0.756. Factor 3 comprises 4 items with factor loadings
ranging from 0.667 to 0.743. Factor 4 consists of 3 items with
factor loadings going from 0.641 to 0.727. Factor 5 comprises 4
items with factor loadings ranging from 0.451 to 0.707. Factor 6
comprises 4 items with factor loadings ranging from 0.483 to
0.726.
The nal step in the Factor Analysis is to remove unnecessary
items (optimization) and to label factors. From the previous
step, 6 factors have been generated. By a rule of thumb in
exploratory Factor Analysis, loadings should be 0.5 or higher to
conrm that independent variables identied a priori are represented by a particular factor (Gorsuch, 1983). Hence, to optimize
the evaluation index system for BRP, we deleted the items
(criteria) whose factor loading were less than 0.5: Improvement of
living quality of local residents (0.477), Technological difculties
and time requirement of browneld remediation (0.451), and Inuence from policy and legislation (0.483) (see Table 5). Then,
according to the characteristics of the items included in each
extracted factor, we labeled factors 1 to 6 as Environmental and

Health Benet Indicators, Financial Indicators, Browneld Site Indicators, Societal Stability Indicators, Policy and Technical Indicators, and Performance Indicators, respectively. So far, we have
obtained the optimized evaluation index system for BRPs, which
is depicted in the rst three columns of the numerical entries in
Table 5.
4. Effectiveness verication of the optimized evaluation
index system for browneld redevelopment projects
Having been optimized, the index system still needs to be tested
regarding the feasibility of its employment in the evaluation of
BRPs. The authors adopted the Structural Equation Modeling (SEM)
method (Pearl, 2000), which is a statistical technique for testing
and estimating causal relations using a combination of statistical
data and qualitative causal assumptions, to conrm the effectiveness of the optimization of the evaluation index system for BRPs.
With the assistance of AMOS software, a parametric test and the
comprehensive evaluation of the model were executed.
4.1. Parametric test
The parametric test mainly conducts an inspection of the
representativeness and rationality of parameters, as well as the
meaning and rationality of evaluation parameters.
Test of the representativeness of parameters in the Structural
Equation is similar to the parametric representative assessment in
the linear regression equation, namely, a test of the parameter t.
Test of the rationality of parameters is used to assess the
realistic meaning of the selected parameters.
AMOS provides an easy and convenient method by using CR
(Critical Ration) to test the representativeness of parameters. The
results of the parametric estimation can be found in Table 6.
From the Un-Standardized Regression Coefcients in Table 6, all

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

179

Table 5
Optimized evaluation index system for BRP and rotated component matrix.
Optimized evaluation
Factors/indicators
Index system for BRP

Criteria

Component
1

Environmental
and health benets

Improvement of the quality of groundwater

Improvement of soil quality
Improvement of air quality
Lowering the health risk of local residents
Increase of green cover percentage
Improvement of remediation technologies
Improvement of living quality of local residents
Financial
Payback period (PP)
Return on investment (ROI)
Total cost of browneld remediation and construction
Ratio of browneld remediation cost to total cost
Net present value (NPV)
Browneld site
Location of browneld
Status of infrastructure facilities of browneld area
Transportation convenience of browneld area
Size of browneld
Societal Stability
Increase local employment rate
Increase local tax base
Improvement of local security status
Policy and Technical Protecting and recycling the land/soil resource
Inuence from other contamination hazards nearby
Easing the pressure on green land development
Technological difculties and time requirement of browneld remediation
Performance
Matchup with city planning
Improvement of image of local community
and government
Increase land value of neighborhood
Inuence from policy and legislation

0.856
0.807
0.776
0.729
0.719
0.600
0.477

0.514
0.348 0.326
0.756
0.720 0.320
0.704
0.663
0.594
0.356
0.743
0.722
0.717
0.667
0.727
0.725
0.641

0.430

0.707
0.580
0.302
0.486 0.568
0.309 0.436
0.451
0.477

0.726
0.674

0.334

0.581
0.443 0.483

Extraction Method: Principal Component Analysis.

Rotation Method: Varimax with Kaiser Normalization.
Result: Rotation converged in 9 iterations.

P values are less than 5%, so all Path Coefcients pass the test of
representativeness.
The rationality of parameters should be determined before
evaluating the Goodness-of-Fit of a model. So it is necessary to test
whether the parametric estimation exceeds the acceptable range.
The results of estimated coefcient of error variance are shown in
Table 7. Referring to the denition by Hair et al. (1998), the

parameters in Table 7 should satisfy the following 2 conditions:

 The negative error variance doesn't exist;
 The Standardized Coefcient doesn't exceed 1 or isn't too close
to 1 (over 0.95).
According to Table 7, no negative error variance exists in

Table 6
Un-standardized and standardized regression coefcient.
Un-standardized regression coefcient

F2)F0
F1)F0
F3)F0
F4)F0
F13)F1
F23)F2

Standardized regression coefcient

Un-standardized coefcient

Standardized error

Probability

Mark

1.044
1.000
0.949
1.125
1.049
0.963

0.105

Critical portion value

9.971

***

par_3

0.099
0.122
0.062
0.070

9.591
9.190
17.001
13.715

***
***
***
***

par_4
par_5
par_1
par_2

Standardized coefcient
F2)F0
F1)F0
F3)F0
F4)F0
F13)F1
F23)F2

0.832
0.707
0.775
0.777
0.804
0.745

*Represents P < 0.05,** represents P < 0.01,*** represents P < 0.001.

Table 7
Estimated coefcient of error variance (partial).

F0
e27
e29
e2
e4
e26
e28
e30
e3
e5

Un-standardized coefcient

Standardized error

Critical Portion value

Probability

Mark

0.883
0.530
0.384
0.868
1.032
0.427
0.734
0.343
1.061
1.051

0.144
0.087
0.108
0.083
0.088
0.078
0.133
0.083
0.098
0.096

6.113
6.109
3.545
10.494
11.741
5.454
5.523
4.131
10.799
10.933

***
***
***
***
***
***
***
***
***
***

par_24
par_26
par_28
par_32
par_34
par_25
par_27
par_29
par_33
par_35

180

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

Table 8
The Fit indices.
Absolute t index

Comparative t Index

Parsimonious t Index

Model

c2

c2 /d f

RMSEA

IFI

CFI

TLI

PNFI

PCFI

AIC

1
2
Evaluation Standard

0.849
0.875
>0.90

0.848
0.888
>0.90

0.832
0.889
>0.90

0.723
0.751
>0.5

0.768
0.792
>0.5

1201.788
848.843
The smaller, the better.

1081.788
740.843
e

3.402
3.012
<3.0

0.172
0.075
<0.08

Fig. 2. Application procedure of the example.

measure error values of error variances. Furthermore, no regression

coefcient exceeds 0.95 based on the standardized regression coefcients in Table 6. Therefore, this model can be tested on the Fit
Index of the comprehensive evaluation.
4.2. The comprehensive evaluation of the model
The comprehensive evaluation is usually implemented by the Fit
Index computation. The Fit Index refers to a statistical parameter
that is constructed from a certain human perspective, to reect the
Goodness-of-t of a model. In general, the t indices include Absolute Fit Index, Comparative Fit Index and Parsimonious Fit Index
(Byrne, 2001).
The Absolute Fit Index mainly consists of c2 (Chi-square),
RMSEA (Root Mean Square Error of Approximation), and SRMR

(Standardized Root Mean Square Residual). The Comparative Fit

Index generally includes NFI (Normed Fit Index), CFI (Comparative
Fit Index), and NNFI (Non-Normed Fit Index). These two types of
indices are used to evaluate the Goodness-of-Fit of a single model.
The Parsimonious Fit Index is a more common index, which includes PNFI (Parsimony Normed Fit Index), PCFI (Parsimonious
Comparative Fit Index), and AIC (Akaike Information Criterion).
The comprehensive evaluation of a model was achieved using
diversied Fit indices. Table 8 lists the Absolute Fit Indices,
Comparative Fit Indices, and Parsimonious Fit Indices of two
models, where Models 1 and 2 respectively refer to the initially
established index system and the optimized index system after
Factor Analysis.
From Table 8, it is noticed that, although the two models do not
reach the evaluation standard, they are very close to that.

Table 9
Values of the proposal options.
Factors/indicators

Environmental and Health Benets

Financial

Browneld Site

Societal Stability

Policy and Technical

Performance

Criteria

Improvement of the quality of groundwater

Improvement of soil quality
Improvement of air quality
Lowering the health risk of local residents
Increase of green cover percentage
Improvement of remediation technologies
Payback period (PP)
Return on investment (ROI)
Total cost of browneld remediation and construction
Ratio of browneld remediation cost to total cost
Net present value (NPV)
Location of browneld
Status of infrastructure facilities of browneld area
Transportation convenience of browneld area
Size of browneld
Increase local employment rate
Increase local tax base
Improvement of local security status
Protecting and recycling the land/soil resource
Inuence from other contamination hazards nearby
Easing the pressure on green land development
Matchup with city planning
Improvement of image of local community and government
Increase land value of neighborhood

Proposal options
A

75
70
78
80
77
75
5
21
1500
15
2580
75
85
80
55
90
85
80
75
60
60
70
70
75

85
82
84
89
87
80
7
30
2700
27
3500
85
82
88
80
92
90
75
90
65
90
85
80
70

77
60
65
68
85
90
4
17
2100
18
2600
80
90
90
85
75
72
88
75
60
85
80
85
85

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

181

Comparing the two models, Model 2 is a better model, because its

Fit Indices are closer to the evaluation standard than Model 1, and
some of its Fit Indices have already reached the evaluation standard. For example, RESMA of Model 2 is 0.075, which lies in the
range from 0.05 to 0.08, so it is considered to be a good t.

evaluation index system. Meanwhile, the criteria of the proposed

index system can serve as the base of their own primary criteria set,
which can be further enhanced by adding and deleting certain
criteria if necessary. The practicability of the proposed index system
is illustrated by a real world illustration example.

5. An illustration example

Acknowledgments

As previously stated, the proposed evaluation index system can

be integrated with designs of other index systems and/or can work
as a starting point of most of the project evaluation techniques.
Herein, we use a brief example, which embedded our index system
into a grey relational evaluation approach, to assist a local government in making a redevelopment decision of an abandoned
coking plant located in the downtown area. The application procedure is explained in Fig. 2.
The decision process starts from an overview of the situation,
from which three alternatives, A) new industrial planning, B)
commercial planning, and C) real estate planning, are identied for
further evaluation. Next, the local government employs the evaluation index system derived in this research, according to which
relevant stakeholders evaluate each alternative based on their own
knowledge and experiences. The aggregated (averaged in this case)
evaluation record (Table 9) is then used as the input to a grey
relational analysis, which computes a grey correlation coefcient
for each alternative. First introduced by Deng (1985), Grey relational analysis is an impact evaluation model which analyzes the
associativity and similarity among multiple alternatives based on
the degree of relation (Ju, 2013). Please note that the detailed
computation in grey relational evaluation approach has been
omitted to avoid distraction from our index system. The interested
readers are referred to Chan and Tong (2007), Liu et al. (2011), and
Ju (2013) for details. Finally, in light of the grey correlation coefcients obtained from the previous step, the best alternative is
selected. In this example, alternative C, real estate planning, gives
the highest coefcient, and thus is selected.

This research was supported in part by the National Social Sciences Foundation of China through the project Research on Life
Span Risk Management of Browneld Redevelopment Project
(10BJY024), by the Provincial Social Sciences Foundation in
Shaanxi, China through the project Market Mechanism and Policy
System for Browneld Redevelopment in Shaanxi Province
(08E023), and by the Humanities and Social Science Foundation of
the Ministry of Education of China through the project Stakeholders Based Multi-Criteria Decision Analysis of Browneld
Redevelopment (08JC630066). Backing was also provided by the
Natural Sciences and Engineering Research Council (NSERC) of
Canada (NSERC-RGPIN-4118-2013 27238), which funded the
Research Discovery Grant for the project entitled Systems Engineering Approaches to Sustainable Environmental Management.
Finally, the Centre for International Governance Innovation (CIGI)
located in Waterloo, Ontario, Canada, also furnished support.

6. Conclusions and suggestions regarding the application of

the proposed evaluation index system for BRPs
The establishment of an evaluation index system plays a very
important role in the evaluation of BRPs. Based on the current
situation of brownelds in China, the requirements of sustainable
development, and the analysis of stakeholders, the authors initially
developed the evaluation index system for BRPs, which includes 27
criteria from four dimensions, including Societal and Economic,
Financial and Accounting, Environmental and Health, and Prospective Value. With the help of SPSS 16.0, the authors calculated
Cronbach's alpha, and conducted KMO and Bartlett's tests. The results of these tests showed that the initialized evaluation index
system for BRP has good reliability and validity. By means of principal components analysis, the authors extracted six factors, labeling them as Environmental and Health Benet Indicators,
Financial Indicators, Browneld Site Indicators, Societal Stability
Indicators, Policy and Technical Indicators, and Performance Indicators, and deleted three criteria whose factor loadings are less
than 0.5. In this way, the authors built up their optimized evaluation index system for BRPs, whose effectiveness of optimization has
been checked and veried by Structure Equation modeling with the
assistance of AMOS 8.0.
The most common problem in browneld redevelopment is to
select proper planning for a certain browneld site, which could be
mutually benecial to all stakeholders The established evaluation
index system for BRPs provides users with informative reference
values by employing the same process to obtain their own

Appendix A. Supplementary data

Supplementary data related to this article can be found at http://
dx.doi.org/10.1016/j.envsoft.2015.09.012.
References
Alker, S., Joy, V., Roberts, P., et al., 2000. The denition of browneld. J. Environ.
Plann. Manag. 43 (1), 49e69.
American Educational Research Association (AERA), American Psychological Association, & National Council on Measurement in Education, 1999. Standards for
Educational and Psychological Testing. American Educational Research Association, Washington, DC.
Bandalos, D.L., Boehm-Kaufman, M.R., 2008. Four Common Misconceptions in
Exploratory Factor Analysis. In: Lance, Charles E., Vandenberg, Robert J. (Eds.),
Statistical and Methodological Myths and Urban Legends: Doctrine, Verity and
Fable in the Organizational and Social Sciences. Taylor & Francis, pp. 61e87.
BenDor, T.K., Metcalf, S.S., Paich, M., 2011. The dynamics of Browneld redevelopment. Sustainability 3, 914e936. http://dx.doi.org/10.3390/su3060914.
Bowie, N., 1988. The moral obligations of multinational corporations. in: Luper-Foy.
In: Problems of International Justice. Westview Press, Boulder, pp. 97e113.
Brebbia, C.A. (Ed.), 2006. Brownelds III: Prevention, Assessment, Rehabilitation
and Development of Browneld Sites. WIT Press, Southampton, UK.
Byrne, B.M., 2001. Structural Equation Modeling with AMOS e Basic Concepts,
Applications, and Programming. LEA. ISBN 0-8058-4104-0.
Cerny, C.A., Kaiser, H.F., 1977. A study of a measure of sampling adequacy for factoranalytic correlation matrices. Multivar. Behav. Res. 12 (1), 43e47.
Chan, W.K., Tong, T.K.L., 2007. Multi-criteria material selections and end-of-life
product strategy: grey relational analysis approach. Mater. Des. 28 (5),
1539e1546.
Chen, Y., Hipel, K.W., Kilgour, D.M., Zhu, Y., 2009. A strategic classication support
system for browneld redevelopment. Environ. Model. Softw. 24 (5), 647e654.
Chrysochoou, M., Brown, K., Dahal, G., 2012. A GIS andindexingschemetoscreenbrowneldsfor area-wideredevelopmentplanning. Landscapeand Urban Plan.
105, 187e198.
Clarkson, M.B.E., 1995. A stakeholder framework for analyzing and evaluating
corporate social performance. Acad. Manag. Rev. 20 (1), 92e117.
Cronbach, L.J., 1951. Coefcient alpha and the internal structure of tests. Psychometrika 16 (3), 297e334.
De Sousa, C.A., 2000. The Browneld Problem in Urban Canada: Issues,
Approachesand Solutions. University of Wisconsin-Milwaukee (PHD
dissertation).
De Sousa, C., 2001. Contaminated sites: the Canadian situation in an international
context. J. Environ. Manag. 62, 131e154.
Deng, J.L., 1985. Grey Control Systems. Huazhong University of Science and Technology Press, Wuhan (in Chinese).
berl, G., Ortmann, M., Frhwirth, W., 2013. Introducing a goal-oriented sustainDo
ability assessment method to support decision making in contaminatedsitemanagement. Environ. Sci. Policy 25, 207e217.
Dziuban, C.D., Shirkey, E.C., 1974. When is a correlation matrix appropriate for factor

182

Y. Zhu et al. / Environmental Modelling & Software 74 (2015) 173e182

analysis? Psychol. Bull. 81, 358e361.

Freeman, R.E., 1984. Strategic Managementda Stakeholder Approach. Pitman
Publishing Inc, Massachusetts, p. 276.
Freeman, R.E., 2003. In: Brebbia, C.A. (Ed.), Stakeholder Theory and Organizational
Ethics. Berrett-Koehler Publishers.
Gass, G., Biggs, S., Kelly, A., 1997. Stakeholders, science and decision making for
poverty-focused rural mechanization research and development. World Dev. 25
(1), 115e126.
Gorsuch, R.L., 1983. Factor Analysis. Lawrence Erlbaum, Hillsdale, NJ.
Grimble, R., Wellard, K., 1997. Stakeholder methodologies in natural resource
management: a review of principles, contexts, experiences and opportunities.
Agric. Syst. 55 (2), 173e193.
Guarini, M., Battisti, F., 2014. Social housing and redevelopment of building complexes on browneld sites: the nancial sustainability of residential projects for
vulnerable social groups. Adv. Mater. Res. 869e870, 3e13.
Hair, J.F., Anderson, R.E., Tatham, R.L., Black, W.C., 1998. Multivariate Data Analysis
with Readings. Prentice Hall, Englewood Cliffs, NJ.
Ju, Y.B., 2013. Methods and Applicationns for Emergency Management under Fuzzy
Environment. Beijing Institute of Technology Press, Beijing.
Kaiser, H.F., 1958. The varimax criterion for analytic rotation in factor analysis.
Psychometrika 23 (3), 187e200.
Kaiser, H.F., 1970. A second generation little jiffy. Psychometrika 35, 401e415.
Li, J.L., Ma, B.L., 2007. Management Statistics Application and Practice. Tsinghua
University Press.
Likert, R., 1932. A technique for the measurement of attitudes. Archives Psychol.
140, 1e55.
Linstone, H.A., Turoff, M., 1975. The Delphi Method: Techniques and Applications.
Adison-Wesley, Reading, Mass. ISBN 9780201042948.
Liu, S.F., Xie, N., Forrest, J., 2011. Novel models of grey relational analysis based on
visual angle of similarity and nearness. In: Grey Systems: Theory and Application, vol. 1(1), pp. 8e18.
National Round Table on the Environment and the Economy (NRTEE), 2003.
Cleaning up the Past, Building the Future: a National Browneld Redevelopment Strategy for Canada. ISBN 1-894737-05-9. Canada, Ottawa.
ODA, 1995. Guidance Note on How to Do Stakeholder Analysis of Aid Projects and
Programs. Overseas Development Administration, Social Development
Department. www.euforic.org/gb/stake1.htm.

Pearl, J., 2000. Causality: Models, Reasoning, and Inference. Cambridge University
Press. ISBN 0521773628.
PMI, 2000. A Guide to the Project Management Body of Knowledge. Project Management Institute, 130 South State Road, Upper Darby (f creative problem
solving).
Rossi, P.,H., Lipsey, W.M., Freeman, H.E., 2004. Evaluation: a Systematic Approach,
seventh ed. SAGE, Thousand Oaks, CA.
Sardinha, I., Craveiro, D., Milheiras, S., 2013. A sustainability framework for redevelopment of rural brownelds: stakeholder participation at SAeO DOMINGOS
mine, Portugal. J. Clean. Prod. 57, 200e208.
Sch
adler, S., Morio, M., Bartke, S., et al., 2011. Designing sustainable and economically attractivebrowneld revitalization options using an integrated assessment
model. J. Environ. Manag. 92, 827e837. http://dx.doi.org/10.1016/
j.jenvman.2010.10.026.
Sch
adler, S., Morio, M., Bartke, S., Finkel, M., 2012. Integrated planning and spatial
evaluation ofmegasite remediation and reuse options. J. Contam. Hydrology
127, 88e100. http://dx.doi.org/10.1016/j.jconhyd.2011.03.003.
Sch
adler, S., Finkel, M., Bleicher, A., et al., 2013. Spatially explicit computation of
sustainability indicatorvalues for the automated assessment of land-use options.
Landsc.
Urban
Plan. 111,
34e45.
http://dx.doi.org/10.1016/
j.landurbplan.2012.12.002.
Snedecor, George W., Cochran, William G., 1983. Statistical Methods, eighth ed. Iowa
State University Press.
Syms, P., 1999. Redeveloping browneld land: the decision-making process. J. Prop.
Invest. Finance 17, 481e494.
United Nations Division for Sustainable Development (UNDSD), 2009. http://www.
un.org/esa/sustdev/index.html. (accessed on 27.05.09.).
US Environmental Protection Agency (USEPA), 2005. Road Map to Understanding
Innovative Technology Options for Brownelds Investigation and Cleanup,
fourth ed. EPA-542-B-05e001.
Varvasovszky, Z., Brugha, R., 2000. How to do (or not to do) A stakeholder analysis.
Health Policy Plan. 15 (3), 338e345.
Wedding, G.C., Crawford-Brown, D., 2007. Measuring site-level success in browneld redevelopments: a focus on sustainability and green building. J. Environ.
Manag. 85, 483e495.
World Business Council on Sustainable Development (WBCSD), 2014. http://www.
wbcsd.org/, (accessed 20.04.14.).