Journal of Geoscience Education

ISSN: 1089-9995 (Print) 2158-1428 (Online) Journal homepage: http://www.tandfonline.com/loi/ujge20

Development and Validation of a Science Inquiry

Skills Assessment

Yiping Lou, Pamela Blanchard & Eugene Kennedy

To cite this article: Yiping Lou, Pamela Blanchard & Eugene Kennedy (2015) Development and
Validation of a Science Inquiry Skills Assessment, Journal of Geoscience Education, 63:1, 73-85,
DOI: 10.5408/14-028.1

To link to this article: https://doi.org/10.5408/14-028.1

Published online: 31 Jan 2018.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=ujge20
 JOURNAL OF GEOSCIENCE EDUCATION 63, 73–85 (2015)

Development and Validation of a Science Inquiry Skills Assessment

Yiping Lou,1,a Pamela Blanchard,2 and Eugene Kennedy2

ABSTRACT
This paper reports on the development and validation of a science inquiry skills assessment for Earth science (iSA–Earth
Science) in middle schools that classroom teachers can use to assess and monitor their students’ science inquiry skills. An
assessment framework with six primary inquiry skills, each with three to six subskills, was developed based on a
comprehensive review of the literature and following the recommendations of the National Science Education Standards (NSES;
NRC, 1996) and A Framework for K–12 Science Education (FSE; NRC, 2012). The six primary skills are: (1) identify questions for
scientific investigations, (2) design scientific investigations, (3) use tools and techniques to gather data, (4) analyze and
describe data, (5) explain results and draw conclusions, and (6) recognize alternative explanations and predictions. The
assessment development and validation process followed guidelines from the Standards for Educational and Psychological
Testing (American Educational Research Association [AERA], American Psychological Association [APA], and National
Council of Measurement in Education [NCME], 1999). Psychometric analysis using both classical test statistics and Item
Response Theory showed the instrument was internally consistent, reliable, and did not function differentially for male and
female students. Implications for science education practice and research are discussed. Ó 2015 National Association of
Geoscience Teachers. [DOI: 10.5408/14-028.1]

Key words: science education, inquiry skill, assessment, validity/reliability, teacher/student empowerment, standards,
formative assessment, Rasch modeling

INTRODUCTION the ways to assist teachers in understanding and helping

Science as inquiry has been at the core of science
education reform for several decades (National Research
Council [NRC], 1996, 2007, 2012, 2013). The National Science
Education Standards (NSES; NRC, 1996) calls for a ‘‘code-
velopment of the skills of students in acquiring science
knowledge, in using high-level reasoning, in applying their
existing understanding of scientific ideas, and in communi-
cating scientific information’’ (145–146). The vision goes
beyond ‘‘science as process,’’ when it was thought that
individual skills, such as observation, inference, and exper-
imentation could be taught independently of content and
context. The integrated character of scientific knowledge and
how science is practiced is again stressed in A Framework for
K–12 Science Education (FSE; NRC, 2012) and the Next
Generation Science Standards (NGSS; NRC, 2013), emphasiz-
ing that the work of scientists and engineers requires the
concurrent use of both science knowledge and scientific
inquiry practices. Furthermore, FSE argues that teachers
need to assess student understanding and monitor their
progress, and calls for assessments that ‘‘test students’
understandings of science as a content domain and their
understanding of science as an approach’’ (263).
Effective educational assessments inform instruction and
improve student learning (NRC, 2000). Helping students
engage in scientific inquiry and develop science inquiry skills
in the context of learning science is one of the most
important goals of science education (NRC, 2007). One of
Received 1 May 2014; revised 19 November 2014; accepted 28 November 2014;
published online 18 February 2015.
1
Department of Educational and Psychological Studies, University of
South Florida, Tampa, Florida 33647
2
School of Education, Louisiana State University, Room 223, Peabody
Hall, Baton Rouge, Louisiana 70803
a
Author to whom correspondence should be addressed. Electronic mail:
ylou@usf.edu. Tel.:813-974-7886.
students develop science inquiry skills is the development
and use of a teacher-friendly science inquiry skill assessment
tool that can help both the teachers and students identify
strengths and weaknesses in their science inquiry skills and
to ‘‘chart students’ progress over time instead of simply
measuring performance at a particular point’’ (NRC, 2012,
318). If students are to learn how science is done, then at
least three important conditions are necessary: (1) teachers
must understand science processes themselves (Lewis,
2008), (2) students must have multiple opportunities to
practice their skills, and (3) students must understand
whether they are progressing in their skill attainment or
not (Radford, DeTure, and Doran, 1992). Validated and
reliable standards-aligned assessments are crucial instru-
ments for aiding teachers in diagnosing their students’
understanding and competencies as well as providing
feedback about their performance (Cooper, Shepardson,
and Harber, 2002). The need for a formative assessment tool
in the area of Earth system science ‘‘is paramount in teaching
and learning to show teachers what their students under-
stand and how to adjust their instruction’’ (Lewis, 2008, 453).
In fact, the FSE encourages researchers to explore ‘‘how new
forms of assessment can be made both accessible to teachers
and practical for use in classrooms’’ and that teachers should
‘‘use assessments to plan for, revise, and adapt instruction’’
(NRC, 2012, 319).
This paper reports on the development and validation of
a science inquiry skills assessment for Earth science (iSA–
Earth Science), which is part of a larger project on designing
and developing two interrelated online tools, an Earth
science inquiry skill analyzer and an Earth science inquiry
activities portal. In this paper, we will first review the
research literature on science inquiry assessment. We will
then describe the iterative development and validation
process using the five types of validation evidence and

1089-9995/2015/63(1)/73/13 73 Q Nat. Assoc. Geosci. Teachers

74 Lou et al.
reliability issues as articulated in the Standards for Educational
and Psychological Testing (AERA, APA, and NCME, 1999).
Finally, we will discuss the implications of the results.
WHY EARTH SCIENCE?
The focus on integrating inquiry into classroom teaching
is not new. Science teachers are well aware that the National
Science Education Standards (NRC, 1996) and the Next
Generation Science Standards (NRC, 2013) both call for the
integration of inquiry skills within the context of science
content. However, for some Earth science teachers integrat-
ing inquiry skills into the teaching of geosciences is a far off
and daunting goal. After all, in Earth science, how can one
isolate variables when dealing with plate tectonics or types
of faults?
As Ault (1998) argues:
Unless balanced by attention to how disciplines have evolved
in response to particular phenomena of interest, the
representation of science as process fails to portray the
rationality of scientific inquiry properly. Good science
teaching depends upon resources that represent the demands
characteristic of solving problems in distinctive fields of
inquiry. . . . Geology is similar to but distinct from other
sciences. . . . In summary, geology is not physics. (210–211)
Unlike physics, where variables can be isolated with
relative ease, many geological inquiries rely on ‘‘ ‘experi-
ments’ that have already been conducted by nature.
Consequently, many geological inquiries are of a retrospec-
tive type—trying to unravel what happened in the past’’
(Orion and Kali, 2005, 387). This is the first difficulty that
Earth science teachers encounter—they generally have little
experience in how geoscientists conduct inquiry. Their
second challenge is recognized in Blueprint for Change:
Report from the National Conference on the Revolution in Earth
and Space Science Education (Barstow and Geary, 2001),
where conference attendees called for teachers to ‘‘under-
stand how to assess student performance in inquiry-based
learning, how to write assessments and score assessment
items, and use assessment results to improve classroom
practice’’ (53). Cooper and others (2002) write that scenario-
based assessments in environmental geosciences profes-
sional development programs should serve both to model
assessment techniques and as an instructional evaluation
method and that ‘‘all of these pieces of information can be
used to design more effective instruction’’ (64).
APPROACHES TO SCIENCE INQUIRY
ASSESSMENT
What and how to assess science inquiry skills have been
the focus of science education researchers for several
decades. Many of the inquiry skills assessments from the
1960s through the mid-1970s were developed to research
the effectiveness of a specific science curriculum’s capacity to
improve students’ science process skills. Examples of
research assessments associated with a particular curriculum
include the Science—A Process Approach elementary
science curriculum (Walbesser, 1965; Walbesser and Carter,
1970; Fyffe, 1971; Ludeman, 1975; McLeod et al., 1975),
Biological Sciences Curriculum Study (BSCS, 1962; Laving-
J. Geosci. Educ. 63, 73–85 (2015)
housez, 1973), the Science Curriculum Improvement Study
(Riley, 1972), and the Science Education for Public Under-
standing Program (SEPUP, 1995). One limitation of curric-
ulum-dependent assessments is that they are only available
to the teachers who adopted the specific curriculum.
Another limitation is that most of these assessments, with
the exception of SEPUP (1995), were constructed primarily
to provide information to curriculum developers and/or
researchers. These tests were not constructed to necessarily
provide useful feedback to teachers or students.
As researchers began to look at inquiry skills indepen-
dent of specific curricula in the 1970s and 1980s, they
developed inquiry assessments designed for particular grade
levels or grade ranges rather than a specific published
curriculum. Examples include research assessments for lower
elementary (Dietz and George, 1970), upper elementary
(Molitor and George, 1976), middle school (Tannenbaum,
1971), middle school to high school (Dillashaw and Okey,
1980; Okey, Wise and Burns, 1982; Burns, Okey and Wise,
1985), and middle school to college (Tobin and Capie, 1982).
While these assessments were not associated with a
particular curriculum, questions were raised about content
coverage and the ability of these tools to provide useful
information for classroom teachers. In arguing for a broad-
based inquiry skill framework, Means and Haertel (2002)
noted that curriculum-specific inquiry assessments, includ-
ing more recent efforts in technology-based assessments, are
not broad enough to cover a variety of science contexts, but
content-independent assessments, on the other hand, may
be too decontextualized.
In the 1990s and early 2000s, more focus was placed on
performance-based assessments, typically graded by use of a
rubric. Examples of performance-based assessments include
work by Alonzo and Aschbacker (2004) and Germann and
others (1996). Although performance-based assessments
have the strength of providing a more authentic assessment,
there are several limitations. First, they are often time
consuming to administer and grade. For example, the three
performance assessments created by Alonzo and Aschbacker
(2004) took over 2.5 hours to administer. Consequently, they
were often used by researchers and not very feasible for
classroom teachers to use, given that they might have
multiple classes of students and a limited amount of class
time to devote to the assessment. Second, limiting inquiry
assessments, and consequently inquiry teaching, to perfor-
mance-based assessments in experimental labs can create a
view of science investigations that is limited to those that can
only be carried out in labs that typically involve one
independent variable that can be manipulated and one
dependent variable (Roberts and Gott, 2003).
A recent assessment, Hypothesis, Observation, Conclu-
sion Test was developed using a multiple-choice format by
Orion and Kali (2005) for a guided-inquiry Earth science unit
entitled ‘‘The Rock Cycle.’’ The test includes a total of 20
items, and each item consists of a statement that the student
was asked to identify as hypothesis, observation, or
conclusion. This assessment is one of the few assessments
meant to assess specific inquiry skills developed within Earth
science content. The limited number of items and the
multiple-choice format made this assessment easy to use in
the classroom and quick to grade. However, as is the case for
many other process skill assessments, the instrument was
J. Geosci. Educ. 63, 73–85 (2015)
developed for assessing the effectiveness of a specific
curriculum. Furthermore, it is limited in scope.
INQUIRY SKILL REPRESENTATION AND THE
NEED FOR VALIDATED INSTRUMENT FOR
CLASSROOM USE
While early inquiry assessments focused on specific
inquiry skills of interest to researchers and were often
curriculum- or grade-specific, most were not comprehensive
with regard to the inquiry skill set they assessed, and none
are aligned with NSES. Furthermore, few were designed to
provide formative feedback to students or teachers.
In trying to fill the need for a comprehensive test of
science inquiry skills, Wenning (2007) developed a 35-item
multiple-choice Scientific Inquiry Literacy Test (ScInqLiT)
for high school students. This assessment was developed
following a framework of nine stages of inquiry based on
Science and Its Ways of Knowing (Hatton and Plouffe, 1997).
However, Wenning did not explicitly align the inquiry skills
in his assessment with the science inquiry abilities as
described in NSES. Two pilot studies were conducted with
high school students. The KR20 reliability coefficient for the
second pilot with 61 students was 0.71. After the second
pilot, one question was replaced and several reworded to
increase clarity for the final version of the test, which,
however, was not field-tested. Based on the limitations in
the development process and the relatively highly ‘‘moti-
vated and homogeneous’’ group of high school students that
participated in the second pilot testing, the author advised
caution in using the test and that the best use of the
instrument was for research purposes only (Wenning, 2007,
23).
In addition to questions of content and instructional use,
early inquiry assessments including the most recent ones by
Orion and Kali (2005) and Wenning (2007) were developed
using classical test development approaches and none used
more recent psychometric techniques based on Item
Response Theory (IRT). IRT has the advantage of producing
sample-free estimates of item parameters, item-free esti-
mates of person ability, and permitting the test developer to
explicitly target coverage (test information) of the underlying
trait reflected in test performance during the test assembly
process (Bond and Fox, 2007). IRT techniques such as Rasch
modeling also facilitate item calibration and scaling of
different forms to enable more accurate tracking of growth
in students’ development of inquiry skills.
Additionally, the development process used in early
assessment of inquiry skills were often not as rigorous as
recommended by the Standards for Educational and Psycho-
logical Testing (AERA, APA, and NCME, 1999), which
advocates demonstration of five types of validity evidence:
content, response process, internal structure, relationship to
other variables, and consequences of testing in addition to
issues of reliability. Most early inquiry assessments focused
on content validity and reliability only.
DEVELOPMENT PLAN FOR THE iSA–EARTH
SCIENCE
Development of the iSA–Earth Science was part of a
larger project to design and develop two interrelated Web-
Science Inquiry Skills Assessment 75
based tools: an Inquiry Skills Analyzer (iSA) and an Inquiry
Activity Portal (iAP), which focused on middle school Earth
science. The iSA is a Web-based assessment system that
teachers can use for their students and themselves to take
iSA–Earth Science, view a variety of tabular and graphic
reports to identify strengths and weaknesses of their science
inquiry skills, and to monitor progress throughout the school
year. The iAP is an online Earth science inquiry activities
portal with matrices that are keyed to both science inquiry
skills and Earth science content typically taught in middle
schools. This paper reports on the iSA–Earth Science
development and validation.
The development of the iSA–Earth Science was a
rigorous and iterative process that spanned three years
while the online iSA application was simultaneously being
developed. It is anchored to a comprehensive inquiry skill
framework and is intended to provide teachers with an
overall measure of the inquiry skills of their students and to
indicate specific areas in which additional development may
be needed.
The construction and validation of the iSA–Earth
Science followed a process designed to ensure that the
resulting measure would provide classroom teachers with
accurate and useful information on the inquiry skills of their
students. This process was framed around conceptions of
validity and reliability as articulated in the Standards for
Educational and Psychological Testing (AERA, APA, and
NCME, 1999).
Validity Evaluation
As noted in the Standards for Educational and Psycholog-
ical Testing (AERA, APA, and NCME, 1999), validity
concerns the body of evidence and theory that supports
the interpretation and use of a test score. The question is
whether or not there is sufficient evidence to justify the
interpretation of test scores and the subsequent use of a test
for a given purpose. The process of constructing and
validating a test are not separate activities, but intricately
interwoven into the validity argument, meaning that a case
can be made that the test scores can be used for a given
purpose. The Standards articulate five types of validity
evidence that test developers should address. These stan-
dards and the methods for addressing each of the five types
of validity evidence for the development of the iSA–Earth
Science are outlined in Table I.
To ensure content validity, we used both a priori efforts
(through careful conceptualization of the assessment frame-
work, item specification, and development based on Science
Education Standards and frameworks [NRC, 1996, 2007, and
2012], and existing science inquiry assessment literature) and
a posteriori efforts (through iterative review by expert
teachers and external panel review of content relevance
and alignment). To determine response process validity, we
conducted a series of pilot studies with PTI project
participants and their students over three years. Based on
responses from students and their teachers, and classical
item analyses, modifications were made. Evidence on
internal structure was examined through exploratory and
confirmatory factor analysis of item responses, dimension-
ality and local independence, and Rasch item–person maps.
Relationship to other variables was examined through
correlation with standardized test scores and growth trends
across academic year from field tests. Consequence of testing
76 Lou et al. J. Geosci. Educ. 63, 73–85 (2015)

TABLE I: Type of validity evidence based on the standards of educational and psychological testing.
Type of Evidence Definition/Questions Methods Used in iSA–Earth Science
Evidence based on test To what extent is the content domain explicitly [ Academic experts and practitioners collaborate on an
content described? iterative and structured process of development of
To what extent does the content domain match an inquiry skills framework, test specifications, and
the intended use? test items.

To what extent does the content of the test match [ Pilot studies and reviews from experts not involved
the targeted domain? in test development are used to revise framework
and items.
Evidence based on To what extent do examinee response processes [ Classroom teacher feedback
response processes reflect targeted behaviors? [ Classic item difficulty and discrimination indices
from pilot and field tests.
[ Rasch item and person fit statistics.
Evidence based on To what extent does the internal structure of the [ Exploratory and confirmatory factor analysis of item
internal structure test match that predicted by theory? responses.
To what extent is the targeted construct [ Dimensionality and local item independence.
adequately covered? [ Rasch item–person maps.
Evidence based on Do test scores relate to other variables in predicted [ Correlations with standardized test scores.
relationships to other ways?
variables Convergent
Do test scores correlate with teacher ratings and
expectations?
Discriminant
Do test scores differentiate between
knowledgeable examinees and naı̈ve examinees?
Criterion
Do test scores correlate with grades?
Evidence based on To what extent do test scores unfairly penalize one [ Differential item functioning.
consequences of testing or more groups?

was examined via differential item functioning (DIF) on
gender. Research suggested that male and female students
often have different interests and achievement levels in
science (Greenfield, 1998; Kutnick, 2000; Hoffmann, 2002).
An important feature of the measure developed for this
project is that it not be differentially biased towards male or
female students, as is recommended by NRC (2014). To
address this, we conducted a DIF analysis for each of the
individual items as well as the overall assessment.
Reliability Evaluation
It is widely accepted that a test cannot be valid without
first being reliable. Reliability refers to the extent to which
test scores are influenced by errors of measurement. While
there are a variety of different ways of determining the
reliability of a measure, for the iSA–Earth Science we
focused on internal consistency of the overall measure as
well as the subscales. Reliability was examined through
classical Cronbach’s alpha in pilot tests and IRT analyses on
person and item separation via Rasch modeling. We also
focused on the standard error of measurement and the
person separation index, which is a measure of the ability of
a test to reliably differentiate among examinees.
In summary, the iSA–Earth Science is designed primar-
ily as an assessment instrument for classroom teachers. The
development and validation of the iSA–Earth Science
occurred over a three-year period. Multiple samples were
involved in pilot and field testing, and a wide range of
statistical techniques were utilized. These details are
presented in the results section below.
EMPIRICAL RESULTS
In this section we detail the development and validation
of the iSA–Earth Science. It is organized around the five
types of validity evidence and issues of reliability as
articulated in the Standards (AERA, APA, and NCME, 1999).
Content Validity Evidence
PTI Science Inquiry Skills Framework
PTI Science Inquiry Skills Framework (iSF; see Table II)
was developed based on a comprehensive review of the
literature on science inquiry and assessment including: the
eight fundamental abilities outlined in the Science as Inquiry
Strand of the NSES content standards for grades 5–8 (NRC,
1996), prior science inquiry assessment (e.g., Dillashaw and
Okey, 1980; Burns, Okey, and Wise, 1985; Smith and
Welliver, 1990; Beeth et al., 2001), state standards such as
the Integrated Louisiana Educational Assessment Program
Assessment Guide (Louisiana Department of Education,
2003, 2006), science education methods textbooks (e.g.,
Koballa and Chiappetta, 2005), the science proficiency
framework, Taking Science to School: Learning and Teaching
Science in Grades K–8 (NRC, 2007), and A Framework for K–12
J. Geosci. Educ. 63, 73–85 (2015)
Science Education: Practices, Crosscutting Concepts, and Core
Ideas (NRC, 2012).
To ensure that inquiry skills were correctly identified and
test items could be appropriately aligned with the iSF, 50
Earth science multiple-choice assessment items were select-
ed from a variety of resources, including Earth science
textbooks, science inquiry process skill textbooks, and
several released items from various state exams. A panel of
four expert Earth science teachers, including two with
extensive experience in serving on state science assessment
development review panels, participated in classifying each
item.
Several modifications were made based on this blind
classification process and piloting results. For example, one
inquiry skill area, the use of mathematical skills, was too
broad to be adequately assessed within the iSA–Earth
Science. The team decided that use of mathematics (as
represented in data collection, measurement, and analysis)
would be more feasible and appropriate to assess within the
scope of the iSA–Earth Science.
Test Specifications and Items Development
Using accepted test development guidelines (Popham,
1980; Haladyna, 1994; Downing and Haladyna, 2006) and
the initial PTI Inquiry Skill Framework, a Test Development
Specifications Document was created to guide item and test
development. This document provides specifications on the
format of the items, percentage of coverage for each primary
inquiry skill area, and included guidelines for developing
assessment items.
Since the purpose for developing the iSA–Earth Science
was to create a teacher-friendly assessment tool that
classroom teachers could use to identify strengths and
weaknesses of their students’ inquiry skills and track their
progress, teacher recommendations were important to the
development of the specifications document. A multiple
choice format for all test items was chosen because of
efficiency in administration and automated scoring via the
Web interface. It was determined by the panel of expert
teachers that each item would consist of a stem describing
an Earth science context and four options. Teacher
recommendations resulted in a blueprint of the iSA–Earth
Science that includes three to six items for each of the
primary skill areas with a total of 30 items that could be
completed in a typical 45–50 minute class. The number of
items in each skill area is noted in parentheses after each
inquiry skill title in Table II. The resulting scores in each
inquiry skill area are considered ‘‘indicators’’ of potential
strengths or weaknesses with respect to a student’s skill set.
Since most state testing programs provide subskill scores
that also function as indicators due to a limited number of
items, the test developers believe that classroom teachers
would be well versed in utilizing this information produc-
tively.
All iSA–Earth Science assessment items were developed
using middle school Earth science context. Most inquiry
assessments used mixed content areas; we decided to choose
one content for two reasons. First, this follows the
recommendations of NESE and the framework for K–12
science education (NRC, 2012) that inquiry practices should
not be separated from the teaching of core disciplinary ideas.
Secondly, many teachers feel that Earth science is especially
difficult for inquiry activities and they tend to teach inquiry
Science Inquiry Skills Assessment 77
using other science content such as physics (Blanchard and
Lou, 2009). By developing all items in Earth science, this
assessment can help the teachers better envision the range
of inquiry-based Earth science practices.
The process of creating and vetting iSA–Earth Science
assessment items was iterative. Items were written to assess
specific subskills from the PTI iSF. Care was taken to provide
sufficient elaboration in the question stem so that the item
tested the target inquiry skill rather than the Earth science
content. A Review Checklist was used by each development
team member to independently identify the subskill
addressed by the item and to evaluate whether each item
met best practices in item design. At periodic team meetings
each item was reviewed and the decision was made to accept
the item as written for piloting, revise the item, or reject the
item. If the item was revised, the team offered suggestions to
improve the item, and the newly revised item was again
evaluated using the review protocol sheet. The review
process was repeated until the item was accepted with
revisions or rejected. Figure 1 presents a sample item before
and after revision to improve clarity.
External Expert Review
A panel, external to the project, of science educators
and researchers with expertise in science inquiry and
measurement independently reviewed the iSF and one of
the iSA–Earth Science forms for the field study in Year 3.
They were asked to review the iSF and the items with
respect to: (1) alignment with broad definition of scientific
investigation practices, (2) coverage and representation of
the six primary inquiry skills and related subskills, (3)
alignment of assessment items with the iSF, and (4)
whether items met technical requirements for multiple
choice items. The goal of this process was to ensure the
construct validity of iSF and content validity of the items;
that is, the items are testing the specific inquiry skills they
were designed to test. The external reviewers agreed that
the iSF provides a good representation of primary skills and
subskills of science inquiry and it aligns with broad
definitions of scientific inquiry practices. All also agreed
with the skill and item matching, the accuracy of the
answer choices, and the technical quality of the items.
These external experts provided additional evidence of
content validity of the assessment instrument.
Response Process Evidence
Response process evidence that examinee responses
reflect targeted behaviors was determined through a series of
pilot studies. Following the initial panel review, a process
was established by which the pool of ‘‘acceptable items’’ was
piloted. While the details of the pilot studies are described
below, all shared similar characteristics. First, test items were
compiled into assessment forms based on the opinions of
the review panel. The goal of this aspect of the development
process was for each form to constitute a representative
sampling of the different skill categories in the framework. A
consensus model was used for decision making concerning
the composition of the forms.
The early pilot studies were often limited to only a few
classes of students and as a result item analysis relied heavily
on classical measures of difficulty and discrimination.
Specifically, p values (item difficulty) were calculated for
each item. The ideal p value is .3 to .7. Item discrimination
78 Lou et al.
TABLE II: PTI Science Inquiry Skills Framework.1
iSkills
1. Asking questions (FSE Practice 1: Asking questions)
1.1 Identify testable questions
1.2 Refine/refocus ill-defined questions
1.3 Formulate hypotheses
2. Planning investigations (FSE Practice 3: Planning and carrying out investigations)
2.1 Design investigations to test a hypothesis
2.2 Identify independent variables, dependent
variables, and variables that need to be
controlled
2.3 Operationally define variables based on
observable characteristics
2.4 Identify flaws in investigative design
2.5 Utilize safe procedures
2.6 Conduct multiple trials
3. Carrying out investigations (FSE Practice 3: Planning and carrying out investigations)
3.1 Gather data by using appropriate tools
and techniques
3.2 Measure using standardized units of
measure
3.3 Compare, group, and/or order objects by
characteristics
3.4 Construct and/or use classification systems Given a classification system, use it appropriately to identify specific objects
3.5 Use consistency and precision in data
collection
3.6 Describe an object in relation to another
object (e.g., its position, motion, direction,
symmetry, spatial arrangement, or shape)
4. Analyzing and interpreting data (FSE Practice 4: Analyzing and interpreting data)
4.1 Differentiate explanation from description
4.2 Construct and use graphical
representations
4.3 Identify patterns and relationships of
variables in data
4.4 Use mathematic skills to analyze and/or
interpret data (FSE Practice 5: Using
mathematics)
5. Constructing explanations (FSE Practice 6: Constructing explanations)
5.1 Differentiate observation from inference
5.2 Propose an explanation based on
observation
5.3 Use evidence to make inferences and/or
predict trends
5.4 Form a logical explanation about the
cause and effect relationships in data from
an experiment
J. Geosci. Educ. 63, 73–85 (2015)
Operational Definition
Given a description of a research interest and investigation set up, identify the
testable question.
Given a description of an investigation and a broad and ill-defined question,
identify a more focused question that can be answered through scientific
investigation.
Given a description of variables involved in an experiment, select a reasonable
hypothesis that is being or can be tested.
Given a hypothesis, select a suitable design for an investigation to test it.
Given a description of an investigation or research question, identify the
independent, dependent, and controlled variables involved in it.
Given a description of an investigation, identify how the variables are operationally
defined.
Given a description of an investigation, identify design flaws.
Given a description of an investigation, identify safety procedures and equipment
to conduct the investigation
Given a description of an investigation, understand the need and reasons for
conducting multiple trials.
Given a description of an investigation and type of data to collect, select
appropriate tools and techniques for gathering the data.
Given a description of an investigation and type of data to collect, identify
appropriate unit of measurement.
Given a description or a graphic representation of a set of objects, identify a
classifying feature.
Given a description of an investigation and the type of data to collect, select an
appropriate unit of measurement to achieve precision and consistency.
Given a graphic representation of investigation data, describe an object in relation
to another object.
Given several statements about a data sample of an object or event, differentiate
explanations from descriptions.
Given a description of an investigation and obtained data, identify a graph that
represents the data; or given data in a table, graph or map representation,
identifying a specific data point.
Given a graph or table of data from an investigation, identify the relationship
between the variables.
Given data such as in a table or figure, use mathematical skills to calculate and
interpret the data.
Given several statements about a data sample of an object or event, differentiate
inference from observations.
Use data and information gathered to develop an explanation of the observation.
Given data in a table, graph, or verbal description, use the evidence to make
inference or prediction.
Given data from an investigation, draw a conclusion about the cause and effect
relationships in the data.
J. Geosci. Educ. 63, 73–85 (2015) Science Inquiry Skills Assessment 79

TABLE II: continued.

iSkills Operational Definition
6. Engaging in arguments from evidence (FSE Practice 7: Engaging in arguments from evidence)
6.1 Consider alternate explanations Given a set of data collected and a possible explanation of the results, identify
another possible explanation.
6.2 Identify faulty reasoning not supported by Given a set of data collected, differentiate concluding statement(s) that follow
data logically from the data and those that may relate to faulty reasoning and
misinterpretation of the data.
1
The PTI Science Inquiry Skills Framework is aligned with six of the eight scientific practices in FSE (NRC, 2012).

was computed based on item-total correlations. These values IRT Analyses

were deemed acceptable if they were positive and greater
than .20. Finally, distractor analysis focused on the extent to
which the distractors functioned as ‘‘foils’’ for less able
examinees. If the proportion of knowledgeable examinees
(upper 25%) selecting a distractor was lower than the
proportion of less knowledgeable examinees (bottom 25%)
that attempted an item, the distractor was deemed to be
acceptable. Interitem correlations and Cronbach’s alpha
were also computed but used cautiously. Based on the
results of these analyses, items were either selected for
revision or set aside for field testing.
During the pilot testing, teachers were also asked to
write down any questions that students asked during the
test. Several problems were identified such as the need for
clarity in wording and for more elaboration of Earth science
context in the assessment items, as well as when some items
relied too heavily on knowledge of Earth science content.
Improvements were made for the problematic items.
Internal Structure Evidence
A field test was conducted in year three with a total of 630
eighth-grade Earth science students in nine middle schools. Of
the nine schools, six were public and three were private
schools. The school sizes ranged from 317 to 944. An average of
50% of the students in the public schools received free or
reduced lunch. Prior to the academic year, training was
provided to the participating teachers on how to utilize the
features of the assessment Web application, iSA. The teachers
had the opportunity to explore iSA features and take the
assessment themselves before giving it to their classes.
Analyses used to examine the internal structure of the test
include: exploratory and confirmatory factor analyses of item
responses, dimensionality (principal component analysis of
Rasch model residuals) and local item independence, and
Rasch item–response maps.
Exploratory and Confirmatory Factor Analyses
The internal structure of the iSA–Earth Science was first
evaluated via exploratory and confirmatory factor analysis. The
PRELIS software package, which is part of LISREL 8, was used
to produce polychoric correlations, which were entered into the
SPSS 18.0 software and subjected to exploratory factor analysis
with principal axis extraction and varimax rotation. The
solution yielded one dominant factor (eigenvalue of 28.9
compared to a maximum of 1.4 for other trailing subfactors).
LISREL 8 (Jöreskog and Sörbom, 1993) was used to perform
confirmatory factor analysis. The results showed that one
dominant factor provided a superior fit to the data.
In addition to factor analysis, the final phase of
development of the iSA–Earth Science was based on item
response theory (IRT). IRT has the advantage of producing
sample-free estimates of item parameters and permitting the
test developer to explicitly target the coverage of the
underlying trait reflected in test performance. There are
numerous IRT models being used in test development and
operational examinations nationally. The most prominent
models for dichotomously scored items (Rasch, 2-parameter
logistic, 3-parameter logistic) vary on the assumptions they
make regarding the influence of guessing behavior by
examinees and whether or not the discrimination of items
on an examination vary. The model that seemed to best fit
the iSA–Earth Science assessment is the 1-parameter logistic
model, also called the Rasch model. The Rasch model is
based on the idea that the probability that an examinee will
get an item correct is a function of the examinee’s ability and
the difficulty of the item. As with other item response theory
models, it has the potential to provide estimates of a
person’s ability that are independent of the specific
examination they have taken and estimates of item difficulty
that are independent of the examinees that attempted the
item. These models have the potential to provide detailed
information about the functioning of items and persons
across the range of ability measured in a test. This model is
currently used in several state achievement testing programs.
To examine and estimate the parameters of the model,
the Winsteps (Version 3.68.2) computer program (Linacre,
2009) was used. The process was as follows: (1) the model
was fit to data and examined to determine if the model
provided a reasonable fit to the data; and (2) as part of the
fitting process, items or persons that appeared to be
inconsistent with model predictions would be discarded.
Item and Person Fit
Random guessing has the impact of increasing the
likelihood that low-ability examinees will obtain a correct
response to difficult items or conversely, high-ability
examinees will fail to correctly respond to an easy item. It
was anticipated that guessing would be minimal, in that the
distracters or foils were selected to represent common
misconceptions among less knowledgeable examinees; a key
goal of the distractor analysis described above was to identify
incorrect options that ‘‘appeared’’ to be effective at
separating more and less knowledgeable examinees. In the
Rasch model, person and item fit statistics provide the best
evidence of randomness in the data. Specifically, we focused
on the mean square (MNSQ) of the infit and outfit statistics
(Linacre, 2009). Values of MNSQ greater than 1.5 would be
80 Lou et al.
FIGURE 1: (a) Sample item for Skill 2.4: Identify flaws in experimental design, before revision. (b) Sample item for
Skill 2.4: Identify flaws in experimental design, after revision.
flagged as potential misfits, meaning that persons or items
were a poor fit to the model (see Linacre, 2009).
Table III presents the infit and outfit, as well as summary
statistics for the test. The infit and outfit statistics measure
the degree to which each item or person fits the model. In
accordance with general practices, we placed more emphasis
on item infit statistics in identifying misfitting items and
persons (Bond and Fox, 2007). Infit is based on the chi-
square statistic with each observation weighted by its
J. Geosci. Educ. 63, 73–85 (2015)
statistical information (model variance) and is more sensitive
to unexpected patterns of observations by persons on items
that are roughly targeted on them (and vice versa). Outfit is
based on the conventional chi-square statistic and is more
sensitive to unexpected observations by persons on items
that are relatively very easy or very hard for them (and vice
versa). A small number of students with infit exceeding 1.50
were removed. No item had infit or outfit statistics exceeding
J. Geosci. Educ. 63, 73–85 (2015) Science Inquiry Skills Assessment 81

TABLE III: Summary statistics for person ability and item Local Independence
difficulty. Local independence was evaluated using a principal
Raw Measure Model Infit Outfit component analysis of the residuals. We also examined the
Score (h) Error relationship between item order and difficulty and item fit, as
MNSQ MNSQ
well as correlations among item residuals (see Linacre, 2009).
Person Ability Local independence was assumed in that the items were
Mean 17.02 0.36 0.44 1.00 1.00 designed to be independent, stand-alone items. Analysis of
correlations between standardized residuals for pairs of items
SD 6.03 1.08 0.08 0.13 0.24
showed that none of the correlations are large in value (with
Max 29.00 3.60 1.02 1.51 2.67 the largest being .20, well below the .70 criteria), indicating that
Min 2.00 -2.85 0.39 0.73 0.30 local dependence is not a challenge for this instrument (Table
Item Difficulty V). For a more detailed discussion of statistics in the table, see
Linacre (2009).
Mean 357.43 0.00 0.09 1.00 1.00
SD 85.58 0.73 0.01 0.11 0.18 Rasch Item–Person Map
Max 508.00 1.52 0.11 1.25 1.44 Figure 2 shows the item–person map for the 30 items
included on the May 2009 field test. The map shows the
Min 176.00 -1.37 0.09 0.79 0.74
range of person abilities on the left side of a vertical line and
item difficulties on the right side of the line. Low ability/
difficulty is represented at the bottom of the map and high
the limit. For a more detailed discussion of statistics in the ability/difficulty at the top. As can be seen on the map, the
table, see Linacre (2009). student abilities and the item difficulties have approximately
normal distributions, and there is a high degree of overlap
Dimensionality between ability and difficulty. The mean item difficulty
The results of the principal component analysis of appears to match the mean person ability, which means that
residuals (differences between expected and actual perfor- the person has a 50% chance of answering the item
mance) from Rasch model predictions were used to correctly. Along most of the range of student ability, there
determine if a common factor was present. If the model are test items that provide a reliable measure of their inquiry
holds, no common factors would be present among the skills, but for the top and lowest part of the ability
residuals. This approach is recommended by Bond and Fox distribution there are few items.
(2007) and Linacre (2009).
The percentage of variance explained by person ability Relationship to Other Variables Evidence
and item difficulty is consistent with predictions of the Rasch Evidence on relationships to other variables was
model, suggesting a good fit of the data to the model (Table determined via correlations of the iSA–Earth Science with
IV). The percentage of unexplained variance in the data is the inquiry skills subtest scores in the Louisiana Educational
also close in value to the level of randomness predicted by Assessment Program (LEAP; Louisiana Department of
the model. The variance associated with the various Education, 2006). These scores are part of the student and
contrasts is similar in magnitude and all are significantly school accountability program in Louisiana and they carry
smaller than that associated with items, a pattern that significant consequences. Teachers on the development
suggests that multidimensionality is not present in these panel emphasized that evidence of a relationship with
data. For a more detailed discussion of statistics in the table, state-mandated tests is critical for building level support of
see Linacre (2009). classroom use of the iSA–Earth Science. To investigate this,

TABLE IV: Standardized residual variance (in eigenvalue units).

Total Raw Variance in Observations Empirical Modeled
Variance Proportion Proportion Proportion
Explained Variance Unexplained Variance of Variance
Total raw variance in observations 39.7 100% 100%
Variance explained by measures 9.7 24.5% 23.1%
Variance explained by persons 4.4 11.0% 10.4%
Variance explained by items 5.3 13.4% 12.7%
Raw unexplained variance (total) 30.0 75.5% 100% 76.9%
1st contrast 1.8 4.5% 5.9%
2nd contrast 1.5 3.8% 5.0%
3rd contrast 1.3 3.4% 4.5%
4th contrast 1.3 3.2% 4.3%
5th contrast 1.3 3.2% 4.3%
82 Lou et al. J. Geosci. Educ. 63, 73–85 (2015)

TABLE V: Correlations between standardized residuals (local iSA–Earth Science scores were correlated with the inquiry
dependence). skills subtests from the grade 7 and grade 8 LEAP tests.
Correlation Between Item Sequence Item Sequence These correlations were .64 and .61, respectively. Both
Standardized Residuals Number Number indicated that the iSA–Earth Science had a high level of
association with these subtests.
0.15 23 27
-0.20 14 24 Consequences of Testing Evidence
-0.19 6 26 Differential item functioning (DIF) across gender was
-0.15 7 12
analyzed to determine if items differentially penalize female
or male students. A difference in difficulty of .5 logits was the
-0.14 6 27 criteria used to determine if an item was problematic.
-0.14 7 17 DIF results showed one item, Item 21, with DIF contrast
-0.13 14 23 of .50 at 0.006 significance of Welch’s t. The value of the
contrast suggests that Item 21 is more challenging for female
-0.13 5 29 examinees. However, this value is small and likely does not
-0.13 15 21 represent a notable difference.
-0.13 4 10
Reliability and Overall Fit
To determine the extent to which the Rasch model
provides an overall fit to the data, we focused on the item

FIGURE 2: Item–person map showing the distribution of student abilities on the left and item difficulties on the right.
The letters and numbers on the right indicate items with associated inquiry skills.
J. Geosci. Educ. 63, 73–85 (2015)
and person separation indexes and reliabilities. The separa-
tion index represents the spread of the abilities or difficulties
and indicates the approximate number of different levels of
difficulty or ability that can be reliably differentiated. A
separation index greater than 1.5 is generally considered
acceptable. We also examined the overall goodness of fit chi-
square.
The person separation and reliability were estimated to
be 2.15 and 0.82 and the item separation and reliability
were estimated to be 7.50 and .98. The overall reliability of
the measure, KR20, was estimated to be .84. These results
suggest that iSA–Earth Science has excellent reliability in
measuring the latent variable of interest science inquiry
skill.
DISCUSSION AND CONCLUSIONS
This paper reports on the development and validation of
iSA–Earth Science, a science inquiry skills assessment, in the
context of middle school Earth science that classroom
teachers can use to assess and monitor their students’
science inquiry skills. The construction and validation of the
iSA–Earth Science followed an iterative and rigorous process
using both classical psychometric analysis and Item Re-
sponse Theory based on the guidelines as articulated in the
Standards for Educational and Psychological Testing (AERA,
APA, and NCME, 1999).
As recommended by the Standards, we examined five
sources of validity evidence: content, response process,
internal structure, relationship to other variables, and
consequences of testing. The results of this study show that
the iSA–Earth Science has strong validity evidence on
content, response process, internal structure, relationship
with other variables, and consequence of testing, and person
and item reliability. The psychometric analyses using both
the classical test development method and Item Response
Theory via Rasch Modeling showed that the instrument was
internally consistent, reliable and did not function differen-
tially for male and female students.
One limitation identified in the item difficulty/person
ability map is the limited number of items at the highest and
lowest end of difficulty levels. One of our future efforts is to
develop more items at these two levels to better separate
students at the highest ability levels and at the lowest ability
levels.
A second issue concerns the question of dimensionality.
The test specifications indicate a complex construct with
several subdimensions. The empirical data consistently
indicated one dominant underlying trait. While this supports
use of the unidimensional Rasch model for these analyses, it
does pose a question as to the utility of the different
subscales. Our recommendation for the user is that the
subscales be used cautiously, much as standard level scores
that are commonly reported in score reports classroom
teachers receive from state testing programs. These scores
tend to be based on few items and are best used as an
‘‘indicator’’ of a specific area in which a student may need
instruction.
Although science inquiry has been the focus of science
reform efforts for decades, teachers have found it difficult to
implement effective inquiry-based science learning in the
classrooms (NRC, 2007). One of the reasons was lack of
understanding of what constitutes authentic and effective
Science Inquiry Skills Assessment 83
science practices. NRC (2000, 2007, 2012) advocates inquiry
assessments as crucial instruments for aiding teachers in
diagnosing their students’ comprehension and competencies
as well as providing feedback about their performance.
The iSA–Earth Science was developed based on a
comprehensive assessment framework with six primary
inquiry skills, each with three to six subskills that are aligned
with the National Science Education Standards, and A
Framework for K–12 Science Education (NRC, 2012). The six
primary skills are: (1) identify questions for scientific
investigations, (2) design scientific investigations, (3) use
tools and techniques to gather data, (4) analyze and describe
data, (5) explain results and draw conclusions, and (6)
recognize alternative explanations and predictions. All
assessment items are integrated within the context of middle
school Earth science.
The iSA–Earth Science instrument has important
pedagogical and research implications. It is hoped that
inquiry skill assessment using a variety of Earth science
inquiry scenarios will encourage teachers to better integrate
inquiry with Earth science content and teach Earth science as
inquiry rather than accumulation of information and facts
only. When students are engaged in more inquiry-based
science, they will be better motivated to learn science,
become scientifically literate, and ultimately contribute to
scientific discoveries in the future. An assessment that is not
gender-biased will encourage both female and male
students to achieve in science learning. Future studies
should also examine other types of possible bias, such as
race and ethnicity or urban and rural environments, to
determine whether the instrument is equally valid in urban
and rural environments and across racial and ethnic groups.
The iSA–Earth Science can also provide opportunities to
evaluate the efficacy of alternative approaches to supporting
inquiry-based science learning and innovative preservice
and inservice professional development programs. Until
now, professional development programs are typically
evaluated via end-of-training surveys that elicit perceived
learning. It is important to measure the actual learning. The
PTI Inquiry Skill Framework as well as the iSA–Earth
Science instrument can be used as a model to develop
inquiry skill assessments that are embedded in other science
content, such as biology, physics, and environmental
science.
The iSA–Earth Science is available free for use at the PTI
project Web site (http://scienceinquiry.cloudapp.net) by
middle school Earth science teachers in their classrooms
and by interested researchers. The online assessment system
is easy to use and provides immediate feedback as well as
archived results to students on their inquiry skills. Teachers
can view basic and comparative students’ scores in a number
of ways including skills and subskills by class, individual
student, and testing time. It is hoped that the results and
feedback from iSA–Earth Science can help teachers and their
students to better understand, track, and improve science
inquiry skills. They can also take the tests themselves to
identify the strengths and limitations of their own science
inquiry skills.
Acknowledgment
This research was supported by the National Science
Foundation under Grant No. 0554556. Any opinions,
findings, and conclusions or recommendations expressed
84 Lou et al.
in this material are those of the authors and do not
necessarily reflect the views of the National Science
Foundation.
REFERENCES
Alonzo, A.C., and Aschbacker, P.R. 2004, March. Value-added? Long
assessment of students’ scientific inquiry skills. Paper presented at
the annual meeting of the American Education Research
Association, San Diego, CA.
American Educational Research Association (AERA), American
Psychological Association (APA), and National Council of
Measurement in Education (NCME). 1999. Standards for
educational and psychological testing. Washington, DC: Amer-
ican Educational Research Association.
Ault, C.R. 1998. Criteria of excellence for geological inquiry: The
necessity of ambiguity. Journal of Research in Science Teaching,
35:189–212.
Barstow, D., and Geary, E., eds. 2002. Blueprint for change: Report
from the national conference on the revolution in Earth and
space science education. Cambridge, MA: Technical Education
Research Centers. Available at http://www.earthscienceed
revolution.org/RevEarthSciEd.pdf (accessed 2 January 2015).
Beeth, M.E., Cross, L., Pearl, C., Pirro, J., Yagnesak, K., and
Kennedy, J. 2001. A continuum for assessing science process
knowledge in grades K–6. Electronic Journal of Science Education,
5(3). Available at: http://wolfweb.unr.edu/homepage/crowther/
ejse/beethetal.html (accessed 19 December 2014).
Biological Sciences Curriculum Study (BSCS). 1962. Processes of
science test. New York: Psychological Corporation.
Black, P., and Wiliam, D. 2004. The formative purpose: Assessment
must first promote learning. In M. Wilson, ed., Toward
coherence between classroom assessment and accountability,
Yearbook of the National Society for the Study of Education,
Part II, 103(2):20–50.
Blanchard, P., and Lou, Y. 2009. Pathways to inquiry—Earth
science: Online tools to strengthen classroom understanding
and integration of science inquiry skills. Paper presented at the
National Association for Research in Science Teaching Annual
International Conference (NARST), Garden Grove, CA, April
17–21.
Bond, T.G., and Fox, C.M. 2007. Applying the Rasch model:
Fundamental measurement in the human sciences, 2nd ed.
Mahwah, NJ: Lawrence Erlbaum Associates.
Burns, J.C., Okey, J.R., and Wise, K.C. 1985. Development of an
integrated process skill test: TIPS II. Journal of Research in
Science Teaching, 22(2):169–177.
Cooper, B.C., Shepardson, D.P., and Harber, J.M. 2002. Assess-
ments as teaching and research tools in an environmental
problem-solving program for in-service teachers. Journal of
Geoscience Education, 50(1):64–71.
Dietz, M.A., and George, K.D. 1970. A test to measure problem-
solving skills in science of children in grades one, two, and
three. Journal of Research in Science Teaching, 7(4):341–351.
Dillishaw, F.G., and Okey, J. 1980. Test of the integrated science
process skills for secondary science students. Science Education,
64(5):601–608.
Downing, S.M., and Haladyna, T.M., eds. 2006. Handbook of test
development. Mahwah, NJ: Lawrence Erlbaum Associates.
Fyffe, D.W. 1971. The development of test items for the integrated
science processes: Formulating hypotheses and defining oper-
ationally [abstract, Ph.D. dissertation]. East Lansing, MI:
Michigan State University. Retrieved from Dissertation Ab-
stracts International, 32, 6823–A.
Germann, P.J., Aram, R., Odom, A.L., and Burke, G. 1996. Student
performance on asking questions, identifying variables, and
formulating hypotheses. School Science and Mathematics,
96(4):192–201.
J. Geosci. Educ. 63, 73–85 (2015)
Greenfield, T.A. 1998. Gender- and grade-level differences in science
interest and participation. Science Education, 81(3):259–276.
Haladyna, T.M. 1994. Developing and validating multiple-choice
test items. Hillsdale, NJ: Lawrence Erlbaum Associates.
Hatton, J., and Plouffe, P.B. 1997. Science and its ways of knowing.
Upper Saddle River, NJ: Prentice Hall.
Hoffmann, L. 2002. Promoting girls’ interest and achievement in
physics classes for beginners. Learning and Instruction,
12(4):447–465.
Jöreskog, K.G., and Sörbom, D. 1993. LISREL 8 [Computer
software]. Chicago, IL: Scientific Software International.
Koballa, T.R., and Chiappetta, E.L. 2005. Science instruction in the
middle and secondary schools: Developing fundamental
knowledge and skills for teaching, 6th ed. Boston, MA: Allyn
& Bacon.
Kutnick, P. 2000. Girls, boys and school achievement: Critical
comments on who achieves in schools and under what
economic and social conditions achievement takes place: A
Caribbean perspective. International Journal of Educational
Development, 20(1):65–84.
Lavinghousez, W.E., Jr., 1973. The analysis of the Biology Readiness
Scale as a measure of inquiry skills required. Paper presented
at BSCS Biology, College of Education, University of Central
Florida, February 25, 1973.
Lewis, E. 2008. Content is not enough: A history of secondary Earth
science teacher preparation with recommendations for today.
Journal of Geoscience Education, 56(5):445–455.
Linacre, J.M. 2009. WINSTEPS Rasch measurement and analysis,
Version 3.68.2. [Computer software]. Beaverton, OR: Win-
steps.
Louisiana Department of Education. 2003. Louisiana grade level
expectations. Available at http://www.louisianabelieves.com/
docs/academics/grade-5.pdf?sfvrsn=2 (accessed 19 December
2014).
Louisiana Department of Education. 2006. LEAP assessment guide:
English language arts, mathematics, science and social studies,
grade 8 (LEAP science, grade 8), p. 3–4. Available at http://
www.doe.state.la.us/lde/uploads/9844.pdf (accessed 7 Decem-
ber 2014).
Ludeman, R.R. 1975. Development of the science processes test
(TSPT) [abstract, Ph.D. dissertation]. East Lansing, MI:
Michigan State University. Retrieved from Dissertation Ab-
stracts International, 36, 203–A.
McLeod, R.J., Berkheimer, G.D., Fyffe, D.W., and Robison, R.W.
1975. The development of criterion-validated test items for
four integrated science processes. Journal of Research in Science
Teaching, 12:415–421.
Means, B., and Haertel, G. 2002. Technology supports for assessing
science inquiry. In National Research Council, eds., Technol-
ogy and assessment: Thinking ahead—Proceedings of a
workshop. Board on Testing and Assessment, Center for
Education, Division of Behavioral and Social Sciences and
Education. Washington, DC: National Academies Press.
Molitor, L.L., and George, K.D. 1976. Development of a test of
science process skills. Journal of Research in Science Teaching,
13:405–412.
National Research Council (NRC). 1996. National science education
standards. Washington, DC: National Academies Press.
National Research Council (NRC). 2000. Inquiry and the national
science education standards. Washington, DC: National
Academies Press.
National Research Council (NRC). 2007. Taking science to school:
Learning and teaching science in grades K–8. Washington,
DC: National Academies Press.
National Research Council (NRC). 2012. A framework for K–12
science education: Practices, crosscutting concepts, and core
ideas. Committee on a Conceptual Framework for New K–12
Science Education Standards. Board on Science Education,
J. Geosci. Educ. 63, 73–85 (2015)
Division of Behavioral and Social Sciences and Education.
Washington, DC: National Academies Press.
National Research Council (NRC). 2013. The next generational
science standards. Washington, DC: National Academies
Press.
National Research Council (NRC). 2014. Developing assessment for
the next generational science standards. Washington, DC:
National Academies Press.
Okey, J.R., Wise, K.C., and Burns, J.C. 1982. Test of integrated
process skills (TIPS II). Unpublished manuscript, Department
of Science Education, University of Georgia, Athens, GA.
Orion, N., and Kali, Y. 2005. The effect of an Earth-science learning
program on students’ scientific thinking skills. Journal of
Geoscience Education, 53(4):387–393.
Polit, D., and Beck, C. 2006. The content validity index: Are you
sure you know what is being reported? Research in Nursing and
Health, 29(5):489–497.
Popham, W.J. 1980. Evaluation in education. Berkeley, CA:
McCutchan Publishing.
Radford, D.L., DeTure, L.R., and Doran, R.L. 1992, March. A
preliminary assessment of science process skills of preservice
elementary teachers. A paper presented at National Associa-
tion of Researchers in Science Teaching, Boston, MA.
Riley, J.W. 1972. The development and use of a group process test
for selected processes of the Science Curriculum Improvement
Study [abstract, Ph.D. dissertation]. East Lansing: Michigan
Science Inquiry Skills Assessment 85
State University. Retrieved from Dissertation Abstracts Inter-
national, 33, 6200A–6201A.
Roberts, R., and Gott, R. 2003. Assessment of biology investiga-
tions. Journal of Biological Education, 37(3):114–121.
Science Education for Public Understanding Project (SEPUP). 1995.
Issues, evidence and you: Teacher’s guide. Berkeley, CA:
University of California, Lawrence Hall of Science.
Smith, K.A., and Welliver, P.W. 1990. The development of a science
process assessment for fourth grade students. Journal of
Research in Science Teaching, 27(8):727–738.
Tannenbaum, R.S. 1971. Development of the test of science
processes. Journal of Research in Science Teaching, 8(2):123–136.
Tobin, K.G., and Capie, W. 1982. Development and validation of a
group test of integrated science processes. Journal of Research in
Science Teaching, 19(2):133–141.
Walbesser, H.H. 1965. An evaluation model and its application.
AAAS Miscellaneous Publication No. 65-9. Washington,
DC: The American Association for the Advancement of
Science.
Walbesser, H.H., and Carter, H.L. 1970. The effects of test results of
changes in task and response format required by altering test
administration from an individual to a group form. Journal of
Research in Science Teaching, 7:1–8.
Wenning, C. 2007. Assessing inquiry skills as a component of
scientific literacy. Journal of Physics Teacher Education Online,
4(2):21–24.