Artificial Intelligence Based Student Learning

Evaluation Tool
G. Pankaj Jain, Varadraj P. Gurupur, and Eileen D. Faulkenberry

Abstract In this paper we describe a tool coined as Artificial

Intelligence Based Student Learning Evaluation Tool (AISLE).
The main purpose of this tool is to improve the use of artificial
intelligence techniques in evaluating a students understanding of
a particular topic of study using concept maps. Here we calculate
the probability distribution of the concepts identified in the
concept map developed by the student. The evaluation of a
students understanding of the topic is assessed by analyzing the
curve of the graph generated by this tool. For the purpose of
experimentation we have taken into consideration concept maps
developed by students in the area of mathematics.
Keywords-probability distribution; concept maps;

I.

INTRODUCTION

We are in the process of developing a tool to evaluate

student learning using concept maps [1][2]. Here, a student
would be given a topic to learn and build a concept map of
their understanding of the topic. This tool, coined as Artificial
Intelligence Based Student Learning Evaluation Tool (AISLE),
would then evaluate the concept map and assess if the student
has captured enough concepts from the given topic. This will
help the instructor in evaluating a students understanding of
the topic.

It will provide a better understanding of the student

learning process which will have practical curriculum
and classroom applications for educational
psychologists[17].

The project will provide the school districts in

northeast Texas with a new educational tool to use in
their classrooms.

Overall, AISLE can assist with identifying the best

ways for students to improve their learning skills.
II.

METHODOLOGY

The method used by the tool to evaluate student

understanding of specific topics as discussed in the class is
different than the regular methods used such as quizzes, oral
presentations, and projects. While most of the instructors
attempt to measure a students understanding of the topics
discussed in the class by evaluating the work done by the
students, this tool automates the task of evaluation. The method
used by the tool can be used by having a deeper measurement
of a students understanding of the domain in discussion by
inspecting the areas of the domain concepts that the student
may be interested in [1][2].

The objectives of this project are as follows:

To develop a tool based on artificial intelligence

techniques to develop curriculum and course
materials[1].

To develop a tool that understands student psychology

in terms of the learning process[1][2] undertaken by
the student using concept maps.

A&M-Commerce is the largest producer of educational

leaders in northeast Texas, with established, active and
engaged alumni. Significantly, these leaders are in school
districts that span the wide-range of socioeconomic conditions
including urban poor (e.g., Dallas Independent School
District), rural poor (e.g., Paris, TX Independent School
District), and suburban (e.g., Mesquite Independent School
District) [3][4]. Germane to all school districts of northeast
Texas is the increasing proportion of English Language
Learners (ELL), especially of Hispanic ancestry. A necessary
outcome of our efforts will be improved knowledge of STEM
and STEM processes, and communication among education
leaders from these diverse, yet interrelated, communities.
This project can have the following impact on the academic
community:

Figure 1. Overview of the methodology involved in using

this tool.
The above figure provides a brief description of the
methodology involved in using our tool. As reflected in Figure
1, this methodology involves the following steps:

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 751

Students develop concept maps after studying from

the prescribed study material.

These concept maps are converted into XML-based

documents.

The XML analyzer module of our tool extracts the

concepts embedded in the XML document.

The analysis module makes an assessment of the

importance of the concepts captured by the students
and provides a summary of the results to the userinterface module.

The instructor perceives the result from the user interface

and makes a judgment on the study carried out by the student.
III.

EXPERIMENTATION

As mentioned before, AISLE is a tool which helps the

instructors to analyze a students understanding of a particular
topic. This analysis is based on the concept maps developed by
the students for a particular topic under study. The instructor
uses the concept map and runs it on AISLE to receive the
statistics based on Probability Density Function [3][4][7]. To
generate this statistics in AISLE we use the hierarchy of the
concepts involved in the concept map. For example, consider
the following concept map and how the statistics can be
obtained for this particular concept map.

Figure 2. Concept map 1 for the instructor.

We first categorize the concepts into different levels based on
their presence in the hierarchy of concepts. The table below
identifies the level associated with every individual concept in
the concept map [18][19].

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 752

TABLE I.

CATEGORIZATION OF CONCEPTS BASED ON INDIVIDUAL

LEVEL OF HIERARCHY.

LEVEL 1 Concepts
Operator
Concrete model
Representation
Fraction as a number
Quotient

Level 2 Concepts
Operator
Concrete model
1.9
Fraction as a number
"Every time I make some money I deposit
1/5 of it into my savings account."

Measurement
Part / whole
Operations
Partitive

Linear model
Discrete model
Pi
"If I have 3 cookies and give each person
1/2 cookie, how many people get a treat?

Topic (level 0 concept)

Rational Numbers

Ratio
Decimal as a number
Division

Quotient
"If 3 cookies makes up of my total bag of
cookies, how many cookies are in a whole
bag?"
Measurement
Addition and Subtraction
Part/whole
Area /array model
Partitive
Multiplication
Ratio
Decimal as a number
"A recipe calls for 2 cups of flour for every
cup of milk. What is the ratio of flour to
milk?"
Division
concept in the hierarchy. From these random scores we
In our process, each student develops a concept map which is
calculate the mean of concept scores for all the concepts and
translated into an XML file[8][9][10].Based on the depth of the
the standard deviation for these scores using the required
hierarchy of concepts we have developed a random scoring
formula.
system that provide a discrimination of random scores for each
concept at a particular level represented by the student. A
A. Using Z Scores of concept to Perform Analysis
particular concept may be related to another concept with a
Based on the aforementioned scores we calculate the z
relation, this relation is pivotal in identifying the level of
scores for every concept in the concept map by using the
hierarchy involved in the concept map. The more concepts a
formula [6] [14] [15] below:
student represents in the concept maps with its properties, the
more we believe the student understands the topic under study.
Thus the number of concepts identified in a concept map plays

a significant role in identifying the depth of knowledge for that

(1)

particular topic conceived by the student. This analysis is

carried out in two steps:
1) First, a random score [3][4] is given for each concept
represented in the concept map for the concepts
identified at the topmost level.

Where 

2) Second, a random score value is derived iteratively for

all the lower level concepts in the hierarchy.

is known as z-value of concept or z-score of concept.

Now, for each of the concepts available in the concept map,

a random score is allotted with regard to the level of the

Now that the z-score of a concept is calculated for every

concept in the concept map [1][2], the random score [2][6] is
now standardized with the z-scores. These standardized values
are tabulated as shown below in Table II.

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 753

TABLE II.

Concept
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22

23
24
25

26
27
28
29
30
31
32
33
34

35

STANDARDIZED VALUES OF Z SCORES FOR EVERY CONCEPT

IN THE CONCEPT MAP SHOWN IN FIGURE 2.

Name of concept

Operator
Concrete model
Representation
Fraction as a number
Quotient
Measurement
Part/whole
Operations
Partitive
Ratio
Decimal as a number
Division
Rational Numbers
Operator
Concrete model
1.9
Fraction as a number
"Every time I make
some money I deposit
1/5 of it into my savings
account."
Linear model
Discrete model
Pi
"If I have 3 cookies and
give each person 1/2
cookie, how many
people get a treat?

Quotient
"If 3 cookies makes up
of my total bag of
cookies, how many
cookies are in a whole
bag?"
Measurement
Addition and subtraction
Part/whole
Area / array model
Partitive
Multiplication
Ratio
Decimal as a number
"A recipe calls for 2 cups
of flour for every cup of
milk. What is the ratio of
flour to milk?"
Division

-1.68
-1.58
-1.49
-1.39
-1.30
-1.20
-1.11
-1.01
-0.92
-0.82
-0.72
-0.63
-0.44
-0.34
-0.25
-0.15
-0.06
0.03

1.

If the z-score of concept is negative (z < 0), this

means that the concept specified by the student to that
value is located below mean of scores of all concepts.

2.

If z- score of concept is positive (z > 0), this means

that the concept specified by the student to that value
is located above the mean of scores of all concepts.

The standard form of Probability Density Function [7][16]

in AISLE is given by equation



 

 

Where  
The above equation (2) clearly represents that Mean of
scores [3][4] and Standard deviation [4] of scores play an
important role in obtaining the normal curve for the concepts
covered at a particular level of hierarchy.
As the scores are standardized with the z-scores of concept,
the probability density equation for AISLE has to relate with
the z-score of concept. The relation of z-score of concept and
Probability Density Function for AISLE can be derived as
shown below:
We know from equation (1):

0.12
0.22
0.31
0.41

= ( +Meanofconceptscore

0.50
0.60
0.69

(2)

(1a)

Now if we differentiate the equation (1a) on both sides, we

get


0.79
0.89
0.98
1.08
1.17
1.27
1.36
1.46
1.55

i.e. 

(1b)

Hence integrating the Probability Density Function with

respect to on both sides,
 )



(3)

1.65

As depicted in Table II, z score of concept can be positive

or it can be zero, or sometimes it can be negative. The
significance of the z score of concept is as follows:

Solving this differential function and expanding the

exponential function, we get

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 754

 ( ) =

-.]

(4)

The above equation 4 is implemented to calculate the area

of the normal probability density function for each of the
concepts that are from the concept maps. The results are
illustrated in Table III:
TABLE III.

Concept
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

19
20
21
22

23
24
25

26
27

PROBABILITY VALUES OF THE CONCEPTS LISTED IN FIGURE 2

Name of concept
Operator
Concrete model
Representation
Fraction as a
number
Quotient
Measurement
Part/whole
Operations
Partitive
Ratio
Decimal as a
number
Division
Rational Numbers
Operator
Concrete model
1.9
Fraction as a
number
"Every time I make
some money I
deposit 1/5 of it
into my savings
account."
Linear model
Discrete model
Pi
"If I have 3 cookies
and give each
person 1/2 cookie,
how many people
get a treat?

Quotient
"If 3 cookies makes
up of my total
bag of cookies,
how many cookies
are in a whole
bag?"
Measurement
Addition and
subtraction

Instructor


















28
29
30
31
32
33
34

35

Part/whole
Area / array model
Partitive
Multiplication
Ratio
Decimal as a
number
"A recipe calls for
2 cups of flour for
every cup of milk.
What is the ratio of
flour to milk?"
Division








The above table represents the concept number with its

concept name. The area under the normal probability of the
concept is calculated with the corresponding equation (4) as
listed above. The graph is plotted based on the probability
value associated with each concept. Here we are taking into
consideration concepts existing at level 1, the topic included in
it , and its level 2 concepts. This is simply because the level 1
has fewer concepts when compared with the level 2 concepts.
The topic is always centered with the mean. The graph is as
shown below.





Figure 3. Graph of Number of concepts and probability of
concept for Concept map 1








The above graph shows the probability distribution of

concepts. By looking at the graph we can easily say that the
number of concepts in the concept map 1 is 35 instead of 25.
This is because the property of the concept at a particular level
may be related with another concept of the same level or at a
different level. For example, in the above concept map 1, the
concept Quotient which exists at two levels is linked with
the concept Division which is again related with the
concepts Partitive and Measurement with its different
properties. This creates a situation where a particular concept
may be at both a higher level as well as a lower level in the
hierarchy of concept maps. Hence for understanding the topic
the student has to know the relation between the concepts. In
our experiment we choose 20 different concept maps each

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 755

developed by an individual student. Out of this, we have

choosen 8 interesting concept maps to analyze the students
understanding of the topic in discussion. Figure 7 represents
the probability distribution of these concept maps.

ACKNOWLEDGMENT
We would like to thank our dear friend Dr. Mario Eraso from
the Department of Curriculum and Instruction at Texas A&M
University Commerce for helping the research team in
finding necessary contacts.
REFERENCES
[1]
[2]

[3]
[4]
[5]

[6]

[7]

Figure 4. Probability distribution of 8 different concept

maps used for experimentation.
By taking a closer look at the set of curves in the above figure
we can conclude that the curve that corresponds to the concept
map developed by the instructor is taken as a standard for
measuring the direction in which the curve is leaning. In the
above figure we can clearly perceive that concept map 521003 has a curve that leans towards the left and the concept
map 521- 006 has a curve that leans more towards the right.
This indicates that the student who developed 521-006 has a
deeper understanding of the topic based on the depth of the
hierarchy depicted in the concept map. However, he may have
missed on a few properties that are used to link between the
individual concepts. On the contrary, in concept map 521-003
we can possibly identify that the student has not indicated a
deeper understanding of the concepts identified in the concept
map. In other words, for example, a student may have a few
concepts identified in the concept map but has not indicated
any child nodes for those concepts.
IV.

CONCLUSION AND FUTURE WORK

In this paper we have described our tool for the purpose of

identifying the level of a students understanding of a
particular topic using concept maps. AISLE has been
developed using Java and XML parsers associated with it to
extract the necessary information from the concept maps. Now
that we have developed a method to assess a students
understanding of the topic in study using Probability Density
Function we would like to advance this science by using
Markov Chain analysis. Overall, we believe that our tool and
the method associated with it will be useful for instructors in
identifying and assessing their ability to induce good
understanding of topics and improve their teaching methods.

[8]
[9]

[10]

[11]

[12]

[13]
[14]
[15]

[16]

[17]
[18]

Shui-Cheng Lin, A new structural knowledge based on weighted

concept maps Computers in Education, Vol. 1, pp. 679-680, Dec. 2002.
R. Castles, Knowledge Maps and their pplications to students and
faculty assessment, Frontiers in Education Conference, pp. S4A-9S4A-14, Oct. 2008.
A. L. Douglas,G.M.William,A.W.Samuel, Statistical techniques in
Business and Economics,Fifteen Edition,Mcgraw Hill,2012.
A.Papoulis, Probability Random Variables,Mcgraw Hill,1965.
H.Wang,Control of conditional output probability density functions for
general nonlinear and non-gaussian dynamic stochastic systems,IEEE
proceedings of Control Theorey and Applications,pp.55-60,Jan.2003.
Y.Zhongyum.Optimization model of credit asset portfolio based on zscore,Management science and Industrial Engineering,pp.112115,Jan.2011.
M.Simon,On the probability density function of squared envelope of
sum of random phase vectors,Communications IEEE
transactions,pp.993-996,Sept.1985.
Y.Zhou,A run time adaptive and code size efficient XML
parser,Computer software and applications,pp.18-21,Sept.2006.
C.K.A.Rangan,A generic parser to parse reconfigure XML files,
Recent advances in Intelligent Computational Systems,pp.823827,Sept.2011.
H.Zhang,Schemas extraction for XML documents by XML element
sequence patterns,Information Science and Engineering,pp.50965099,Dec.2009.
G.Zhenghong,P.Yinfei,Z.Yang,C.Kenneth,A high performance schemaspecific XML parser,Third IEEE International Conference on e-science
and Grid Computing,2007.
Y.Zhou,Q.Mingbin, A run time adaptive and code size efficient XML
parser,Proceedings of 30 th Annual International Computer Software
and Application Conference,IEEE 2006.
W.Zhang and A.V.Robert, An adaptive XML parser for developing high
performance web services,IEEE 2008.
C.Graff,Z-score transformation of T-wave mprphology values to a
standardized scale,Computing in Cardiology,pp.737-740,Sept.2011.
J.Mcqueen,Some methods for classification and analysis of multivariate
observations,Proc. 5 th Symp. Math. Statistics. Prob., pp.281297,Sept.1967.
D.A.Hill,Probability density function of power received in a
reverberation chamber, IEEE Trans.Electromagn. Compat., vol
50,pp.1019-1019,Nov.2008.
W.Wang,D.Yisheng,Teaching thinking and experiment of database
system course, ZheShe edition, 2003,vol 2,pp.23-28.
A.J. Caas, G. Hill, R. Carff, N. Suri, J. Lott,T. Eskridge, G. Gmez, M.
Arroyo, & R. Carvajal, CmapTools: A knowledge modeling and sharing
environment. In A. J. Caas, J. D. Novak, & F. M. Gonzlez (Eds.),
(2004), Concept maps: Theory, methodology, technology. Proceedings
of the first international conference on concept mapping (Vol. I, pp. 125133). Pamplona, Spain: Universidad Pblica de Navarra.

[19] V.P. Gurupur, S. C. Suh, R. R. Selvaggi, P. R. Karla, J. S. Nair, and S. Ajit,

An Approach for Building a Personal Health Information System Using
Conceptual Domain Knowledge, Journal of Medical Systems, 2012, Vol. 36,
Issue 6, pp. 3685 3693.

978-1-4673-6110-1/13/$31.00 2013 IEEE

Technische Universitt Berlin, Berlin, Germany, March 13-15, 2013
2013 IEEE Global Engineering Education Conference (EDUCON)
Page 756