Performance Analysis of Scheduling Techniques
in Distributed Parallel Environments
Giulianna Marega Marques, Ricardo José Sabatine, Kalinka R. L. J. Castelo Branco
LAS – Laboratório de Arquitetura e Sistemas Computacionais
UNIVEM – Centro Universitário “Eurípides Soares da Rocha” de Marília
Marília – São Paulo – Brasil – Caixa Postal 2041
{giuliannamm, sabatine, kalinka}@univem.edu.br
Abstract - The objective of this paper is to offer alternatives of
process scheduling and to compare their performances. The
environments obtained were considered by applying parallel
processing of medical images using the comparison of their
performance's time to measure them.
I. INTRODUCTION
The process of obtaining load index is not common
considering the dynamic nature and does not determine the
executed applications, therefore good load indexes can
improve the global performance observed in the system.
Traffic bigger than that expected can be generated and as a
result hinders the system performance as a whole. It depends
on the load frequency information collected for the calculation
of the index.
A distributed parallel computational system is usually
compound by processing elements both configurationally and
architecturally heterogeneous. This characteristic affects the
results of the load indexes and consequently the load
scheduling, resulting either in the improvement or not of the
platform’s final performance as a whole.
As there was no existing literature on the subject, Branco
[3] proposed a performance index that could provide
information on the work load and the operation situation of
each system element involved in the process, considering
different sorts of heterogeneity.
From this information, the importance of the process
scheduling in the distributed parallel environments can be
observed, as well as the obstacles that can be found to achieve
a good performance.
Thus, the objective of this article is to offer alternatives of
process scheduling aiming at load index collection and to
compare their performances. Therefore, it was necessary to
provide instruments to the message passage library so that its
process scheduling could have a basis for the load and
performance indexes to carry out load scheduling. The
environments achieved were considered by applying parallel
processing of medical images using the comparison of their
performance time to measure them.
978-1-4244-1706-3/08/$25.00 ©2008 IEEE.
II. PERFORMANCE AND LOAD INDEXES
A load index is a metric that quantifies the workload
submitted to a system resource [9][15] and the objective is to
indicate if the resource analyzed is idle, moderate, or
overloaded [3].
If the load is attributed to the resource, it can be observed
that the resource is idle when its workload is inexistent or has
a value which is too small, and then it is able to receive a load.
When the resource is considered moderate, the workload is
regular, and therefore the resource can still receive more
workloads. When a resource receives a workload that goes
over a determined limit, it is overloaded and therefore must
transfer and is not able to receive a load.
A load index can be defined as a numerical variable, entire
and not negative which has a value of zero when the resource
is idle and, as the load of this resource increases, its value is
added [9][15][3].
The load index quality is related to the process scheduling
performance as its objective is compound by the scheduling
algorithm. Therefore, the way the system load information is
collected and used and the periodicity of the collection
influences the efficiency of the load balancing.
In distributed parallel computational systems, it is of utmost
importance to be aware of the communication. If the
frequency loads observed in the processing elements are not
coherent with the real necessity, it is likely that there will be
undesired results.
If the load is often updated, an overload occurs in the
interconnection network and the general performance of the
system is reduced; but if it is not often updated, the load
balancing does not have the real information and will be
carried out in a wrong way, hindering the performance of the
system. Therefore, the purpose of the load index is to foresee
the future of the load behavior based on the current/past
behavior and to provide this information to the scheduler to
carry out load balancing [9][25][3].
Many varieties of load indexes can be mentioned, among
them the CPU row length (instantaneous), CPU row middle
length in a determined time, utilization of the CPU, response
time, response time normalized, quantity of available memory,
rate of the context load, and others [9][25][3]. On the whole,
they can be distributed into groups based on the size of the
access row to the resource, in percentage of resource
utilization, and in the execution/answer time [9] [25].
Most of the load indexes in the literature are for
environments with homogeneous configuration and
architecture, and indexes for architectural and configurable
heterogeneous environments can not be found easily.
As there was no information found in the literature
concerning the load index for heterogeneous environments,
Branco [3] proposed a performance index that can provide
information regarding the work load and the situation of the
operation of each element of the system involved in the
process, considering the configurational, architectural, and
temporal indexes [4].
A good performance index, such as the load index, must be
able to estimate the future by means of actual values and past
factors and then its base must be established in the load index
to achieve a good performance index. Another important
characteristic of the performance index is the time it has been
used, therefore as work loads are volatile, consequently the
load indexes and performance are also [3].
The types of applications usually considered in the
scheduling of distributed parallel applications are [9] [15]
[25]:
• CPU-Bound: applications with a high prosecution by
processing a small activity of in and out (I/O);
• Memory-Bound: applications with a high prosecution by
memory;
• I/O-Bound: applications with a high prosecution by in and
out. They are also called Disk-Bound applications, which
are defined as applications with a high prosecution by disc,
i.e., they need a lot of access to the disc, both for reading
and writing;
• Network-Bound: applications with a high prosecution by
communication between processes, which results in having
large traffic in the network.
The performance index relies on the Euclidian distance
between the point of origin (when the machine is idle) and the
result point, between the load values of the machine before
receiving a certain application and the load vector already
imposed on the application load. The machine which is more
suitable to receive the application is the one that achieves the
smaller Euclidian distance [3].
Taking into account the relationship among the different
resources that make up a machine, and considering that the
allocation of the processes is carried out in a balanced way,
the performance index can be achieved using equation (1):
(1)
Therefore, one of the main factors of obtaining a good
computational performance of the distributed platform is not
only the load index, but also how it is achieved, because if its
use does not have a good structure, it can degrade the platform
performance as a whole, when it should improve it.
Since the load collection and the period of this collection
affects the use of the load indexes, the utilization of the agents
can improve significantly in the collection stage, as well as the
calculation of these indexes.
III. MOBILE AGENTS
A mobile agent can be characterized as a program that acts in
an autonomous way, searching for an objective previously
programmed. Taking this into account, it can utilize
interactions with other agents or mobile environments, if
necessary [13]. In other words, it is free to transit among the
hosts of an inter-communication network carrying its code and
condition to the other environment, where execution can
restart [18].
Investigations based on using mobile agents to implement
various mechanisms in environments on a large scale can be
found in the literature, such as resource management,
discovery of resources and services [11], and scheduling of
functions [5].
The motivation and interest for mobile agents apply to the
benefits found in the distributed system development when it
is used. The benefits can be synthesized in the following ways
[2][17]: reduction in the network traffic; overcoming latency;
autonomous and asynchrony execution; support for the mixed;
robustness and tolerance to failure; interaction with systems in
real time; instantaneous extension of services; paradigm of
software development and updates.
Most of the disadvantages of mobile agents listed in the
literature are concerned with security and complexity. The
problems regarding security can be of three types: showing
information, negation of service, and corruption of
information [14]. However, issues concerning security will not
be taken into consideration in this article as this is not the
objective of this paper.
IV. RELATED WORK
In paper, [6] the proposed load balancing framework
technique is described using a mobile agent to distribute loads
among nodes in a homogeneous cluster. Instead of balancing
the load in clusters by process migration, or by moving an
entire process to a less loaded computer, the authors made an
attempt to balance loads by splitting up processes into separate
jobs and then balancing them to nodes by our novel load
balancing method. Load balancing protocol was implemented
using AgletTM, a Java-compliant mobile agent platform from
IBM. Although it cannot be claimed that a mobile agent
approach will always outperform messages passing based load
balancing, such an alternative approach to designing load
balancing protocols has several advantages. Preliminary
experimental results demonstrated that the proposed
framework is effective in some points.
The aim of the paper [7] was to apply the mobile agent
technology to provide better scheduling to MPI applications
executing a cluster configuration. According to the authors
this approach could enhance the load balancing of the parallel
processes in a distributed cluster environment. MPI in clusters
of heterogeneous machines could lead parallel programmers to
obtain results which are frustrating, mainly because of the lack
of an even distribution of the workload in the cluster. As a
result, before submitting a MPI application to a cluster, a
JOTA mobile agent [7] approach to acquire more precise
information of the machine workload was used. Therefore,
with a more precise knowledge of the load and characteristics
in each machine, lightweight workstations to form a better
cluster were used. The empirical results indicate that less
elapsed time can be spent when considering the execution of a
parallel application using the agent approach in comparison to
an ordinary MPI environment.
V. IMPLEMENTATION
The distributed systems must provide an improvement in
performance mainly concerning the execution time, utilization
of resources, communication in the inter-communication
network and mixed hardware and software.
Analyzing the advantages to use mobile agents in
distributed environments, it is believed that by using a
collection of loads and performance indexes in these
environments can provide a reduction in traffic in the network
and, consequently, an improvement in the general
performance of the system in question.
To evaluate the performance of scheduling processes that
use load indexes collected by mobile agents in distributed
computational systems, the following is necessary: to change
the way of scheduling carried out by the library of passage
message JPVM (Java Parallel Virtual Machine) [8], as its
standard is the round-robin; to develop a mobile agent
(PAgent – Performance Agent) using the Java programming
language in the μCode (muCode) environment [22], to
develop the resource monitor (PRM – Performance Resource
Monitor), which has been incorporated to the library, and to
compare the scheduling achieved with the scheduling round-
robin [19].
To initiate the modified JPVM daemon in each machine that
composes the system, the PRM is started. These PRMs are
responsible for collecting the raw load from the processor,
from the memory, from the disc and from the network by the
dstat command (Linux pack installed to evaluate the
performance of the machine resources). They are also
responsible for the performance and load index calculation of
these resources helped by standardized benchmarks to deal
with the environment heterogeneity, if it exists.
When a machine is introduced into the distributed virtual
parallel environment (as shown in Figure 1), a message is
discharged, requesting that the servant software μCode must
be started in a determined gate connection. After introducing
all the machines, the PAgent is started by the master machine
as it goes through the environment. Therefore, the PAgent is
responsible for collecting the performance and load indexes
calculated by the PRMs and for judging which machines are
able to share as load receivers in the load distribution process.
Figure 1: Distributed Parallel Environments using JPVM
PAgent
Parallel medical image processing requires quality and
quantity of hardware resources, i.e., strong computational
power. Datum loss, precision and short processing space time
is not allowed [1]. This has made medical image processing an
interesting application in terms of evaluating the real gain
achieved when it is paralleled [18].
The basic requirements of a parallel image processing
system consists of an infra-structure compound essentially of
adequate datum distribution and communication functions that
can efficiently execute any image algorithms [1].
Edge techniques of softening and detention were chosen in
order to observe the cost/benefit relation of the application of
distributed parallel computation technology when compared to
scheduling with and without help of mobile agents.
The softening techniques are used to reduce noise and to
remove small details of an image before the segmentation
[12][20]. A common softening technique is the median filter
that consists of substituting the value of a pixel determined by
the median value of the neighborhood. The median value is
the central value achieved when the pixels of the
neighborhood are in order.
The Edge detection is another example of algorithm that
uses operations based on the neighborhood. It is mainly used
when one wants to know the size and the form of the objects
represented in the image [21].
The Edge detection process uses the gradient concept to
enhance the points that show a large difference with their
neighborhood. For an image f(x,y) where x and y are the
spatial co-coordinating – line and column – the gradient of f in
the co-coordinating x and y can be defined according to
equation 2 and its magnitude according to equation 3.
ª ∂f º
« »
(2) ∇f = « ∂x »
« ∂f »
«¬ ∂y »¼
 1
the image sort and a larger neighborhood indicates a better
2 2 processing result.
ª§ ∂f · § ∂f ·
2 º
(3) ∇f = mag(∇f )«¨ ¸ + ¨¨ ¸¸ » 15 executions of each algorithm were carried out and the
«¬© ∂x ¹ © ∂y ¹ »¼ averages of execution time in the following scenarios were
achieved:
The gradient concept mentioned was introduced using Sobel (I) Homogeneous environment using JPVM;
operators [21], which calculate the approximate absolute value (II) Homogeneous environment using JPVM
of the gradient in each point of the image analyzed, showing instrumented (with process scheduling using mobile
that the areas with spatial frequency have a high value and agents);
correspond to the image edges [12]. (III) Heterogeneous environment using JPVM; and
The selection criterion of these algorithms is based on how (IV) Heterogeneous environment using JPVM
instrumented (with process scheduling using mobile
many resources are used. The median filter requires an
agents).
intensive memory use, due to the utilization of a sub-vector,
while the Sobel operator requests an intensive processor use, In Figure 2 and Figure 3, it can be observed that the
as successive multiplications are carried out. medium execution time of Sobel processors and median filter
The parallel of the algorithms is based on dividing images algorithms were close both in the homogeneous environment
in parts distributed by processors, processing at the same time using the standard JPVM and in the homogeneous
many parts of a same image. The master host divides and environment using the instrumented JPVM.
distributes the datum to the other hosts; each one processes its
received part and then sends it to the master that recomposes
the image. Hom ogeneous: JPVM - Standard JPVM - PAgent

50000
40000
VI. RESULTS
Exe c ut io n 30000
A parallel homogeneous and heterogeneous distributed T ime ( ms )
20000
environment has been achieved. The homogeneous 10000
environment consisted of seven homogeneous 4 2.7GHz IBM 0
Pentium machines with 512MB of memory. Moreover, the M 4P M 7P M 11P M 14P
pro c e ss e s
heterogeneous environment consisted of seven machines: a 4
2.66GHz Pentium with 1024MB of memory, three IBM 4
Figure 2: Comparison of average execution time of the median
2.7GHz Pentiums with 512MB of memory, a 4 1600 MHz filters in scenarios I and II.
Pentium with 256MB of memory, a 4 1600 MHz Pentium with
256MB of memory, and a 3 733MHz Pentium with 256MB of
Hom ogeneous: JPVM - Standard JPVM - PAgent
memory. All the machines were interlinked by a network
Ethernet of 100 Mbps, and used Linux Kernel 2.6. 20000
To analyze the performance of the environments, the
15000
execution times of the algorithms of parallel medical image E xe c ut io n
10000
processing were compared. First of all, a message passage T im e ( m s )

library JPVM standard was used and after the JPVM 5000

instrumented to provide the processes scheduling. The 0

algorithms used were: S4P S7P S11P S14P
pro c e s s es
• (M4P) median with 4 processes;
• (M7P) median with 7 processes; Figure 3: Comparison of average execution time of the edge
• (M11P) median with 11 processes; detection filters in scenarios I and II.
• (M14P) median with 14 processes;
In Figure 4 and Figure 5, the marked difference between the
• (S4P) Sobel with 4 processes;
average execution time of the algorithms in the heterogeneous
• (S7P) Sobel with 7 processes;
environment when compared to the scheduling of the standard
• (S11P) Sobel with 11 processes;
JPVM and the instrumented JPVM can be observed. The
• (S14P) Sobel with 14 processes.
profit achieved by means of the instrumented JPVM in the
heterogeneous environment was:
A mammography image was used in a TIFF (Tagged Image
File Format) format with a resolution of 16 bits of elevated • 37.95% when the median filter algorithm was performed
with 4 processes;
size, almost 21 Megabytes. The median filter was evaluated
• 8.09 % when the median filter algorithm was performed
with a template of size 7x7 and by using the algorithm of
with 7 processes;
ordering “quicksort”. The edge detection filter was executed
• 29.72% when the median filter algorithm was performed
with a template of size 11x11. The template size is what
with 11 processes;
defines the neighborhood size to be considered, depending on
• 3.44% when the median filter algorithm was performed
JPVM - PAgent: Hom ogeneous Heterogeneous
with 14 processes;
• 37.49% when the edges detection filter algorithm was 60000
performed with 4 processes; 50000

• 37.29% when the edges detection filter algorithm was Exe c ut io n

40000
30000
performed with 7 processes; T ime ( ms )
20000
• 6.59% when the edges detection filter algorithm was
10000
performed with 11 processes;
0
• 6.44% when the edges detection filter algorithm was M 4P M 7P M 11P M 14P S4P S7P S11P S14P
performed with 14 processes. pro c e s s e s

Figure 7: Comparison of average execution time of the

Heterogeneous: JPVM - Standard JPVM - PAgent algorithms in scenarios II and IV.
80000
70000 A satisfactory gain in the execution time of the algorithms
60000
E xe c ut io n 50000 in a heterogeneous distributed parallel environment using the
40000
T im e ( m s )
30000
instrumented JPVM was achieved, due to the heterogeneity
20000 established in its code. This heterogeneity consists of:
10000
0 collecting raw load resources, calculating performance and
M 4P M 7P M 11P M 14P load indexes, and providing scheduling by means of mobile
pro c e s s es
agents. The information collected by the agent to do the
Figure 4: Comparison of average execution time of the median scheduling does not provide significant traffic in the network.
filters in scenarios III and IV. The scheduling asserts the processing elements with a
coherent load to the respective computational power when it
Heterogeneous: JPVM - Standard JPVM - PAgent executes load scheduling based on this information and then
has a superior performance.
30000
25000
20000
E xe c ut io n VII. CONCLUSIONS
15000
T im e ( m s )
10000 The purpose of this article showed the benefit achieved using
5000
mobile agents in parallel computation applied to medical
0
S4P S7P S11P S14P image processing, always maintaining the workload
pro c e s s es information (reflected by the performance and load indexes)
updated. Thus, it was possible to provide a better performance
Figure 5: Comparison of average execution time of the edges in the application execution time that uses these indexes to
detection filters in scenarios III and IV. supply load scheduling. As a sub-aim, unnecessary traffic
should be avoided in order to generate this agent in the inter-
In Figure 6 and Figure 7, it is demonstrated how the communication network, which influences the performance
environment heterogeneity influences the system performance system.
as a whole, emphasizing that graphs were used to analyze the From the results achieved, it can be observed that there is a
same algorithms with the same passage library, however with huge benefit concerning process scheduling helped by the
different machines. mobile agents to collect performance and load indexes for
parallel medical image processing, i.e., there is a significant
decrease in the response time when the proposed environment
JPVM - Standard: Hom ogeneous Heterogeneous is used compared to the environment that uses the scheduling
80000
standard round-robin.
70000 In the future the following could be studied: performance
60000
50000 evaluations of the proposed environment with others types of
E xe c ut io n
40000 parallel applications, as well as a comparison of mobile agents
T im e ( m s )
30000
20000 developed in the μCode environment with agents developed in
10000 the Aglets environment.
0
M 4P M 7P M 11P M 14P S4P S7P S11P S14P
pro c e s s es

Figure 6: Comparison of average execution times of the

algorithms in scenarios I and III.

REFERENCES
[1] BARBOSA, J. M. G. PARALELISMO EM
PROCESSAMENTO E ANÁLISE DE IMAGEM
MÉDICA. (2000) Tese apresentada ao Departamento
de Engenharia Electrotécnica e de Computadores –
Faculdade de Engenharia da Universidade do Porto.
[2] BIESZCZAD, A.; WHITE, T.; PAGUREK, B. (1998)
Mobile Agents for Network Management. In IEEE
Communications Surveys, September.
[3] BRANCO, K. R. L. J. C. (2004) Índice de Carga E
Desempenho em Ambientes Paralelos/ Distribuídos –
Modelagem e Métricas. Tese de mestrado. ICMC-USP.
[4] BRANCO, K. R. L. J. C.; MORENO, E. D. (2006)
"Load Indices - Past, Present and Future," ichit, pp.
206-214, 2006 International Conference on Hybrid
Information Technology - Vol2 (ICHIT'06).
[5] CHAKRAVARTI, A. J.; BAUMGARTNER, G.;
LAURIA, M. (2004) Application-Specific Scheduling
for the Organic Grid. In Proceedings of 5th IEEE/ACM
International Workshop on Grid Computing, p. 146-
155.
[6] CHO CHO MYINT; KHIN MAR LAR TUN. (2005) A
Framework of Using Mobile Agent to Achieve
Efficient Load Balancing in Cluster. Information and
Telecommunication Technologies. APSITT 2005
Proceedings. 6th Asia-Pacific Symposium on Volume,
Issue, 09-10 Nov. 2005, p. 66 – 70.
[7] DANTAS, M.A.R.; LOPES, J.G.R.C.; RAMOS, T.G.
(2002) An Enhanced Scheduling Approach in a
Distributed Parallel Environment Using Mobile Agents.
Proceedings of the 16th Annual International
Symposium on High Performance Computing Systems
and Applications, p. 177 – 181.
[8] FERRARI, A. J. (2007) JPVM The Java Parallel
Virtual Machine. Disponível em: < Distributed Systems. IEEE Computer Society Press.
http://www.cs.virginia.edu/~ajf2j/jpvm.html >. Acesso
em Maio de 2007.
[9] FERRARI, D.; ZHOU, S. (1987) An Empirical
Investigation of Load Indices for Load Balancing
Applications. In Proceedings of Performance'87, the
12th Int'l Symposium on Computer Performance
Modeling, Measurement, and Evaluation.
[10] FUGGETTA, A.; PICCO, G.; VIGNA, G. (1998)
Understanding Code Mobility. IEEE Transactions on
Software Engineering, vol. 24, p. 343-353.
[11] FUKUDA, M.; TANAKA, Y.; BIC, L. F.;
KOBAYASHI, S. (2003) A Mobile-Agent-Based PC
Grid. IEEE Computer.
[12] GONZALEZ, R. C.; WOODS, R. E. Processamento de
Imagens Digitais. Ed Edgard Blücher, São Paulo, 2002.
[13] JANSEN, W. (2000) Countermeasures for Mobile
Agent Security. Computer Communications, vol. 23, p.
1667-1676.
[14] JANSEN, W.; KARYGIANNIS, T. (1999) Mobile
Agent Security. National Institute of Standards and
Technology - Computer Security Division, NIST
Special Publication 800-19.
[15] KUNZ, T. (1991) The Influence of Different Workload
Descriptions on a Heuristic Load Balancing Scheme.
IEEE Transactions on Software Engineering.
[16] LANGE, D. B.; OSHIMA, M. (1998) Mobile Agents
with Java: The Aglet API. World Wide Web, 1(3).
[17] LANGE, D. B.; OSHIMA, M. (1999) Seven Good
Reasons for Mobile Agents. Communications of the
ACM, 42(3):88-89.
[18] MARQUES, G. M.; BRANCO, K. R. L. J. C. (2007)
Coleta de Índice de Desempenho em Sistemas Paralelos
Distribuídos Heterogêneos com Agentes Móveis – Uma
Proposta. In: VII SIPM 2007, Anais do VII - Simpósio
de Informática do Planalto Médio, Passo Fundo/RS -
Brasil.
[19] MARQUES, G. M.; SABATINE, R. J.; NUNES, F. L.
S. M; BRANCO, K. R. L. J. C. (2007) Processamento
de Imagens Médicas em Ambiente Paralelo Distribuído
Utilizando Agentes Móveis. In: II C3N 2007. Anais do
II Congresso da Academia Trinacional de Ciências, Foz
do Iguaçu/PR - Brasil.
[20] NUNES, F. L. S. M. (2001) Investigações em
Processamento de Imagens Mamográficas para Auxílio
ao Diagnóstico de Mamas Densas. Tese apresentada ao
Instituto de Física de São Carlos – Universidade de São
Paulo.
[21] NUNES, F. L. S. (2006) Introdução ao Processamento
de Imagens Médicas para Auxílio ao Diagnóstico. In:
Karin Breitman; Ricardo Anido. (Org.). Atualizações
em Informática. 1 ed. Rio de Janeiro: PUC-Rio, v. 1, p.
73-126.
[22] PICCO, G. (2007) A Mobile Code Toolkit. Disponível
em: <http://mucode.sourceforge.net>. Acesso em
Fevereiro de 2007.
[23] SHIRAZI, B.A.; HURSON, A. R.; KAVIN, K. M.
(1995) Scheduling and Load Balancing in Parallel and
[24] ZALUSKA, E. J. (1991) Research Lines in Distributed
Computing Systems and Concurrent Computation.
Workshop em Programação Concorrente, Sistemas
Distribuídos e Engenharia de Software, ICMC/USP,
São Carlos/SP.
[25] ZHOU, S.; ZHENG, X.; WANG, J.; DELISLE, P.
(1993) Utopia: a Load Sharing Facility for Large,
Heterogeneous Distributed Computer Systems.
Software: Practice and Experience, v.23.