2011

Lagos State University External System Afolorunso, A. A.

[CSC 337]
Data Structures II
1

Course Topics

Introduction: why we need clever algorithms and data structures. Performance and Big O: how much time and storage is needed as data sizes become large Trees and Tree Recursion Balanced Trees and Maps Hashing Graphs Memory management and garbage collection Recursive programs Use of Macros

Introduction A data structure is a particular way of storing and organizing data in a computer so that it can be used efficiently. Different kinds of data structures are suited to different kinds of applications, and some are highly specialized to specific tasks. For example, B-trees are particularly well-suited for implementation of databases, while compiler implementations usually use hash tables to look up identifiers. Data structures are used in almost every program or software system. Data structures provide a means to manage huge amounts of data efficiently, such as large databases and internet indexing services. Usually, efficient data structures are a key to designing efficient algorithms. Some formal design methods and programming languages emphasize data structures, rather than algorithms, as the key organizing factor in software design. Data structures are generally based on the ability of a computer to fetch and store data at any place in its memory, specified by an address a bit string that can be itself stored in memory and manipulated by the program. Thus the record and array data structures are based on computing the addresses of data items with arithmetic operations; while the linked data structures are based on storing addresses of data items within the structure itself. Many data structures use both principles, sometimes combined in non-trivial ways (as in XOR linking) The implementation of a data structure usually requires writing a set of procedures that create and manipulate instances of that structure. The efficiency of a data structure cannot be analyzed separately from those operations. This observation motivates the theoretical concept of an abstract data type, a data structure that is defined indirectly by the operations that may be performed on it, and the mathematical properties of those operations (including their space and time cost).

Common data structures

Common data structures include: array, linked list, hash-table, heap, B-tree, red-black tree, trie, stack, and queue.
2 |Page

Language support
Most Assembly languages and some low-level languages, such as BCPL, generally lack support for data structures. Many high-level programming languages, and some higher-level assembly languages, such as MASM, on the other hand, have special syntax or other built-in support for certain data structures, such as vectors (one-dimensional arrays) in the C language and multi-dimensional arrays in Pascal. Most programming languages feature some sorts of library mechanism that allows data structure implementations to be reused by different programs. Modern languages usually come with standard libraries that implement the most common data structures. Examples are the C++ Standard Template Library, the Java Collections Framework, and Microsoft's .NET Framework. Modern languages also generally support modular programming, the separation between the interface of a library module and its implementation. Some provide opaque data types that allow clients to hide implementation details. Object-oriented programming languages, such as C++, Java and .NET Framework use classes for this purpose. Many known data structures have concurrent versions that allow multiple computing threads to access the data structure simultaneously In this course, we will be interested in performance, especially as the size of the problem increases. ``An engineer can what any fool can do for a dollar.'' Thus, engineer / fool &asymp 10 Big O and Performance We are very interested in performance of algorithms and how the cost of an algorithm increases as the size of the problem increases. Cost can include:

do

for

dime

Time required Space (memory) required Network bandwidth Other costs

The terms efficiency and complexity are also used. Big O Big O (often pronounced order) is an abstract function that describes how fast a function grows as the size of the problem becomes large. The order given by Big O is a least upper bound on the rate of growth.

3 |Page

We say that a function T(n) has order O(f(n)) if there exist positive constants c and n0 such that: T(n) &le c * f(n) when n &ge n0. For example, the function T(n) = 2 * n2 + 5 * n + 100 has order O(n2) because 2 * n2 + 5 * n + 100 &le 3 * n2 for n &ge 13. We don't care about performance for small values of n, since small problems are usually easy to solve. In fact, we can make that a rule: Rule: Any algorithm is okay if we know that the size of the input is small. In such cases, the simplest (easiest to code) algorithm is often best. Rules for Big O There are several rules that make it easy to find the order of a function:

Any constant multipliers are dropped. The order of a sum is the maximum of the orders of its summands. A higher power of n beats a lower power of n. n beats log(n). 2n beats any power of n.

For example, in analyzing T(n) = 2 * n2 + 5 * n + 100, we first drop all the constant multipliers to get n2 + n + 1 and then drop the lower-order terms to get O(n2). Classes of Algorithms f(n) Name Example 1 Constant + log(n) Logarithmic binary search log2(n) Log-squared n Linear max of array n*log(n) Linearithmic quicksort n2 Quadratic selection sort 3 n Cubic matrix multiply k n Polynomial 2n Exponential knapsack problem

When we use log(n), we often mean log2(n); however, the base of the log(n) does not matter: logk(n) = log(n) / log(k) Thus, a log(n) is converted to another base by multiplying by a constant; but we eliminate constants when determining Big O. Log n Grows Slowly The function log(n) grows so slowly that it can almost be considered to be the same as 1. A good rule: log2(1000) 10 bits &asymp 3 decimal digits.
4 |Page

&asymp

10,

since

210

1024.

students in CS 315 students at UT people in US people on earth national debt Library of Congress, bytes earth surface area, mm2 atoms in universe

n log2(n) 120 7 50,000 16 300,000,000 28 7,000,000,000 33 11,700,000,000,000 44 20*1012 45 20 5*10 69 80 10 266

Thus, we can say that log2(n) < 300 for any problem that we are likely to see. If we simply wrote 300 instead of log2(n), our rule tells us to eliminate the constant 300. Powers of 10: SI Prefixes Prefix Symbol 10n Prefix Symbol 2k Yotta Y 1024 Yobi Yi 280 Zetta Z 1021 Zebi Zi 270 18 Exa E 10 Exbi Ei 260 Peta P 1015 Pebi Pi 250 Tera T 1012 Tebi Ti 240 Giga G 109 Gibi Gi 230 Mega M 106 Mebi Mi 220 Kilo K 103 Kibi Ki 210 -3 milli m 10 micro &mu 10-6 nano n 10-9 pico p 10-12 femto f 10-15 atto a 10-18 zepto z 10-21 yocto y 10-24 Word

trillion billion million thousand

Definitions
The order of a function T(N) is a description of its rate of growth as N becomes very large. When analyzing the order of algorithms, the following definitions are commonly used:

T(N) = O(f(N)) if there are positive constants c and when .

such that

5 |Page

As an example, if T(N) = 2N + 1, then T(N) = O(N). To prove this, all we need to do is find c and such that when . If we try c = 3, then we can . This is clearly true

subtract 2N from both sides of the inequality and get when , so is 1.

It may seem strange that in this case f(N) is clearly less than T(N) for all positive values of N, and yet T(N) = O(f(N)). The actual values of T(N) and f(N) are not the issue- the important issue is whether a c and can be found that satisfy this rule.

if there are positive constants c and when .

such that

This is similar to the big-O notation, but from the other direction- the order of f(N) is typically as large as possible without making it larger than the order of T(N). As an example, if T(N) = 2N + 1, then T(N) = O(N). To prove this, once again all we need to do is find c and such that when . If we try c = 2, . This is

then we can subtract 2N from both sides of the inequality and get clearly true for any N.

if and only if T(N) = O(f(N)) and T(N) = o(f(N)) if and only if T(N) = O(f(N)) and .

Remember that in this notation, our goal is to find f(N) (an order), given T(N) (a function). For example, imagine that we have a function things about it using big-O notation. It is correct to say ``g(N) is correct to say ``O(g(N)) is ''. , and we would like to say '' but it is usually not

Big-O Conventions
It may seem strange that the order of f(N) might actually be greater than the order of T(N). For example, if , then it is also true that , and so on.

By general convention, f(N) is chosen to be the smallest order that is known to be greater or equal to the order of T(N). In this example, you would never say that if

you knew that . It would be correct, at least in a mathematical sense, but potentially very misleading to the reader. In fact, in many contexts, big-O is used synonymously with big-theta.
6 |Page

As another convention, any constant factors are dropped from the order. For example, according to the definition given above, it is true that . However, it is also

true that . The latter form, where the constant factor has been omitted, is the standard way of expressing the big-O of a function. Similarly, it is also a convention to drop all but the ``largest'' term of the order. The largest term is the dominant term (see section 2.2.1 for more information). For example, , since as n becomes large, the quadratic term will grow more quickly than the others.

Why Not Use

All The Time?

At first glance, it would appear that is all we really want or need- after all, it's the ``right'' answer, where the order of T(N) is exactly f(N). If we know the real order of T(N), then we know .

Unfortunately, life is not so simple. Consider the following code snippet:

1 2 3 4 5 6 7 8 9 10 11 void { strange1 (int A[N]) int i;

for (i = 0; i < N; i++) { if (A[N] == 0) { break; } printf ("A [%d] = %d\n", i, A[i]); } }

What is the order of strange1? It is impossible to know without knowing what the contents of array A are: if there is always a zero element in the array, and the first zero element is always at a fixed location in this array (for example, the first location), then this function is - it always takes the same amount of time to run, regardless of N. If we know that there isn't a zero element anywhere in the array, then this algorithm is , since it visits every location in the array, and the array is of length N. If we don't know anything about the contents of the array, however, then we can't be sure what the order is, so we don't know what the true order of is. We know that the order is no less than order 1, so . Similarly, we know that the order is no larger than N, so .

7 |Page

Finding the Big-O of a Function

Although the definitions given above completely define what the big-O of a function is, it is not always immediately obvious how to use them to actually discover the big-O of a function. Finding the order of many useful functions can be a challenging task, and there are even functions that have defied all attempts at a complete analysis! Luckily, since there has been so much work done in the area already, finding the big-O of many functions is simply a matter of finding a known result for a similar function and using it. The following rules provide a jumping off point:
1. If 1. 2. 2. If T(N) is a polynomial of degree k, then . and , then:

The fact that

follows from the previous rule. Showing that

is slightly more complicated, but can be proven by showing that there is always a choice of c and that satisfies the definition of .

3.

, for any constant k.

This is true, but of relatively limited use. Since O(N) is an upper bound, it is OK to say that anything with a lower order is O(N). This is handy as a last resort, when trying to find the big-O of a nasty function that includes logarithms, but whenever possible it is useful to draw a distinction between O(N) and example, an algorithm that has a big-O of an algorithm that is . . For

is (usually) much better than

The Dominance of Functions

Another method of finding the big-O of a function is to find the dominant term of the function, and find its order. The order of the dominant term will also be the order of the function. The dominant term is the term that grows most quickly as n becomes large. Some rules of dominance include the following:

Any exponential function of n dominates any polynomial function of n. Any polynomial function of n dominates any logarithmic function of n.

8 |Page

Any logarithmic function of n dominates a constant term. A polynomial of degree k dominates a polynomial of degree l if and only if k > l.

There are additional rules that you can discover for yourself. The key is that term x(n) dominates function y(n) if and only if x(n)/y(n) grows as n grows large. Using these rules can sometimes make finding the order of a complicated function easy. For example, the function , is clumsy to manipulate algebraically. However, thanks to the rules of dominance, we know that we the term will dominate the order of this function, so we can simply find the order of , which is itself. Therefore, we can immediately say that .

There are other methods that can be used find the order of functions. but we will not introduce them yet. Later in the semester, when we analyze several recursive algorithms, methods for analyzing the order of recursive functions will be introduced.

Properties of Orders
Since the orders of functions look like ordinary functions, it is tempting to treat them as such, and manipulate them as though they were ordinary functions. This is usually a huge mistake. For example, let and . It should be clear that the both f(n) and g(n)

are . What is the big-O of g(n) - f(n)? If we compute the order of the difference as the difference of the orders, then we would mistakenly conclude that (g(n) - f(n)) = the order of f(n) less the order of g(n), or compute g(n) - f(n), however, . This answer is obviously wrong. If we actually we arrive at the correct conclusion: .

Big-O Notation and Algorithm Analysis

Now that we have seen the basics of big-O notation, it is time to relate this to the analysis of algorithms. In our study of algorithms, nearly every function whose order we are interested in finding is a function that defines the quantity of some resource consumed by a particular algorithm in relationship to the parameters of the algorithm. (This function is often referred to as a complexity of the algorithm, or less frequently as the cost function of the algorithm.) This function is usually not the same as the algorithm itself, but is a property of the algorithm. For example, when we are analyzing an algorithm that multiplies two numbers, the functions we might be interested in are the relationships between the number of digits in each number and the length of time or amount of memory required by the algorithm.
9 |Page

Although big-O notation is a way of describing the order of a function, it is also often meant to represent the time complexity of an algorithm. This is sloppy use of the mathematics, but unfortunately not uncommon. Usually it is clear from the surrounding context what the big-O refers to, but when it is not clear, please ask. Note that this notation, like the orders themselves, doesn't tell us how quickly or slowly the algorithms actually execute for a given input. This information can be extremely important in practice- during the semester, we will study several algorithms that address the same problems and have the same order running time, but take substantially different amounts of time to execute. Similarly, the order doesn't tell us how fast an algorithm will run for a small N. This may also be quite important in practice- during the semester, we will study at least one group of algorithms where the choice of the best algorithm depends on N- when N is small, one algorithm is best, but when N is large, a different algorithm is much better.

The Size of a Problem

Big-O notation is used to express an ordering property among functions. In the context of our study of algorithms, the functions are the amount of resources consumed by an algorithm when the algorithm is executed. This function is usually denoted T(N). T(N) is the amount of the resource (usually time or the count of some specific operation) consumed when the input to the algorithm is of size N. Sometimes it is not obvious what the ``size'' of the input is, so here are few examples:

Multiplication

If we multiply two numbers together using the algorithm we learned in grade school, the number of single-digit multiplications and additions that we do depends on the number of digits in the two numbers being multiplied. (If we multiply two numbers together using a calculator, then the number of keystrokes we need to type also depends on the number of digits in the two numbers.) Therefore, for an algorithm that performs multiplication (or any other arithmetic operation) the ``size'' of the input is usually the number of digits in the input numbers. (As an aside, remember that most modern computers can perform arithmetic operations on reasonably-sized numbers (i.e. numbers that are 32 or 64 bits in length) in a single instruction, regardless of the number of non-zero digits in the numbers. Therefore, for most purposes, we will assume that arithmetic operations take a constant amount of time. There are many important applications that deal with numbers large enough so that this is not the case, however.)

Searching

If we have a set of N unknown items, arranged in an unknown order, and we want to do something involving all of them, then the size of the problem is N. For example, if we want to find the smallest element in an array of N numbers, then the size of the problem is N.
10 | P a g e

What Does It All Mean?

The big-O complexity of an algorithm is important, but it does not tell the entire story. Any honest and complete analysis of an algorithm should attempt to address the question of what the real cost of executing the algorithm is for real or typical problems. For example, imagine that (through some some wonderful twist of fate), you must choose between job offers from two companies. The first company offers a contract that will double your salary every five years. The second company offers you a contract that gives you a raise of N1000 per year. Your salary at the first company increases at , while your salary at the second company increases at a rate of O(n). Which position would you choose? From this information, it is impossible to say. If your starting salaries are the same at the two companies, then what the starting salary is will make all the difference (and if the first company gives you a starting salary of N0, then you are really in trouble!). If the starting salary at the second company is much higher than the first company, then even the wonderful raises the first company offers might not actually make any difference before you retire. In this case, n isn't going to get too large (assuming an ordinary career), so the behavior of your salary as you become infinitely old is not an issue.

A Scientist Should Be Careful, Skeptical

Is the input data good? Garbage in, garbage out! o Is the data source authoritative? ( Don't trust it!) o Was the data measured accurately? o How much noise does the data have? o Was the data recorded accurately? o Does the data mean what it is alleged to mean? Is the algorithm correct? Is the algorithm an accurate model of reality? Are the answers numerically accurate? Is the data represented accurately? o 32-bit int: 9 decimal digits o 64-bit long: 19 decimal digits o 32-bit float: 7 decimal digits ( avoid! ) Patriot missile: inaccurate float conversion caused range gate error: 28 killed, 98 injured. o 64-bit double: 15+ decimal digits o Ariane 5: 64-bit double converted to 16-bit int: overflowed, rocket exploded, $500 million loss.

Finding Big O: Log-log Graph There are two ways to find the order of a computation:

Analytically before developing software.

11 | P a g e

Experimentally with existing software. o Log-log graph o Time ratios

A plot of time versus n on a log-log graph allows Big O to be found directly. Polynomial algorithms show up as straight lines on log-log graphs, and the slope of the line gives the order of the polynomial.

Log-log Example Suppose that f(n) = 25 * n2 . n 2 3 4 5 6 f(n) 100 225 400 625 900

The log-log graph makes it obvious that f(n) is O(n2).

12 | P a g e

Searching Computer systems are often used to store large amounts of data from which individual records must be retrieved according to some search criterion. Thus the efficient storage of data to facilitate fast searching is an important issue. In this section, we shall investigate the performance of some searching algorithms and the data structures which they use. Sequential Searches Let's examine how long it will take to find an item matching a key in the collections we have discussed so far. We're interested in: a. the average time b. the worst-case time and c. the best possible time. However, we will generally be most concerned with the worst-case time as calculations based on worst-case times can lead to guaranteed performance predictions. Conveniently, the worstcase times are generally easier to calculate than average times. If there are n items in our collection - whether it is stored as an array or as a linked list - then it is obvious that in the worst case, when there is no item in the collection with the desired key, then n comparisons of the key with keys of the items in the collection will have to be made. To simplify analysis and comparison of algorithms, we look for a dominant operation and count the number of times that dominant operation has to be performed. In the case of searching, the dominant operation is the comparison, since the search requires n comparisons in the worst case, we say this is a O(n) (pronounce this "big-Oh-n" or "Oh-n") algorithm. The best case - in which the first comparison returns a match - requires a single comparison and is O(1). The average time depends on the probability that the key will be found in the collection - this is something that we would not expect to know in the majority of cases. Thus in this case, as in most others, estimation of the average time is of little utility. If the performance of
13 | P a g e

the system is vital, i.e. it's part of a life-critical system, then we must use the worst case in our design calculations as it represents the best guaranteed performance. Binary Search However, if we place our items in an array and sort them in either ascending or descending order on the key first, then we can obtain much better performance with an algorithm called binary search. In binary search, we first compare the key with the item in the middle position of the array. If there's a match, we can return immediately. If the key is less than the middle key, then the item sought must lie in the lower half of the array; if it's greater then the item sought must lie in the upper half of the array. So we repeat the procedure on the lower (or upper) half of the array. Our FindInCollection function can now be implemented: static void *bin_search( collection c, int low, int high, void *key ) { int mid; /* Termination check */ if (low > high) return NULL; mid = (high+low)/2; switch (memcmp(ItemKey(c->items[mid]),key,c->size)) { /* Match, return item found */ case 0: return c->items[mid]; /* key is less than mid, search lower half */ case -1: return bin_search( c, low, mid-1, key); /* key is greater than mid, search upper half */ case 1: return bin_search( c, mid+1, high, key ); default : return NULL; } } void *FindInCollection( collection c, void *key ) { /* Find an item in a collection Pre-condition: c is a collection created by ConsCollection c is sorted in ascending order of the key key != NULL Post-condition: returns an item identified by key if one exists, otherwise returns NULL */ int low, high; low = 0; high = c->item_cnt-1; return bin_search( c, low, high, key ); } Points to note:

14 | P a g e

a. bin_search is recursive: it determines whether the search key lies in the lower or upper

half of the array, then calls itself on the appropriate half. b. There is a termination condition (two of them in fact!) i. If low > high then the partition to be searched has no elements in it and ii. If there is a match with the element in the middle of the current partition, then we can return immediately. c. AddToCollection will need to be modified to ensure that each item added is placed in its correct place in the array. The procedure is simple: i. Search the array until the correct spot to insert the new item is found, ii. Move all the following items up one position and iii. Insert the new item into the empty position thus created. d. bin_search is declared static. It is a local function and is not used outside this class: if it were not declared static, it would be exported and be available to all parts of the program. The static declaration also allows other classes to use the same name internally. static reduces the visibility of a function and should be used wherever possible to control access to functions! Analysis

Each step of the algorithm divides the block of items being searched in half. We can divide a set of n items in half at most log2 n times. Thus the running time of a binary search is proportional to log n and we say this is a O(log n) algorithm.

15 | P a g e

Binary search requires a more complex program than our original search and thus for small n it may run slower than the simple linear search. However, for large n,

Thus at large n, log n is much smaller than n, consequently an O(log n) Plot of n and log n vs n . algorithm is much faster than an O(n) one. We will examine this behaviour more formally in a later section. First, let's see what we can do about the insertion (AddToCollection) operation. In the worst case, insertion may require n operations to insert into a sorted list.
1. We can find the place in the list where the new item belongs using binary search in

O(log n) operations. 2. However, we have to shuffle all the following items up one place to make way for the new one. In the worst case, the new item is the first in the list, requiring n move operations for the shuffle! A similar analysis will show that deletion is also an O(n) operation. If our collection is static, ie it doesn't change very often - if at all - then we may not be concerned with the time required to change its contents: we may be prepared for the initial build of the collection and the occasional insertion and deletion to take some time. In return, we will be able to use a simple data structure (an array) which has little memory overhead.
16 | P a g e

However, if our collection is large and dynamic, ie items are being added and deleted continually, then we can obtain considerably better performance using a data structure called a tree. Key terms Big Oh A notation formally describing the set of all functions which are bounded above by a nominated function. Binary Search A technique for searching an ordered list in which we first check the middle item and - based on that comparison - "discard" half the data. The same procedure is then applied to the remaining half until a match is found or there are no more items left.

Trees A tree is a kind of graph, composed of nodes and links, such that:

A link is a directed pointer from one node to another. There is one node, called the root, that has no incoming links. Each node, other than the root, has exactly one incoming link from its parent. Every node is reachable from the root.

A node can have any number of children. A node with no children is called a leaf ; a node with children is an interior node.

Trees occur in many places in computer systems and in nature. Arithmetic Expressions as Trees Arithmetic expressions can be represented as trees, with operands as leaf nodes and operators as interior nodes. y = m * x + b
17 | P a g e

(= y (+ (* m x) b))

Computer Programs as Trees When a compiler parses a program, it often creates a tree. When we indent the source code, we are emphasizing the tree structure. if ( x > y ) j = 3; else j = 1;

This is called an abstract syntax tree or AST. English Sentences as Trees Parsing is the assignment of structure to a linear string of words according to a grammar; this is much like the diagramming of a sentence taught in grammar school.

18 | P a g e

The speaker wants to communicate a structure, but must make it linear in order to say it. The listener needs to re-create the structure intended by the speaker. Parts of the parse tree can then be related to object symbols in memory. File Systems as Trees Most computer operating systems organize their file systems as trees.

A directory or folder is an interior node; a file is a leaf node. Representations of Trees Many different representations of trees are possible:

Binary tree: contents and left and right links. First-child/next-sibling: contents, first-child, next sibling. Linked list: the first element contains the contents, the rest are a linked list of children. Implicit: a node may contain only the contents; the children can be generated from the contents or from the location of the parent.

Binary Trees The simplest form of tree is a binary tree. A binary tree consists of
19 | P a g e

a. a node (called the root node) and

b. left and Both the sub-trees are themselves binary trees. You now have a recursively

right defined data

sub-trees. structure.

A binary tree The nodes at the lowest levels of the tree (the ones with no sub-trees) are called leaves. In an ordered binary tree, 1. the keys of all the nodes in the left sub-tree are less than that of the root, 2. the keys of all the nodes in the right sub-tree are greater than that of the root, 3. the left and right sub-trees are themselves ordered binary trees. Data Structure The data structure for the tree implementation simply adds left and right pointers in place of the next pointer of the linked list implementation. [Load the tree struct.] The AddToCollection method is, naturally, recursive. [ Load the AddToCollection method.] Similarly, the FindInCollection method is recursive. [ Load the FindInCollection method.] Analysis Complete Trees Before we look at more general cases, let's make the optimistic assumption that we've managed to fill our tree neatly, ie that each leaf is the same 'distance' from the root.
20 | P a g e

This forms a complete tree, whose height is defined as the number of links from the root to the deepest leaf. A complete tree First, we need to work out how many nodes, n, we have in such a tree of height, h. Now, n = 1 + 21 + 22 + .... + 2h From which we have, n = 2h+1 - 1 and h = floor( log2n ) Examination of the Find method shows that in the worst case, h+1 or ceiling( log2n ) comparisons are needed to find an item. This is the same as for binary search. However, Add also requires ceiling( log2n ) comparisons to determine where to add an item. Actually adding the item takes a constant number of operations, so we say that a binary tree requires O(logn) operations for both adding and finding an item - a considerable improvement over binary search for a dynamic structure which often requires addition of new items. Deletion is also an O(logn) operation. General binary trees However, in general addition of items to an ordered tree will not produce a complete tree. The worst case occurs if we add an ordered list of items to a tree. What will happen? Think before you click here! This problem is readily overcome: we use a structure known as a heap. However, before looking at heaps, we should formalise our ideas about the complexity of algorithms by defining carefully what O(f(n)) means. Key terms Root Node
21 | P a g e

Node at the "top" of a tree - the one from which all operations on the tree commence. The root node may not exist (a NULL tree with no nodes in it) or have 0, 1 or 2 children in a binary tree. Leaf Node Node at the "bottom" of a tree - farthest from the root. Leaf nodes have no children. Complete Tree Tree in which each leaf is at the same distance from the root. A more precise and formal definition of a complete tree is set out later. Height Number of nodes which must be traversed from the root to reach a leaf of a tree. Tree operations A binary tree can be traversed in a number of ways: pre-order 1. Visit the root 2. Traverse the left sub-tree, 3. Traverse the right sub-tree 1. Traverse the left sub-tree, 2. Visit the root 3. Traverse the right sub-tree 1. Traverse the left sub-tree, 2. Traverse the right sub-tree 3. Visit the root If we traverse the standard ordered binary tree in-order, then we will visit all the nodes in sorted order. Binary Search Trees In the introduction we used the binary search algorithm to find data stored in an array. This method is very effective, as each iteration reduced the number of items to search by one-half. However, since data was stored in an array, insertions and deletions were not efficient. Binary search trees store data in nodes that are linked in a tree-like fashion. For randomly inserted data, search time is O(lg n). Worst-case behavior occurs when ordered data is inserted. In this case the search time is O(n).] for a more detailed description.

in-order

post-order

22 | P a g e

Theory A binary search tree is a tree where each node has a left and right child. Either child, or both children, may be missing. Figure 3-2 illustrates a binary search tree. Assuming k represents the value of a given node, then a binary search tree also has the following property: all children to the left of the node have values smaller than k, and all children to the right of the node have values larger than k. The top of a tree is known as the root, and the exposed nodes at the bottom are known as leaves. In Figure 3-2, the root is node 20 and the leaves are nodes 4, 16, 37, and 43. The height of a tree is the length of the longest path from root to leaf. For this example the tree height is 2.

Figure 3-2: A Binary Search Tree To search a tree for a given value, we start at the root and work down. For example, to search for 16, we first note that 16 < 20 and we traverse to the left child. The second comparison finds that 16 > 7, so we traverse to the right child. On the third comparison, we succeed. Each comparison results in reducing the number of items to inspect by one-half. In this respect, the algorithm is similar to a binary search on an array. However, this is true only if the tree is balanced. For example, Figure 3-3 shows another tree containing the same values. While it is a binary search tree, its behavior is more like that of a linked list, with search time increasing proportional to the number of elements stored.

23 | P a g e

Figure 3-3: An Unbalanced Binary Search Tree Insertion and Deletion Let us examine insertions in a binary search tree to determine the conditions that can cause an unbalanced tree. To insert an 18 in the tree in Figure 3-2, we first search for that number. This causes us to arrive at node 16 with nowhere to go. Since 18 > 16, we simply add node 18 to the right child of node 16 (Figure 3-4). Now we can see how an unbalanced tree can occur. If the data is presented in an ascending sequence, each node will be added to the right of the previous node. This will create one long chain, or linked list. However, if data is presented for insertion in a random order, then a more balanced tree is possible. Deletions are similar, but require that the binary search tree property be maintained. For example, if node 20 in Figure 3-4 is removed, it must be replaced by node 37. This results in the tree shown in Figure 3-5. The rationale for this choice is as follows. The successor for node 20 must be chosen such that all nodes to the right are larger. Therefore we need to select the smallest valued node to the right of node 20. To make the selection, chain once to the right (node 38), and then chain to the left until the last node is found (node 37). This is the successor for node 20.

24 | P a g e

Figure 3-4: Binary Tree After Adding Node 18

Figure 3-5: Binary Tree After Deleting Node 20 Implementation An ANSI-C implementation for a binary search tree is included. Typedef T and comparison operators compLT and compEQ should be altered to reflect the data stored in the tree. Each Node consists of left, right, and parent pointers designating each child and the parent. Data is stored in the data field. The tree is based at root, and is initially NULL. Function insertNode allocates a new node and inserts it in the tree. Function deleteNode deletes and frees a node from the tree. Function findNode searches the tree for a particular value. Rooted Trees We can use a recursive definition to specify what we mean by a ``rooted tree''. A rooted tree is either (1) empty, or (2) consists of a node called the root, together with two rooted trees called the left subtree and right subtree of the root. A binary tree is a rooted tree where each node has at most two descendants, the left child and the right child.
25 | P a g e

A binary tree can be implemented where each node has left and right pointer fields, an (optional) parent pointer, and a data field. Rooted trees in Real Life Rooted trees can be used to model corporate heirarchies and family trees. Note the inherently recursive structure of rooted trees. Deleting the root gives rise to a certain number of smaller subtrees. In a rooted tree, the order among ``brother'' nodes matters. Thus left is different from right. The five distinct binary trees with five nodes: Binary Search Trees A binary search tree is a binary tree where each node contains a key such that:

All keys in the left subtree precede the key in the root. All keys in the right subtree succeed the key in the root. The left and right subtrees of the root are again binary search trees.

Left: A binary search tree. Right: A heap but not a binary search tree. For any binary tree on n nodes, and any set of n keys, there is exactly one labeling to make it a binary search tree!! Binary Tree Search Searching a binary tree is almost like binary search! The difference is that instead of searching an array and defining the middle element ourselves, we just follow the appropriate pointer! The type declaration is simply a linked list node with another pointer. Left and right pointers are identical types. TYPE T = BRANDED REF RECORD key: ElemT; left, right: T := NIL; END; (*T*) Dictionary search operations are easy in binary trees. The algorithm works because both the left and right subtrees of a binary search tree are binary search trees - recursive structure, recursive algorithm. Search Implementation PROCEDURE Search(tree: T; e: ElemT): BOOLEAN = (*Searches for an element e in tree. Returns TRUE if present, else FALSE*)
26 | P a g e

BEGIN IF tree = NIL THEN RETURN FALSE (*not found*) ELSIF tree.key = e THEN RETURN TRUE (*found*) ELSIF e < tree.key THEN RETURN Search(tree.left, e) (*search in left tree*) ELSE RETURN Search(tree.right, e) (*search in right tree*) END; (*IF tree...*) END Search; This takes time proportional to the height of the tree, O(h). Good, balanced trees have height , while bad, unbalanced trees have height O(n). Building Binary Trees To insert a new node into an existing tree, we search for where it should be, then replace that NIL pointer with a pointer to the new node. Each NIL pointer defines a gap in the space of keys! The pointer in the parent node must be modified to remember where we put the new node. Insertion Routine PROCEDURE Insert(VAR tree: T; e: ElemT) = BEGIN IF tree = NIL THEN tree:= NEW(T, key:= e); (*insert at proper place*) ELSIF e < tree.key THEN Insert(tree.left, e) (*search place in left tree*) ELSE Insert(tree.right, e) (*search place in right tree*) END; (*IF tree...*) END Insert; Tree Shapes and Sizes Suppose we have a binary tree with n nodes. How many levels can it have? At least and at most n.

How many pointers are in the tree? There are n nodes in tree, each of which has 2 pointers, for a total of 2n pointers regardless of shape.

27 | P a g e

How many pointers are NIL, i.e ``wasted''? Except for the root, each node in the tree is pointed to by one tree pointer Thus the number of NILs is for . ,

Traversal of Binary Trees How can we print out all the names in a family tree? An essential component of many algorithms is to completely traverse a tree data structure. The key is to make sure we visit each node exactly once. The order in which we explore each node and its children matters for many applications. There are six permutations of {left, right, node} which define traversals. The most interesting traversals are inorder {left, node, right}, preorder {node, left, right}, postorder {left, right, node}, Why do we care about different traversals? Depending on what the tree represents, different traversals have different interpretations. An in-order traversals of a binary serach tree sorts the keys! Inorder traversal: 748251396, Preorder traversal: 124785369, Postorder traversal: 784529631 Reverse Polish notation is simply a post order traversal of an expression tree, like the one below for expression 2+3*4+(3*4)/5. PROCEDURE Traverse(tree: T; action: Action; order := Order.In; direction := Direction.Right) = PROCEDURE PreL(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN action(x.key, depth); PreL(x.left, depth + 1); PreL(x.right, depth + 1); END; (*IF x # NIL*) END PreL; PROCEDURE PreR(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN action(x.key, depth); PreR(x.right, depth + 1); PreR(x.left, depth + 1); END; (*IF x # NIL*) END PreR;
28 | P a g e

PROCEDURE InL(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN InL(x.left, depth + 1); action(x.key, depth); InL(x.right, depth + 1); END; (*IF x # NIL*) END InL; PROCEDURE InR(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN InR(x.right, depth + 1); action(x.key, depth); InR(x.left, depth + 1); END; (*IF x # NIL*) END InR; PROCEDURE PostL(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN PostL(x.left, depth + 1); PostL(x.right, depth + 1); action(x.key, depth); END; (*IF x # NIL*) END PostL; PROCEDURE PostR(x: T; depth: INTEGER) = BEGIN IF x # NIL THEN PostR(x.right, depth + 1); PostR(x.left, depth + 1); action(x.key, depth); END; (*IF x # NIL*) END PostR; BEGIN (*Traverse*) IF direction = Direction.Left THEN CASE order OF | Order.Pre => PreL(tree, 0); | Order.In => InL(tree, 0); | Order.Post => PostL(tree, 0); END (*CASE order*) ELSE (* direction = Direction.Right*) CASE order OF | Order.Pre => PreR(tree, 0); | Order.In => InR(tree, 0); | Order.Post => PostR(tree, 0); END (*CASE order*) END (*IF direction*)
29 | P a g e

END Traverse; Deletion from Binary Search Trees Insertion was easy because the new node goes in as a leaf and only its parent is affected. Deletion of a leaf is just as easy - set the parent pointer to NIL. But what if the node to be deleted is an interior node? We have two pointers to connect to only one parent!! Deletion is somewhat more tricky than insertion, because the node to die may not be a leaf, and thus effect other nodes. Case (a), where the node is a leaf, is simple - just NIL out the parents child pointer. Case (b), where a node has one chld, the doomed node can just be cut out. Case (c), relabel the node as its predecessor (which has at most one child when z has two children!) and delete the predecessor! PROCEDURE Delete(VAR tree: T; e: ElemT): BOOLEAN = (*Deletes an element e in tree. Returns TRUE if present, else FALSE*) PROCEDURE LeftLargest(VAR x: T) = VAR y: T; BEGIN IF x.right = NIL THEN (*x points to largest element left*) y:= tree; (*y now points to target node*) tree:= x; (*tree assumes the largest node to the left*) x:= x.left; (*Largest node left replaced by its left subtree*) tree.left:= y.left; (*tree assumes subtrees ...*) tree.right:= y.right; (*... of deleted node*) ELSE (*Largest element left not found*) LeftLargest(x.right) (*Continue search to the right*) END; END LeftLargest; BEGIN IF tree = NIL THEN RETURN FALSE ELSIF e < tree.key THEN RETURN Delete(tree.left, e) ELSIF e > tree.key THEN RETURN Delete(tree.right, e) ELSE (*found*) IF tree.left = NIL THEN tree:= tree.right; ELSIF tree.right = NIL THEN tree:= tree.left; ELSE (*Target node has two nonempty subtrees*) LeftLargest(tree.left) (*Search in left subtree*) END; (*IF tree.left...*) RETURN TRUE
30 | P a g e

END; (*IF tree...*) END Delete; Balanced Binary Trees We have seen that searching a binary tree is very efficient, O(log(n)); however, this is only true if the tree is balanced, i.e. the two branches of a node have approximately the same height. If the tree is unbalanced, search time could be worse, perhaps even O(n). There are several clever algorithms that maintain self-balancing binary trees; these algorithms re-balance the tree as needed.

AVL Tree: heights of subtrees differ by at most 1. A node contains a balance value of -1, 0, or 1, which is the difference in height of its subtrees. If the balance goes out of this range, the tree is rebalanced. Red-Black Tree: nodes are colored red or black. The longest path from root to a leaf is no more than twice the length of the shortest path. Splay Tree: the tree is rebalanced so that recently accessed elements can be accessed quickly the next time.

Red-Black Tree A red-black tree is a binary search tree with one extra attribute for each node: the colour, which is either red or black. We also need to keep track of the parent of each node, so that a red-black tree's node structure would be: struct t_red_black_node { enum { red, black } colour; void *item; struct t_red_black_node *left, *right, *parent; } For the purpose of this discussion, the NULL nodes which terminate the tree are considered to be the leaves and are coloured black. Definition of a red-black tree A red-black tree is a binary search tree which has the following red-black properties: 1. Every node is either red or black. 2. Every leaf (NULL) is black. 3. If a node is red, then both its children are black. 4. Every simple path from a node to a descendant leaf contains the same number of black nodes.
31 | P a g e

3. implies that on any path from the root to a leaf, red nodes must not be adjacent. However, any number of black nodes may appear in a sequence.

A basic red-black tree

Basic red-black tree with the sentinel nodes added. Implementations of the red-black tree algorithms will usually include the sentinel nodes as a convenient means of flagging that you have reached a leaf node. They are the NULL black nodes of property 2. The number of black nodes on any path from, but not including, a node x to a leaf is called the black-height of a node, denoted bh(x). We can prove the following lemma: Lemma A red-black tree with n internal (For a proof, see Cormen, p 264) nodes has height at most 2log(n+1).

This demonstrates why the red-black tree is a good search tree: it can always be searched in O(log n) time. As with heaps, additions and deletions from red-black trees destroy the red-black property, so we need to restore it. To do this we need to look at some operations on red-black trees. Rotations A rotation is a local operation in a search tree that preserves in-order traversal key ordering. Note that in both trees, an in-order traversal yields: AxByC
32 | P a g e

The left_rotate operation may be encoded: left_rotate( Tree T, node x ) { node y; y = x->right; /* Turn y's left sub-tree into x's right sub-tree */ x->right = y->left; if ( y->left != NULL ) y->left->parent = x; /* y's new parent was x's parent */ y->parent = x->parent; /* Set the parent to point to y instead of x */ /* First see whether we're at the root */ if ( x->parent == NULL ) T->root = y; else if ( x == (x->parent)->left ) /* x was on the left of its parent */ x->parent->left = y; else /* x must have been on the right */ x->parent->right = y; /* Finally, put x on y's left */ y->left = x; x->parent = y; } Insertion Insertion is somewhat complex and involves a number of cases. Note that we start by inserting the new node, x, in the tree just as we would for any other binary tree, using the tree_insert function. This new node is labelled red, and possibly destroys the red-black property. The main loop moves up the tree, restoring the red-black property. rb_insert( Tree T, node x ) { /* Insert in the tree in the usual way */ tree_insert( T, x ); /* Now restore the red-black property */ x->colour = red; while ( (x != T->root) && (x->parent->colour == red) ) { if ( x->parent == x->parent->parent->left ) { /* If x's parent is a left, y is x's right 'uncle' */ y = x->parent->parent->right; if ( y->colour == red ) { /* case 1 - change the colours */ x->parent->colour = black; y->colour = black; x->parent->parent->colour = red; /* Move x up the tree */ x = x->parent->parent; }
33 | P a g e

else { /* y is a black node */ if ( x == x->parent->right ) { /* and x is to the right */ /* case 2 - move x up and rotate */ x = x->parent; left_rotate( T, x ); } /* case 3 */ x->parent->colour = black; x->parent->parent->colour = red; right_rotate( T, x->parent->parent ); } } else { /* repeat the "if" part with right and left exchanged */ } } /* Colour the root black */ T->root->colour = black; } Red-Black Tree Operation Here's an example of insertion into a red-black tree (taken from Cormen, p269).

Here's the original tree .. Note that in the following diagrams, the black sentinel nodes have been omitted to keep the diagrams simple.

34 | P a g e

The tree insert routine has just been called to insert node "4" into the tree. This is no longer a red-black tree - there are two successive red nodes on the path 11 - 2 - 7 - 5 - 4 Mark the new node, x, and it's uncle, y. y is red, so we have case 1 ...

Change the colours of nodes 5, 7 and 8.

Move x up to its grandparent, 7. x's parent (2) is still red, so this isn't a redblack tree yet. Mark the uncle, y. In this case, the uncle is black, so we have case 2 ...

35 | P a g e

Move x up and rotate left.

Still not a red-black tree .. the uncle is black, but x's parent is to the left ..

Change the colours of 7 and 11 and rotate right ..

This is now a redblack tree, so we're finished! O(logn) time!

36 | P a g e

AVL Tree An AVL tree is another balanced binary search tree. Named after their inventors, AdelsonVelskii and Landis, they were the first dynamically balanced trees to be proposed. Like redblack trees, they are not perfectly balanced, but pairs of sub-trees differ in height by at most 1, maintaining an O(logn) search time. Addition and deletion operations also take O(logn) time. Definition of an AVL tree An AVL tree is a binary search tree which has the following properties: 1. The sub-trees of every node differ in height by at most one. 2. Every sub-tree is an AVL tree. Balance requirement for an AVL tree: the left and right sub-trees differ by at most 1 in height. You need to be careful with this definition: it permits some apparently unbalanced trees! For example, here are some trees: Tree AVL tree?

Yes Examination shows that each left sub-tree has a height 1 greater than each right sub-tree.

37 | P a g e

No Sub-tree with root 8 has height 4 and sub-tree with root 18 has height 2

Insertion As with the red-black tree, insertion is somewhat complex and involves a number of cases. Implementations of AVL tree insertion may be found in many textbooks: they rely on adding an extra attribute, the balance factor to each node. This factor indicates whether the tree is left-heavy (the height of the left sub-tree is 1 greater than the right sub-tree), balanced (both sub-trees are the same height) or right-heavy (the height of the right sub-tree is 1 greater than the left sub-tree). If the balance would be destroyed by an insertion, a rotation is performed to correct the balance. A new item has been added to the left subtree of node 1, causing its height to become 2 greater than 2's right subtree (shown in green). A right-rotation is performed to correct the imbalance.

Advantage: approximately O(log(n)) search and insert Disadvantage: complex code (120 - 200 lines).

General n-ary trees If we relax the restriction that each node can have only one key, we can reduce the height of the tree. An m-way search tree
38 | P a g e

Or in plain English ..

a. is empty or b. consists of a root containing j (1<=j<m) keys, kj, and a set of sub-trees, Ti, (i = 0..j), such that i. if k is a key in T0, then k <= k1 ii. if k is a key in Ti (0<i<j), then ki <= k <= ki+1 iii. if k is a key in Tj, then k > kj and
iv.

b. A node generally has m-1 keys and m

children. Each node has alternating sub-tree pointers and keys: sub-tree | key | sub-tree | key | ... | key | sub_tree i. All keys in a sub-tree to the left of a key are smaller than it. ii. All keys in the node between two keys are between those two keys. iii. All keys in a sub-tree to the right of a key are greater than it. iv. This is the "standard" recursive part of the definition.

all Ti are nonempty m-way search trees or all Ti are empty

The height of a complete m-ary tree with n nodes is ceiling(logmn). A B-tree of order m is an m-way tree in which a. all leaves are on the same level and b. all nodes except for the root and the leaves have at least m/2 children and at most m children. The root has at least 2 children and at most m children. A variation of the B-tree, known as a B+-tree considers all the keys in nodes except the leaves as dummies. All keys are duplicated in the leaves. This has the advantage that is all the leaves are linked together sequentially, the entire tree may be scanned without visiting the higher nodes at all. Key terms n-ary trees (or n-way trees) Trees in which each node may have up to n children. B-tree Balanced variant of an n-way tree. B+-tree
39 | P a g e

B-tree in which all the leaves are linked to facilitate fast in order traversal.

B-Trees Dictionaries for very large files typically reside on secondary storage, such as a disk. The dictionary is implemented as an index to the actual file and contains the key and record address of data. To implement a dictionary we could use red-black trees, replacing pointers with offsets from the beginning of the index file, and use random access to reference nodes of the tree. However, every transition on a link would imply a disk access, and would be prohibitively expensive. Recall that low-level disk I/O accesses disk by sectors (typically 256 bytes). We could equate node size to sector size, and group several keys together in each node to minimize the number of I/O operations. This is the principle behind B-trees. Theory Figure 4-3 illustrates a B-tree with 3 keys/node. Keys in internal nodes are surrounded by pointers, or record offsets, to keys that are less than or greater than, the key value. For example, all keys less than 22 are to the left and all keys greater than 22 are to the right. For simplicity, I have not shown the record address associated with each key.

Figure 4-3: B-Tree We can locate any key in this 2-level tree with three disk accesses. If we were to group 100 keys/node, we could search over 1,000,000 keys in only three reads. To ensure this property holds, we must maintain a balanced tree during insertion and deletion. During insertion, we examine the child node to verify that it is able to hold an additional node. If not, then a new sibling node is added to the tree, and the child's keys are redistributed to make room for the new node. When descending for insertion and the root is full, then the root is spilled to new children, and the level of the tree increases. A similar action is taken on deletion, where child nodes may be absorbed by the root. This technique for altering the height of the tree maintains a balanced tree. B-Tree data stored in any node B*-Tree any node B+-Tree leaf only B++-Tree leaf only

on insert, split 1 x 1 >2 x 1/2 2 x 1 >3 x 2/3 1 x 1 >2 x 1/2 3 x 1 >4 x 3/4
40 | P a g e

on delete, join 2 x 1/2 >1 x 1 3 x 2/3 >2 x 1 2 x 1/2 >1 x 1 3 x 1/2 >2 x 3/4 Table 4-1: B-Tree Implementations Several variants on the B-tree are listed in Table 4-1. The standard B-tree stores keys and data in both internal and leaf nodes. When descending the tree during insertion, a full child node is first redistributed to adjacent nodes. If the adjacent nodes are also full, then a new node is created, and half the keys in the child are moved to the newly created node. During deletion, children that are 1/2 full first attempt to obtain keys from adjacent nodes. If the adjacent nodes are also 1/2 full, then two nodes are joined to form one full node. B*-trees are similar, only the nodes are kept 2/3 full. This results in better utilization of space in the tree, and slightly better performance.

Figure 4-4: B+-Tree Figure 4-4 illustrates a B+-tree. All keys are stored at the leaf level, with their associated data values. Duplicates of the keys appear in internal parent nodes to guide the search. Pointers have a slightly different meaning than in conventional B-trees. The left pointer designates all keys less than the value, while the right pointer designates all keys greater than or equal to (GE) the value. For example, all keys less than 22 are on the left pointer, and all keys greater than or equal to 22 are on the right. Notice that key 22 is duplicated in the leaf, where the associated data may be found. During insertion and deletion, care must be taken to properly update parent nodes. When modifying the first key in a leaf, the tree is walked from leaf to root. The last GE pointer found while descending the tree will require modification to reflect the new key value. Since all keys are in the leaf nodes, we may link them for sequential access. The last method, B++-trees, is something of my own invention. The organization is similar to B+-trees, except for the split/join strategy. Assume each node can hold k keys, and the root node holds 3k keys. Before we descend to a child node during insertion, we check to see if it is full. If it is, the keys in the child node and two nodes adjacent to the child are all merged and redistributed. If the two adjacent nodes are also full, then another node is added, resulting in four nodes, each 3/4 full. Before we descend to a child node during deletion, we check to see if it is 1/2 full. If it is, the keys in the child node and two nodes adjacent to the child are all merged and redistributed. If the two adjacent nodes are also 1/2 full, then they are merged
41 | P a g e

into two nodes, each 3/4 full. This is halfway between 1/2 full and completely full, allowing for an equal number of insertions or deletions in the future. Recall that the root node holds 3k keys. If the root is full during insertion, we distribute the keys to four new nodes, each 3/4 full. This increases the height of the tree. During deletion, we inspect the child nodes. If there are only three child nodes, and they are all 1/2 full, they are gathered into the root, and the height of the tree decreases. Another way of expressing the operation is to say we are gathering three nodes, and then scattering them. In the case of insertion, where we need an extra node, we scatter to four nodes. For deletion, where a node must be deleted, we scatter to two nodes. The symmetry of the operation allows the gather/scatter routines to be shared by insertion and deletion in the implementation. Implementation An ANSI-C implementation of a B++-tree is included. In the implementation-dependent section, you'll need to define bAdrType and eAdrType, the types associated with B-tree file offsets and data file offsets, respectively. You'll also need to provide a callback function which is used by the B++-tree algorithm to compare keys. Functions are provided to insert/delete keys, find keys, and access keys sequentially. Function main, at the bottom of the file, provides a simple illustration for insertion. The code provided allows for multiple indices to the same data. This was implemented by returning a handle when the index is opened. Subsequent accesses are done using the supplied handle. Duplicate keys are allowed. Within one index, all keys must be the same length. A binary search was implemented to search each node. A flexible buffering scheme allows nodes to be retained in memory until the space is needed. If you expect access to be somewhat ordered, increasing the bufCt will reduce paging. Advantages of B-Trees

The desired record is found at a shallow depth (few disk accesses). A tree with 256 keys per node can index millions of records in 3 steps or 1 disk access (keeping the root node and next level in memory). In general, there are many more searches than insertions. Since a node can have a wide range of children, m / 2 to m, an insert or delete will rarely go outside this range. It is rarely necessary to rebalance the tree. Inserting or deleting an item within a node is O(blocksize), since on average half the block must be moved, but this is fast compared to a disk access. Rebalancing the tree on insertion is easy: if a node would become over-full, break it into two half-nodes, and insert the new key and pointer into its parent. Or, if a leaf node becomes over-full, see if a neighbor node can take some extra children. In many cases, rebalancing the tree on deletion can simply be ignored: it only wastes disk space, which is cheap.

Programs and Trees

42 | P a g e

Fundamentally, programs are trees, sometimes called abstract syntax trees or AST. Parsing converts programs in the form of character strings (source code) into trees. It is easy to convert trees back into source code form ( unparsing). Parsing - Transformation - Unparsing allows us to transform programs.

Recursion A recursive program calls itself as a subroutine. Recursion allows one to write programs that are powerful, yet simple and elegant. Often, a large problem can be handled by a small program which:
1. Tests for a base case and computes the value for this case directly.

2. Otherwise, 1. calls itself recursively to do smaller parts of the job, 2. computes the answer in terms of the answers to the smaller parts. (defun factorial (n) (if (<= n 0) 1 (* n (factorial (- n 1))) ) ) Rule: Make sure that each recursive call involves an argument that is strictly smaller than the original; otherwise, the program can get into an infinite loop.
43 | P a g e

A good method is to use a counter or data whose size decreases with each call, and to stop at 0; this is an example of a well-founded ordering. Recursion is a method where the solution to a problem depends on solutions to smaller instances of the same problem. The approach can be applied to many types of problems, and is one of the central ideas of computer science. "The power of recursion evidently lies in the possibility of defining an infinite set of objects by a finite statement. In the same manner, an infinite number of computations can be described by a finite recursive program, even if this program contains no explicit repetitions." Most computer programming languages support recursion by allowing a function to call itself within the program text. Some functional programming languages do not define any looping constructs but rely solely on recursion to repeatedly call code. Computability theory has proven that these recursive-only languages are mathematically equivalent to the imperative languages, meaning they can solve the same kinds of problems even without the typical control structures like while and for.

About Recursive Programs

If you think about what a recursive program does--how it computes its result--then that is quite difficult. If you think about what the program defines, then it is relatively simple. Recursive programs are quite closely related to mathematical induction in mathematics. In fact, you can prove that a recursive program works by mathematical induction, and that is what you must do to be able to use them really effectively. Let me explain using an example. First, let me give you an example of mathematical induction. In mathematical induction, first you show that something is true for n=1. Then you prove that if it is true for n = k, it is also true for n = k+1. Then the idea is that from the first statement you know that it is true for n = 1. From that and the second statement you know that it must be true for n=2. From that and the second statement, you know it must be true for n=3, etc. It must be true for all values of n. More specifically, the sum of the first n odd numbers is equal to n^2 (or n*n). In symbols,
1+3+5+ . . . +2n-1 = n*n;

That is true if n = 1--both sides of that equation are just 1. Now let me assume that it is true for n = k-1, i. e.
1+3+5+ . . . +2(k-1)-1 = (k-1)*(k-1)= k*k - 2*k + 1

Then what happens if we add one more odd number? The next odd number is 2k-1:
1+3+5+ . . . + 2(k-1)-1 + 2k-1 = k*k - 2*k + 1 + 2k-1 = k*k

And therefore if the statement is true for n = k-1, it is also true for n = k. Then we can conclude that it is true for all n. Now consider the following program:
44 | P a g e

int f(int n) { if(n==1) return 1; else return f(n-1)+2*n-1; }

I claim that this function always returns n*n. If n is equal to 1, then the if condition is true, and f returns 1, which is the correct answer. Assume that this returns k*k if k<n. Then by the hypothesis, f(n-1) = (n-1)*(n-1) and f(n-1) + 2*n -1 = n*n - 2*n + 1 + 2*n - 1 = n*n. So, this function returns n*n if n equals 1. Also, for any k, if it works for n = k-1, it also works for n equal to k. Therefore, taking k = 2, we conclude it works for n = 2. then taking k = 3, it works for n = 3, etc. It works for all n. This may seem a little abstract, but it is certainly simpler than trying to figure out exactly what this function does step by step. I think that if you can adopt this point of view, it is not terribly difficult to understand the recursive-descent parser.

Recursive Functions for Trees

There are a lot of neat things that you can do with recursive functions, but I think the best are functions for trees. If you have to traverse a tree, it is much simpler to do recursively than using an ordinary sequential program. Your approach should be similar to the above. Let's consider binary trees. First you should do the simplest case. That is usually the case of a tree with no nodes--an empty tree. (Sometimes you may find that it works better to do a tree that has only one node.) Otherwise, the tree has a left and a right pointer and each is pointing to a subtree. You ask yourself, if you apply this function to each of those and get the answers, then can you get the answer for the whole tree? (In the process of doing that, assume that the program will work. After you finish it, if you are inclined, you will be able to prove that it will work, using mathematical induction.) In this whole process, don't even think about exactly what steps the program goes through. That is complicated, and you don't have to even think about it. You can understand how recursive functions work. For example, you can put debug print statements in recursive functions and then look at the output and see exactly what happened. However, when you are making a recursive program, you will be a lot better off if you don't even think about what the thing is going to do. Here are a couple of examples. Let me assume trees in which each node has a number called k and left and right pointers to the subtrees. As a first example, consider a program that is supposed to get the sum of all those numbers. The base case is a tree with no nodes, and in that case, what should the answer be? Zero, of course. So you start
int sum(Struct Node *h) { if(h==NULL) return 0; else . . .

Now here is the picture:

45 | P a g e

If h is not NULL, then it is pointing to a node. If you can calculate sum(h->left) and sum(h>right) for that node, can you calculate the answer? If I say sum(left) is 100 and sum(right) is 20, then I guess you can tell me the answer--it is 137. All you have to do is to add those three numbers:
int sum(Struct Node *h) { if(h==NULL) return 0; else return h->k + sum(h->left) + sum(h->right); }

Now let's define a function int exists(struct Node *h, n); I want this function to return T (1) if there is a node in the tree that h points to in which the number is equal to n, and F (0) otherwise. Again, it works to use an empty tree as the base case.
int exists(struct Node *h, int n) { if(h == NULL) return F; else . . .

Now if h is not NULL, then it is pointing to a node, and we have this situation:

Suppose I want to know the value of exists(h, 17); I assume that exists() works OK on the left and right nodes, so I can calculate exists(h->left, 17) and exists(h->right, 17). (Don't worry about how exists calculates these values at this time.) Then can I calculate exists(h, 17)? I guess that if I told you that exists(h->left) is F and exists(h->right) is T you could tell me the answer. I guess you would say that you know that there is a 17 somewhere in the tree, in fact in the right subtree. Now clearly you know how to get the answer if you know exists(h->left, n) and exists(h->right, n). You have to be able to tell the computer how to coumpute that answer. I hope you can do that.
int exists(struct Node *h, int n) {

46 | P a g e

if(h == NULL) return F; else return ((h->k)==n || exists(h->left, n) || exists(h->right, n); }

Let's do one more example. Lets find the maximum value in the tree, int max(struct Node *h). Let's assume that all the numbers in the nodes are positive numbers. Again let's use an empty tree as the base case and define the maximum value of an empty tree to be zero. Then we can start just as before:
int max(struct Node *h) { if(h==NULL) return 0; else . . .

Now if h is not NULL, it must be pointing to a node, and this is the situation:

This node has left and right pointers. Assume that max works for them--it can calculate max(h->left) and max(h->right). Then can you calculate max for the whole tree? I guess if I told you that max(h->left) is 59 and max(h->right) is 11, you could tell me the answer--the max in the whole tree is 59.You would look at those three values, max(h->left), max(h>right) and h->k and see which is largest. You have to write a program to do that, and that is not completely trivial but it isn't very hard, either. Here is the way I like to do that:
int max(struct Node *h) { int a, c, c; if(h==NULL) return 0; else { a = h->k; b = max(h->left); c = max(h->right); if(a>=b && a>=c) return a; else if(b>=c) return b; else return c; } }

Hash Tables Direct Address Tables

47 | P a g e

If we have a collection of n elements whose keys are unique integers in (1,m), where m >= n, then we can store the items in a direct address table, T[m], where Ti is either empty or contains one of the elements of our collection. Searching a direct address table is clearly an O(1) operation: for a key, k, we access Tk,

if it contains an element, return it, if it doesn't then return a NULL.

There are two constraints here: 1. the keys must be unique, and 2. the range of the key must be severely bounded. If the keys are not unique, then we can simply construct a set of m lists and store the heads of these lists in the direct address table. The time to find an element matching an input key will still be O(1). However, if each element of the collection has some other distinguishing feature (other than its key), and if the maximum number of duplicates is ndupmax, then searching for a specific element is O(ndupmax). If duplicates are the exception rather than the rule, then ndupmax is much smaller than n and a direct address table will provide good performance. But if ndupmax approaches n, then the time to find a specific element is O(n) and a tree structure will be more efficient. The range of the key determines the size of the direct address table and may be too large to be practical. For instance it's not likely that you'll be able to use a direct address table to store elements which have arbitrary 32-bit integers as their keys for a few years yet! Direct addressing is easily generalised to the case where there is a function, h(k) => (1,m) which maps each value of the key, k, to the range (1,m). In this case, we place the element in T[h(k)] rather than T[k] and we can search in O(1) time as before.
48 | P a g e

Mapping functions The direct address approach requires that the function, h(k), is a one-to-one mapping from each k to integers in (1,m). Such a function is known as a perfect hashing function: it maps each key to a distinct integer within some manageable range and enables us to trivially build an O(1) search time table. Unfortunately, finding a perfect hashing function is not always possible. Let's say that we can find a hash function, h(k), which maps most of the keys onto unique integers, but maps a small number of keys on to the same integer. If the number of collisions (cases where multiple keys map onto the same integer), is sufficiently small, then hash tables work quite well and give O(1) search times. Handling the collisions In the small number of cases, where multiple keys map to the same integer, then elements with different keys may be stored in the same "slot" of the hash table. It is clear that when the hash function is used to locate a potential match, it will be necessary to compare the key of that element with the search key. But there may be more than one element which should be stored in a single slot of the table. Various techniques are used to manage this problem: 1. 2. 3. 4. 5. 6. chaining, overflow areas, re-hashing, using neighbouring slots (linear probing), quadratic probing, random probing, ...

Chaining One simple scheme is to chain all collisions in lists attached to the appropriate slot. This allows an unlimited number of collisions to be handled and doesn't require a priori knowledge of how many elements are contained in the collection. The tradeoff is the same as with linked lists versus array implementations of collections: linked list overhead in space and, to a lesser extent, in time. Re-hashing

49 | P a g e

Re-hashing schemes use a second hashing operation when there is a collision. If there is a further collision, we re-hash until an empty "slot" in the table is found. The re-hashing function can either be a new function or a re-application of the original one. As long as the functions are applied to a key in the same order, then a sought key can always be located. Linear probing One of the simplest re-hashing functions is +1 (or -1), ie on a collision, look in the neighbouring slot in the table. It calculates the new address extremely quickly and may be extremely efficient on a modern RISC h(j)=h(k), so the next hash function, processor due to efficient cache utilisation h1 is used. A second collision occurs, The animation gives you a practical demonstration of so h2 is used. the effect of linear probing: it also implements a quadratic re-hash function so that you can compare the difference. Clustering Linear probing is subject to a clustering phenomenon. Re-hashes from one location occupy a block of slots in the table which "grows" towards slots to which other keys hash. This exacerbates the collision problem and the number of re-hashed can become large. Quadratic Probing Better behaviour is usually obtained with quadratic probing, where the secondary hash function depends on the re-hash index: address = h(key) + c i2 on the tth re-hash. (A more complex function of i may also be used.) Since keys which are mapped to the same value by the primary hash function follow the same sequence of addresses, quadratic probing shows secondary clustering. However, secondary clustering is not nearly as severe as the clustering shown by linear probes. Re-hashing schemes use the originally allocated table space and thus avoid linked list overhead, but require advance knowledge of the number of items to be stored. However, the collision elements are stored in slots to which other key values map directly, thus the potential for multiple collisions increases as the table becomes full.
50 | P a g e

Overflow area Another scheme will divide the pre-allocated table into two sections: the primary area to which keys are mapped and an area for collisions, normally termed the overflow area. When a collision occurs, a slot in the overflow area is used for the new element and a link from the primary slot established as in a chained system. This is essentially the same as chaining, except that the overflow area is pre-allocated and thus possibly faster to access. As with re-hashing, the maximum number of elements must be known in advance, but in this case, two parameters must be estimated: the optimum size of the primary and overflow areas. Of course, it is possible to design systems with multiple overflow tables, or with a mechanism for handling overflow out of the overflow area, which provide flexibility without losing the advantages of the overflow scheme. Summary: Hash Table Organization Organization Advantages Chaining

Disadvantages number number of of

Unlimited elements Unlimited collisions

Overhead of multiple linked lists

Re-hashing

Fast re-hashing Fast access through of main table space Fast access Collisions don't use primary table space use

Maximum number of elements must be known Multiple collisions may become probable Two parameters which govern performance need to be estimated

Overflow area

Uses of Hashing There are many useful applications of hashing, including:

A compiler keeps a symbol table containing all the names declared within a program, together with information about the objects that are named.

51 | P a g e

A hash table can be used to map names to numbers. This is useful in graph theory and in networking, where a domain name is mapped to an IP address: willie.cs.utexas.edu = 128.83.130.16 Programs that play games such as chess keep hash tables of board positions that have been seen and evaluated before. In general, hashing is a good way to look up items that do not have a natural sort order. The Rabin-Karp string search algorithm uses hashing to tell whether strings of interest are present in the text being searched. This algorithm is used in plagiarism detection and DNA matching.

If an expensive function may be called repeatedly with the same argument value, pairs of (argument, result) can be saved in a hash table, with argument as the key, and result can be reused. This is called memorization. Key Terms hash table Tables which can be searched for an item in O(1) time using a hash function to form an address from the key. hash function Function which, when applied to the key, produces a integer which can be used as an address in a hash table. collision When a hash function maps two different keys to the same table address, a collision is said to occur. linear probing A simple re-hashing scheme in which the next slot in the table is checked on a collision. quadratic probing A re-hashing scheme in which a higher (usually 2nd) order function of the hash index is used to calculate the address. clustering. Tendency for clusters of adjacent slots to be filled when linear probing is used. secondary clustering. Collision sequences generated by addresses calculated with quadratic probing. perfect hash function
52 | P a g e

Function which, when applied to all the members of the set of items to be stored in a hash table, produces a unique set of integers within some suitable range. Graphs A graph G = (V, E) has a set of vertices or nodes V and a set of edges or arcs or links E, where each edge connects two vertices. We write this mathematically as E &sube V X V, where X is called the Cartesian product of two sets. We can write an edge as a pair (v1, v2), where v1 and v2 are each a vertex.

A path is a sequence of vertexes connected by edges: v1, v2, ..., vn where (vi, vi+1) &isin E. A simple path is a path with no nodes repeated, except possibly at the ends. The length of a path is the number of edges in it. A cycle is a path from a node back to itself; a graph with no cycles is acyclic. An edge may have a weight or cost associated with it. Examples of Graphs There are many examples of graphs:

The road network forms a graph, with cities as vertices and roads as edges. The distance between cities can be used as the cost of an edge. The airline network is a graph, with airports as vertices and airline flights as edges. Communication networks such as the Internet: computers and switches are nodes, connections between them are links. Social networks such as the graph of people who call each other on the telephone, or friends on MySpace: people are nodes and there are links to the people they communicate with. Distribution networks that model the flow of goods: an oil terminal is a node, and an oil tanker or pipeline is a link.

Directed Acyclic Graph

53 | P a g e

If connections are one-way, from v1 to v2, we say the graph is directed ; otherwise, it is undirected. A directed graph is sometimes called a digraph. Directed acyclic graphs or DAG representations such as trees are important in Computer Science.

Graph Representations We want the internal representation of a graph to be one that can be efficiently manipulated. If the external representation of a node is a string, such as a city name, we can use a Map or symbol table to map it to an internal representation such as:

a node number, convenient to access arrays of information about the node a pointer to a node object.

A graph is called dense if |E| = O(|V|2); if |E| is less, the graph is called sparse . Most graphs used in applications are sparse. Adjacency List In the adjacency list representation of a graph, each node has a list of nodes that are adjacent to it, i.e. connected by an edge. A linked list is a natural representation.

1 (5 2)
54 | P a g e

2 3 4 5 6

(3 5 1) (4 2) (6 3 5) (4 2 1) (4)

This graph is undirected, so each link is represented twice. The storage required is O(|V| + |E|). This is a good representation if the graph is sparse, i.e. each node is not linked to many others. Implicit Graphs Some graphs must be represented implicitly because they cannot be represented explicitly. For example, the graph of all possible chess positions is larger than the number of elementary particles in the universe. In such cases, only part of the graph will be explicitly considered, such as the chess positions that can be reached from the current position in 7 moves or less. Topological Sort Some graphs specify an order in which things must be done; a common example is the course prerequisite structure of a university. A topological sort orders the vertices of a directed acyclic graph (DAG) so that if there is a path from vertex vi to vj, vj comes after vi in the ordering. A topological sort is not necessarily unique. An example of a topological sort is a sequence of taking classes that is legal according to the prerequisite structure. An easy way to find a topological sort is:

initialize a queue to contain all vertices that have no incoming arcs. While the queue is not empty, o remove a vertex from the queue, o put it into the sort order o remove all of its arcs o If the target of an arc now has zero incoming arcs, add the target to the queue.

Topological Sort Some graphs specify an order in which things must be done; a common example is the course prerequisite structure of a university. A topological sort orders the vertices of a directed acyclic graph (DAG) so that if there is a path from vertex vi to vj, vj comes after vi in the ordering. A topological sort is not necessarily unique. An example of a topological sort is a sequence of taking classes that is legal according to the prerequisite structure.
55 | P a g e

An easy way to find a topological sort is:

initialize a queue to contain all vertices that have no incoming arcs. While the queue is not empty, o remove a vertex from the queue, o put it into the sort order o remove all of its arcs o If the target of an arc now has zero incoming arcs, add the target to the queue.

Uses of Topological Sort Topological Sort has many uses:

PERT technique for scheduling of tasks Instruction scheduling in compilers Deciding what to update in spreadsheets Deciding what files to re-compile in makefiles Resolving symbol dependencies in linkers.

PERT Chart: Calculating Times The time of events can be found as follows:

The time of the initial node is 0. For each node j in the topological sort after the first, timej = max (i,j) &isin E (timei + costi, j)

By considering nodes in topological sort order, we know that the time of each predecessor will be computed before it is needed. We can compute the latest completion time for each node, in reverse topological order, as:

The latest time of the final node n is timen. For node i, latesti = min (i,j) &isin E (latestj - costi, j)

Slack for an edge (i,j) is: slacki,j = latestj - timei - costi, j Shortest Path Problem An important graph problem is the shortest path problem, namely to find a path from a start node to a goal node such that the sum of weights along the path is minimized. We will assume that all weights are non-negative. Shortest path routing algorithms are used by web sites that suggest driving directions, such as MapQuest. Dijkstra's Algorithm public void dijkstra( Vertex s ) { for ( Vertex v : vertices ) { v.visited = false;
56 | P a g e

v.cost = 999999; } s.cost = 0; s.parent = null; PriorityQueue<Vertex> fringe = new PriorityQueue<Vertex>(20, new Comparator<Vertex>() { public int compare(Vertex i, Vertex j) { return (i.cost - j.cost); }}); fringe.add(s); while ( ! fringe.isEmpty() ) { Vertex v = fringe.remove(); // lowest-cost if ( ! v.visited ) { v.visited = true; for ( Edge e : v.edges ) { int newcost = v.cost + e.cost; if ( newcost < e.target.cost ) { e.target.cost = newcost; e.target.parent = v; fringe.add(e.target); } } } } } Adjacency Matrix In the adjacency matrix representation of a graph, a Boolean matrix contains a 1 in position (i,j) iff there is a link from vi to vj, otherwise 0.

1 2 3 4 5 6

123456 010010 101010 010100 001011 110100 000100

Since this graph is undirected, each link is represented twice, and the matrix is symmetric.

57 | P a g e

The storage required is O(|V|2); even though only one bit is used for each entry, the storage can be excessive. PERT Chart PERT, for Program Evaluation and Review Technique, is a project management method using directed graphs.

Nodes represent events, milestones, or time points. Arcs represent activities, which take time and resources. Each arc is labeled with the amount of time it takes. A node has the maximum time of its incoming arcs. An activity cannot start before the time of its preceding event. The critical path is the longest path from start to finish. The slack is the amount of extra time available to an activity before it would become part of the critical path.

Dijkstra's Algorithm public void dijkstra( Vertex s ) { for ( Vertex v : vertices ) { v.visited = false; v.cost = 999999; } s.cost = 0; s.parent = null; PriorityQueue<Vertex> fringe = new PriorityQueue<Vertex>(20, new Comparator<Vertex>() { public int compare(Vertex i, Vertex j) { return (i.cost - j.cost); }}); fringe.add(s); while ( ! fringe.isEmpty() ) { Vertex v = fringe.remove(); // lowest-cost if ( ! v.visited ) { v.visited = true; for ( Edge e : v.edges )
58 | P a g e

{ int newcost = v.cost + e.cost; if ( newcost < e.target.cost ) { e.target.cost = newcost; e.target.parent = v; fringe.add(e.target); } } } } } Adjacency Matrix In the adjacency matrix representation of a graph, a Boolean matrix contains a 1 in position (i,j) iff there is a link from vi to vj, otherwise 0.

1 2 3 4 5 6

123456 010010 101010 010100 001011 110100 000100

Since this graph is undirected, each link is represented twice, and the matrix is symmetric. The storage required is O(|V|2); even though only one bit is used for each entry, the storage can be excessive. PERT Chart PERT, for Program Evaluation and Review Technique, is a project management method using directed graphs.

59 | P a g e

Nodes represent events, milestones, or time points. Arcs represent activities, which take time and resources. Each arc is labeled with the amount of time it takes. A node has the maximum time of its incoming arcs. An activity cannot start before the time of its preceding event. The critical path is the longest path from start to finish. The slack is the amount of extra time available to an activity before it would become part of the critical path.

Dijkstra's Algorithm public void dijkstra( Vertex s ) { for ( Vertex v : vertices ) { v.visited = false; v.cost = 999999; } s.cost = 0; s.parent = null; PriorityQueue<Vertex> fringe = new PriorityQueue<Vertex>(20, new Comparator<Vertex>() { public int compare(Vertex i, Vertex j) { return (i.cost - j.cost); }}); fringe.add(s); while ( ! fringe.isEmpty() ) { Vertex v = fringe.remove(); // lowest-cost if ( ! v.visited ) { v.visited = true; for ( Edge e : v.edges ) { int newcost = v.cost + e.cost; if ( newcost < e.target.cost ) { e.target.cost = newcost; e.target.parent = v; fringe.add(e.target); } } } } } Dijkstra's Algorithm Example
60 | P a g e

Dijkstra's Algorithm finds the shortest path to all nodes from a given start node, producing a tree.

Minimum Spanning Tree A minimum spanning tree is a subgraph of a given undirected graph, containing all the nodes and a subset of the arcs, such that:

All nodes are connected. The resulting graph is a tree, i.e. there are no cycles. The sum of weights on the remaining arcs is as low as possible.

Example: Connect a set of locations to the Internet using a minimum length of cable. The minimum spanning tree may not be unique, but all MST's will have the same total cost of arcs.

61 | P a g e

Memory Management and Garbage Collection

Memory Management Memory management is the act of managing computer memory. In its simpler forms, this involves providing ways to allocate portions of memory to programs at their request, and freeing it for reuse when no longer needed. The management of main memory is critical to the computer system. Virtual memory systems separate the memory addresses used by a process from actual physical addresses, allowing separation of processes and increasing the effectively available amount of RAM using paging or swapping to secondary storage. The quality of the virtual memory manager can have a big impact on overall system performance. Garbage collection is the automated allocation and deallocation of computer memory resources for a program. This is generally implemented at the programming language level and is in opposition to manual memory management, the explicit allocation and deallocation of computer memory resources. Region-based memory management is an efficient variant of explicit memory management that can deallocate large groups of objects simultaneously.

Requirements
Memory management systems on multi-tasking operating systems usually deal with the following issues.
Relocation

In systems with virtual memory, programs in memory must be able to reside in different parts of the memory at different times. This is because there is often not enough free space in one location of memory to fit the entire program. The virtual memory management unit must also deal with concurrency. Memory management in the operating system should therefore be able to relocate programs in memory and handle memory references and addresses in the code of the program so that they always point to the right location in memory.
62 | P a g e

Protection

Processes should not be able to reference the memory for another process without permission. This is called memory protection, and prevents malicious or malfunctioning code in one program from interfering with the operation of other running programs. Even though the memory for different processes is normally protected from each other, different processes sometimes need to be able to share information and therefore access the same part of memory. Shared memory is one of the fastest techniques for Inter-process communication.
Logical organization

Programs are often organized in modules. Some of these modules could be shared between different programs, some are read only and some contain data that can be modified. The memory management is responsible for handling this logical organization that is different from the physical linear address space. One way to arrange this organization is segmentation.
] Physical organization

Memory is usually divided into fast primary storage and slow secondary storage. Memory management in the operating system handles moving information between these two levels of memory.

Collection (GC) Garbage collection is a form of automatic memory management. The garbage collector, or just collector, attempts to reclaim garbage, or memory occupied by objects that are no longer in use by the program. Garbage collection was invented by John McCarthy around 1959 to solve problems in Lisp. Garbage collection is often portrayed as the opposite of manual memory management, which requires the programmer to specify which objects to deallocate and return to the memory system. However, many systems use a combination of the two approaches, and other techniques such as stack allocation and region inference can carve off parts of the problem. There is an ambiguity of terms, as theory often uses the terms manual garbage collection and automatic garbage collection rather than manual memory management and garbage collection, and does not restrict garbage collection to memory management, rather considering that any logical or physical resource may be garbage collected. Garbage collection does not traditionally manage limited resources other than memory that typical programs use, such as network sockets, database handles, user interaction windows, and file and device descriptors. Methods used to manage such resources, particularly destructors, may suffice as well to manage memory, leaving no need for GC. Some GC systems allow such other resources to be associated with a region of memory that, when collected, causes the other resource to be reclaimed; this is called finalization. Finalization may introduce complications limiting its usability, such as intolerable latency between disuse
63 | P a g e

and reclaim of especially limited resources, or a lack of control over which thread performs the work of reclaiming. The basic principles of garbage collection are:
1. Find data objects in a program that cannot be accessed in the future 2. Reclaim the resources used by those objects

Many computer languages require garbage collection, either as part of the language specification (e.g., Java, C#, and most scripting languages) or effectively for practical implementation (e.g., formal languages like lambda calculus); these are said to be garbage collected languages. Other languages were designed for use with manual memory management, but have garbage collected implementations available (e.g., C, C++). Some languages, like Ada, Modula-3, and C++/CLI allow both garbage collection and manual memory management to co-exist in the same application by using separate heaps for collected and manually managed objects; others, like D, are garbage collected but allow the user to manually delete objects and also entirely disable garbage collection when speed is required. While integrating garbage collection into the language's compiler and runtime system enables a much wider choice of methods,[citation needed] post hoc GC systems exist, including some that do not require recompilation. (Post-hoc GC is sometimes distinguished as litter collection.) The garbage collector will almost always be closely integrated with the memory allocator.
Benefits

Garbage collection frees the programmer from manually dealing with memory deallocation. As a result, certain categories of bugs are eliminated or substantially reduced:

Dangling pointer bugs, which occur when a piece of memory is freed while there are still pointers to it, and one of those pointers is then used. By then the memory may have been re-assigned to another use, with unpredictable results. Double free bugs, which occur when the program tries to free a region of memory that has already been freed, and perhaps already been allocated again. Certain kinds of memory leaks, in which a program fails to free memory occupied by objects that will not be used again, leading, over time, to memory exhaustion.

Some of these bugs can have security implications.

Disadvantages

Typically, garbage collection has certain disadvantages:

Garbage collection consumes computing resources in deciding which memory to free, reconstructing facts that may have been known to the programmer. The penalty for the convenience of not annotating object lifetime manually in the source code is overhead, oftenleading to decreased or uneven performance. Interaction with memory hierarchy

64 | P a g e

effects can make this overhead intolerable in circumstances that are hard to predict or to detect in routine testing. The moment when the garbage is actually collected can be unpredictable, resulting in stalls scattered throughout a session. Unpredictable stalls can be unacceptable in real-time environments such as device drivers, in transaction processing, or in interactive programs. This unpredictability also means that techniques such as RAII, which allows for automatic cleaning up of resources (e.g. closing file handles, deleting temporary files, etc.), cannot be used, since the cleanup might happen too late. See the Determinism section for a detailed discussion. Memory may leak despite the presence of a garbage collector, if references to unused objects are not themselves manually disposed of. This is described as a logical memory leak.[3] For example, recursive algorithms normally delay release of stack objects until after the final call has completed. Caching and memoizing, common optimization techniques, commonly lead to such logical leaks. The belief that garbage collection eliminates all leaks leads many programmers not to guard against creating such leaks. In virtual memory environments typical of modern desktop computers, it can be difficult for the garbage collector to notice when collection is needed, resulting[citation needed] in large amounts of accumulated garbage, a long, disruptive collection phase, and other programs' data being swapped out. Programs that rely on garbage collectors often exhibit poor locality (interacting badly with cache and virtual memory systems), occupy more address space than the program actually uses at any one time, and touch otherwise idle pages. These may combine in a phenomenon called thrashing, in which a program spends more time copying data between various grades of storage than performing useful work.

Tracing garbage collectors

Tracing garbage collectors are the most common type of garbage collector. They first determine which objects are reachable (or potentially reachable), and then discard all remaining objects.
Reachability of an object

Informally, a reachable object can be defined as an object for which there exists some variable in the program environment that leads to it, either directly or through references from other reachable objects. More precisely, objects can be reachable in only two ways:
1. A distinguished set of objects are assumed to be reachable: these are known as the roots. Typically, these include all the objects referenced from anywhere in the call stack (that is, all local variables and parameters in the functions currently being invoked), and any global variables. 2. Anything referenced from a reachable object is itself reachable; more formally, reachability is a transitive closure.

The reachability definition of "garbage" is not optimal, insofar as the last time a program uses an object could be long before that object falls out of the environment scope. A distinction is sometimes drawn between syntactic garbage, those objects the program cannot possibly
65 | P a g e

reach, and semantic garbage, those objects the program will in fact never again use. For example:
Object x = new Foo(); Object y = new Bar(); x = new Quux(); /* at this point, we know that the Foo object will * never be accessed: it is syntactic garbage */ if(x.check_something()) { x.do_something(y); } System.exit(0); /* in the above block, y *could* be semantic garbage, * but we won't know until x.check_something() returns * some value: if it returns at all */

The problem of precisely identifying semantic garbage can easily be shown to be partially decidable: a program that allocates an object X, runs an arbitrary input program P, and uses X if and only if P finishes would require a semantic garbage collector to solve the halting problem. Although conservative heuristic methods for semantic garbage detection remain an active research area, essentially all practical garbage collectors focus on syntactic garbage. Another complication with this approach is that, in languages with both reference types and unboxed value types, the garbage collector needs to somehow be able to distinguish which variables on the stack or fields in an object are regular values and which are references: in memory, an integer and a reference might look alike. The garbage collector then needs to know whether to treat the element as a reference and follow it, or whether it is a primitive value. One common solution is the use of tagged pointers.
Strong and weak references

The garbage collector can reclaim only objects that have no references. An object that is reachable cannot be garbage collected by the garbage collector. Such a reference is known as a strong reference. An object can also be referred to as a weak reference. An object is eligible for garbage collection if there are no strong references to it (even though there still might be some weak references to it). In some implementations, notably in Microsoft .NET, the weak references are divided into two further subcategories: long weak references (tracks resurrection) and short weak references.
Basic algorithm

Tracing collectors are so called because they trace through the working set of memory. These garbage collectors perform collection in cycles. A cycle is started when the collector decides (or is notified) that it needs to reclaim memory, which happens most often when the system is low on memory. The original method involves a nave mark-and-sweep in which the entire memory set is touched several times.
66 | P a g e

Nave mark-and-sweep In the nave mark-and-sweep method, each object in memory has a flag (typically a single bit) reserved for garbage collection use only. This flag is always cleared, except during the collection cycle. The first stage of collection does a tree traversal of the entire 'root set', marking each object that is pointed to as being 'in-use'. All objects that those objects point to, and so on, are marked as well, so that every object that is ultimately pointed to from the root set is marked. Finally, all memory is scanned from start to finish, examining all free or used blocks; those with the in-use flag still cleared are not reachable by any program or data, and their memory is freed. (For objects which are marked in-use, the in-use flag is cleared again, preparing for the next cycle.) This method has several disadvantages, the most notable being that the entire system must be suspended during collection; no mutation of the working set can be allowed. This will cause programs to 'freeze' periodically (and generally unpredictably), making real-time and timecritical applications impossible. In addition, the entire working memory must be examined, much of it twice, potentially causing problems in paged memory systems. Tri-colour marking Because of these pitfalls, most modern tracing garbage collectors implement some variant of the tri-colour marking abstraction, but simple collectors (such as the mark-and-sweep collector) often do not make this abstraction explicit. Tri-colour marking works as follows:
1. Create initial white, grey, and black sets; these sets will be used to maintain progress during the cycle. o Initially the white set or condemned set is the set of objects that are candidates for having their memory recycled. o The black set is the set of objects that cheaply can be proven to have no references to objects in the white set; in many implementations the black set starts off empty. o The grey set is all the objects that are reachable from root references but the objects referenced by grey objects haven't been scanned yet. Grey objects are known to be reachable from the root, so cannot be garbage collected: grey objects will eventually end up in the black set. The grey state means we still need to check any objects that the object references. o The grey set is initialised to objects which are referenced directly at root level; typically all other objects are initially placed in the white set. o Objects can move from white to grey to black, never in the other direction. 2. Pick an object from the grey set. Blacken this object (move it to the black set), by greying all the white objects it references directly. This confirms that this object cannot be garbage collected, and also that any objects it references cannot be garbage collected. 3. Repeat the previous step until the grey set is empty. 4. When there are no more objects in the grey set, then all the objects remaining in the white set have been demonstrated not to be reachable, and the storage occupied by them can be reclaimed.

67 | P a g e

The 3 sets partition memory; every object in the system, including the root set, is in precisely one set. The tri-colour marking algorithm preserves an important invariant:
No black object points directly to a white object.

This ensures that the white objects can be safely destroyed once the grey set is empty. (Some variations on the algorithm do not preserve the tricolour invariant but they use a modified form for which all the important properties hold.) The tri-colour method has an important advantage: it can be performed 'on-the-fly', without halting the system for significant time periods. This is accomplished by marking objects as they are allocated and during mutation, maintaining the various sets. By monitoring the size of the sets, the system can perform garbage collection periodically, rather than as-needed. Also, the need to touch the entire working set each cycle is avoided.
Implementation strategies

In order to implement the basic tri-colour algorithm, several important design decisions must be made, which can significantly affect the performance characteristics of the garbage collector. Moving vs. non-moving Once the unreachable set has been determined, the garbage collector may simply release the unreachable objects and leave everything else as it is, or it may copy some or all of the reachable objects into a new area of memory, updating all references to those objects as needed. These are called "non-moving" and "moving" garbage collectors, respectively. At first, a moving GC strategy may seem inefficient and costly compared to the non-moving approach, since much more work would appear to be required on each cycle. In fact, however, the moving GC strategy leads to several performance advantages, both during the garbage collection cycle itself and during actual program execution:

No additional work is required to reclaim the space freed by dead objects; the entire region of memory from which reachable objects were moved can be considered free space. In contrast, a non-moving GC must visit each unreachable object and somehow record that the memory it alone occupied is available. Similarly, new objects can be allocated very quickly. Since large contiguous regions of memory are usually made available by the moving GC strategy, new objects can be allocated by simply incrementing a 'free memory' pointer. A non-moving strategy may, after some time, lead to a heavily fragmented heap, requiring expensive consultation of "free lists" of small available blocks of memory in order to allocate new objects. If an appropriate traversal order is used (such as cdr-first for list conses), objects that refer to each other frequently can be moved very close to each other in memory, increasing the likelihood that they will be located in the same cache line or virtual memory page. This can significantly speed up access to these objects through these references.

68 | P a g e

One disadvantage of a moving garbage collector is that it only allows access through references that are managed by the garbage collected environment, and does not allow pointer arithmetic. This is because any native pointers to objects will be invalidated when the garbage collector moves the object (they become dangling pointers). For interoperability with native code, the garbage collector must copy the object contents to a location outside of the garbage collected region of memory. An alternative approach is to pin the object in memory, preventing the garbage collector from moving it and allowing the memory to be directly shared with native pointers (and possibly allowing pointer arithmetic).[5] Copying vs. mark-and-sweep vs. mark-and-don't-sweep To further refine the distinction, tracing collectors can also be divided by considering how the three sets of objects (white, grey, and black) are maintained during a collection cycle. The most straightforward approach is the semi-space collector, which dates to 1969. In this moving GC scheme, memory is partitioned into a "from space" and "to space". Initially, objects are allocated into "to space" until they become full and a collection is triggered. At the start of a collection, the "to space" becomes the "from space", and vice versa. The objects reachable from the root set are copied from the "from space" to the "to space". These objects are scanned in turn, and all objects that they point to are copied into "to space", until all reachable objects have been copied into "to space". Once the program continues execution, new objects are once again allocated in the "to space" until it is once again full and the process is repeated. This approach has the advantage of conceptual simplicity (the three object color sets are implicitly constructed during the copying process), but the disadvantage that a (possibly) very large contiguous region of free memory is necessarily required on every collection cycle. This technique is also known as stop-and-copy. Cheney's algorithm is an improvement on the semi-space collector. A mark and sweep garbage collector maintains a bit (or two) with each object to record whether it is white or black; the grey set is either maintained as a separate list (such as the process stack) or using another bit. As the reference tree is traversed during a collection cycle (the "mark" phase), these bits are manipulated by the collector to reflect the current state. A final "sweep" of the memory areas then frees white objects. The mark and sweep strategy has the advantage that, once the unreachable set is determined, either a moving or non-moving collection strategy can be pursued; this choice of strategy can even be made at runtime, as available memory permits. It has the disadvantage of "bloating" objects by a small amount. A mark and don't sweep garbage collector, like the mark-and-sweep, maintains a bit with each object to record whether it is white or black; the gray set is either maintained as a separate list (such as the process stack) or using another bit. There are two key differences here. First, black and white mean different things than they do in the mark and sweep collector. In a "mark and don't sweep" system, all reachable objects are always black. An object is marked black at the time it is allocated, and it will stay black even if it becomes unreachable. A white object is unused memory and may be allocated. Second, the interpretation of the black/white bit can change. Initially, the black/white bit may have the sense of (0=white, 1=black). If an allocation operation ever fails to find any available (white) memory, that means all objects are marked used (black). The sense of the black/white bit is then inverted (for example, 0=black, 1=white). Everything becomes white. This momentarily breaks the invariant that reachable objects are black, but a full marking phase follows
69 | P a g e

immediately, to mark them black again. Once this is done, all unreachable memory is white. No "sweep" phase is necessary. Generational GC (ephemeral GC) It has been empirically observed that in many programs, the most recently created objects are also those most likely to become unreachable quickly (known as infant mortality or the generational hypothesis). A generational GC (also known as ephemeral GC) divides objects into generations and, on most cycles, will place only the objects of a subset of generations into the initial white (condemned) set. Furthermore, the runtime system maintains knowledge of when references cross generations by observing the creation and overwriting of references. When the garbage collector runs, it may be able to use this knowledge to prove that some objects in the initial white set are unreachable without having to traverse the entire reference tree. If the generational hypothesis holds, this results in much faster collection cycles while still reclaiming most unreachable objects. In order to implement this concept, many generational garbage collectors use separate memory regions for different ages of objects. When a region becomes full, those few objects that are referenced from older memory regions are promoted (copied) up to the next highest region, and the entire region can then be overwritten with fresh objects. This technique permits very fast incremental garbage collection, since the garbage collection of only one region at a time is all that is typically required. Generational garbage collection is a heuristic approach, and some unreachable objects may not be reclaimed on each cycle. It may therefore occasionally be necessary to perform a full mark and sweep or copying garbage collection to reclaim all available space. In fact, runtime systems for modern programming languages (such as Java and the .NET Framework) usually use some hybrid of the various strategies that have been described thus far; for example, most collection cycles might look only at a few generations, while occasionally a mark-and-sweep is performed, and even more rarely a full copying is performed to combat fragmentation. The terms "minor cycle" and "major cycle" are sometimes used to describe these different levels of collector aggression. Stop-the-world vs. incremental vs. concurrent Simple stop-the-world garbage collectors completely halt execution of the program to run a collection cycle, thus guaranteeing that new objects are not allocated and objects do not suddenly become unreachable while the collector is running. This has the obvious disadvantage that the program can perform no useful work while a collection cycle is running (sometimes called the "embarrassing pause"). Stop-the-world garbage collection is therefore mainly suitable for non-interactive programs. Its advantage is that it is both simpler to implement and faster than incremental garbage collection. Incremental and concurrent garbage collectors are designed to reduce this disruption by interleaving their work with activity from the main program. Incremental garbage collectors perform the garbage collection cycle in discrete phases, with program execution permitted between each phase (and sometimes during some phases). Concurrent garbage collectors do not stop program execution at all, except perhaps briefly when the program's execution stack
70 | P a g e

is scanned. However, the sum of the incremental phases takes longer to complete than one batch garbage collection pass, so these garbage collectors may yield lower total throughput. Careful design is necessary with these techniques to ensure that the main program does not interfere with the garbage collector and vice versa; for example, when the program needs to allocate a new object, the runtime system may either need to suspend it until the collection cycle is complete, or somehow notify the garbage collector that there exists a new, reachable object. Precise vs. conservative and internal pointers Some collectors can correctly identify all pointers (references) in an object; these are called precise (also exact or accurate) collectors, the opposite being a conservative or partly conservative collector. Conservative collectors assume that any bit pattern in memory could be a pointer if, interpreted as a pointer, it would point into an allocated object. Conservative collectors may produce false positives, where unused memory is not released because of improper pointer identification. This is not always a problem in practice unless the program handles a lot of data that could easily be misidentified as a pointer. False positives are generally less problematic on 64-bit systems than on 32-bit systems because the range of valid memory addresses tends to be a tiny fraction of the range of 64-bit values. Thus, an arbitrary 64-bit pattern is unlikely to mimic a valid pointer. Whether a precise collector is practical usually depends on the type safety properties of the programming language in question. An example for which a conservative garbage collector would be needed is the C language, which allows typed (non-void) pointers to be type cast into untyped (void) pointers, and vice versa. A related issue concerns internal pointers, or pointers to fields within an object. If the semantics of a language allow internal pointers, then there may be many different addresses that can refer to parts of the same object, which complicates determining whether an object is garbage or not. An example for this is the C++ language, in which multiple inheritance can cause pointers to base objects to have different addresses. Even in languages like Java, internal pointers can exist during the computation of, say, an array element address. In a tightly-optimized program, the corresponding pointer to the object itself may have been overwritten in its register, so such internal pointers need to be scanned.
Performance implications

Tracing garbage collectors require some implicit runtime overhead that may be beyond the control of the programmer, and can sometimes lead to performance problems. For example, commonly used stop-the-world garbage collectors, which pause program execution at arbitrary times, may make garbage collection inappropriate for some embedded systems, high-performance server software, and applications with real-time needs.
Manual heap allocation

search for best/first-fit block of sufficient size free list maintenance

Garbage collection 71 | P a g e

locate reachable objects copy reachable objects for moving collectors read/write barriers for incremental collectors search for best/first-fit block and free list maintenance for non-moving collectors

It is difficult to compare the two cases directly, as their behavior depends on the situation. For example, in the best case for a garbage collecting system, allocation just increments a pointer, but in the best case for manual heap allocation, the allocator maintains freelists of specific sizes and allocation only requires following a pointer. However, this size segregation usually cause a large degree of external fragmentation, which can have an adverse impact on cache behaviour. Memory allocation in a garbage collected language may be implemented using heap allocation behind the scenes (rather than simply incrementing a pointer), so the performance advantages listed above don't necessarily apply in this case. In some situations, most notably embedded systems, it is possible to avoid both garbage collection and heap management overhead by preallocating pools of memory and using a custom, lightweight scheme for allocation/deallocation.[6] The overhead of write barriers is more likely to be noticeable in an imperative-style program which frequently writes pointers into existing data structures than in a functional-style program which constructs data only once and never changes them. Some advances in garbage collection can be understood as reactions to performance issues. Early collectors were stop-the-world collectors, but the performance of this approach was distracting in interactive applications. Incremental collection avoided this disruption, but at the cost of decreased efficiency due to the need for barriers. Generational collection techniques are used with both stop-the-world and incremental collectors to increase performance; the trade-off is that some garbage is not detected as such for longer than normal.
Determinism

Tracing garbage collection is not deterministic. An object which becomes eligible for garbage collection will usually be cleaned up eventually, but there is no guarantee when (or even if) that will happen. This can cause problems:

Most environments with tracing GC require manual deallocation of limited non-memory resources, as an automatic deallocation during the garbage collection phase (usually using a finalizer) may run too late or in the wrong circumstances. The performance impact caused by GC is seemingly random and hard to predict.

Reference counting
Reference counting is a form of automatic memory management where each object has a count of the number of references to it. An object's reference count is incremented when a
72 | P a g e

reference to it is created, and decremented when a reference is destroyed. The object's memory is reclaimed when the count reaches zero. There are two major disadvantages to reference counting:

If two or more objects refer to each other, they can create a cycle whereby neither will be collected as their mutual references never let their reference counts become zero. Some garbage collection systems using reference counting (like the one in CPython) use specific cycle-detecting algorithms to deal with this issue.[7] In naive implementations, each assignment of a reference and each reference falling out of scope often require modifications of one or more reference counters. However, optimizations to this are described in the literature.[clarification needed] When used in a multithreaded environment, these modifications (increment and decrement) may need to be interlocked. This may be an expensive operation for processors without atomic operations such as compare-and-swap.[citation needed]

One important advantage of reference counting is that it provides deterministic garbage collection (as opposed to tracing GC).

Escape analysis
Escape analysis can be used to convert heap allocations to stack allocations, thus reducing the amount of work needed to be done by the garbage collector.

Availability
Generally speaking, higher-level programming languages are more likely to have garbage collection as a standard feature. In languages that do not have built in garbage collection, it can often be added through a library, as with the Boehm garbage collector for C and C++. This approach is not without drawbacks, such as changing object creation and destruction mechanisms. Most functional programming languages, such as ML, Haskell, and APL, have garbage collection built in. Lisp, which introduced functional programming, is especially notable for introducing this mechanism. Other dynamic languages, such as Ruby (but not Perl 5, or PHP, which use reference counting), also tend to use GC. Object-oriented programming languages such as Smalltalk, Java and ECMAScript usually provide integrated garbage collection. Notable exceptions are C++ and Delphi which have destructors. Objective-C has not traditionally had it, but ObjC 2.0 as implemented by Apple for Mac OS X uses a runtime collector developed in-house, while the GNUstep project uses a Boehm collector. Historically, languages intended for beginners, such as BASIC and Logo, have often used garbage collection for variable-length data types, such as strings and lists, so as not to burden programmers with manual memory management. On early microcomputers, with their limited memory and slow processors, BASIC garbage collection could often cause apparently random, inexplicable pauses in the midst of program operation.
73 | P a g e

Limited environments
Garbage collection is rarely used on embedded or real-time systems because of the perceived need for very tight control over the use of limited resources. However, garbage collectors compatible with such limited environments have been developed.[8] The Microsoft .NET Micro Framework and Java Platform, Micro Edition are embedded software platforms that, like their larger cousins, include garbage collection

Scripts & Macros

Scripts are used for many things on computers. Everything from customizing and automating repetitious tasks to changing the way the computer functions can be controlled with scripts. One example of a script is a batch file and the most common of these is the AUTOEXEC.BAT file. With older versions of Windows, this script contained the steps that the computer went through when starting up. The CONFIG.SYS file controls how your computer's hardware is configure each time you restart it. These type of files contain instructions for your computer; one instruction per line. These instructions are operating system commands and can be modified in any text editor. Always be sure to make a backup before modifying a *.BAT file. One of the most common scripts that the average user will come in contact with are macros. Most programs use some form of macro. A macro, at it's simplest, is a recorded series of keystrokes that help automate repetitive tasks. These tasks, once copied into a script, can be accomplished with a few keystrokes. You can use macros to help you write letters, create memos, or build reports. Some macros stop and beep when you need to enter information. Some present a screen with detailed information and multiple choices. Many programs allow the user to record personalized macros for their own unique use such as inserting your name and address. Most computer users will use scripts in some way, perhaps without realizing it. One common script that users often use are Wizards or scripts that install new software. These type of scripts will take you step by step through complex processes and stop a certain points to offer users different choices. On the Internet there are a number of script languages including JavaScript, Perl, VBScript, PHP and many others. These programming script languages allow website programmers to create many interesting and useful functions. These scripts are often written into web pages or stored on the server that you connect to. These type of scripts are used for processing forms, keeping statistics, counting visitors to website, querying databases as well as limitless other processes with more being introduced each day. There are many scripting languages and programming languages designed to be used with programming tools or as stand-alone programs

74 | P a g e