How does Big-O Notation work, and can you provide an example?

First and foremost, do not even walk into a software interview without knowing what Big O Analysis is all about you will embarrass yourself. Its simply something that you must know if you expect to get a job in this industry. Here we present a tutorial on Big O Notation, along with some simple examples to really help you understand it.

What is Big O Analysis a tutorial:

When solving a computer science problem there will usually be more than just one solution. These solutions will often be in the form of different algorithms, and you will generally want to compare the algorithms to see which one is more efficient. This is where Big O analysis helps it gives us some basis for measuring the efficiency of an algorithm. A more detailed explanation of Big O analysis would be this: it measures the efficiency of an algorithm based on the time it takes for the algorithm to run as a function of the input size. Think of the input simply as what goes into a function whether it be an array of numbers, a linked list, etc. Sounds quite boring, right? Its really not that bad at all and it is something best illustrated by an example with actual code samples.

Example of Big O Analysis

Lets suppose that we are given a problem in which we want to create a function that, when given an array of integers greater than 0, will return the integer that is the smallest in that array. In order to best illustrate the way Big-O analysis works, we will come up with twodifferent solutions to this problem, each with a different Big-O efficiency. Heres our first function that will simply return the integer that is the smallest in the array. The algorithm will just iterate through all of the values in the array and keep track of the smallest integer in the array in the variable called curMin. Lets assume that the array being passed to our function contains 10 elements this number is something we arbitrarily chose. We could have said it contains 100, or 100000 elements either way it would have made no difference for our purposes here.

int CompareSmallestNumber (int array[ ]) { int x, curMin;

// set smallest value to first item in array curMin = array[0]; // iterate through array to find smallest value for (x = 1; x < 10; x++) { if( array[x] < curMin) { curMin = array[x]; } } // return smallest value in the array return curMin; }

As promised, we want to show you another solution to the problem. In this solution, we will use a different algorithm. What we do is compare each value in the array to all of the other numbers in the array, and if that value is less than or equal to all of the other numbers in the array then we know that it is the smallest number in the array.

int CompareToAllNumbers (int array[ ]) { bool is Min; int x, y; // iterate through each for (int x = 0; x < 10; x++) { isMin = true; for (int y = 0; y < 10; y++) { /* compare the value in array[x] to the other values if we find that array[x] is greater than any of the values in array[y] then we know that the value in array[x] is not the minimum remember that the 2 arrays are exactly the same, we are just taking out one value with index 'x' and comparing to the other values in the array with index 'y' */

if( array[x] > array[y]) isMin = false; } if(isMin) break; } return array[x]; }

Now, you've seen 2 functions that solve the same problem - but each one uses a different algorithm. We want to be able to say which algorithm is more efficient, and Big-O analysis allows us to do exactly that.

Big O analysis in action

For our purposes, we assumed an input size of 10 for the array. But when doing Big O analysis, we don't want to use specific numbers for the input size - so we say that the input is of size n. Remember that Big-O analysis is used to measure the efficiency of an algorithm based on the time it takes for the algorithm to run as a function of the input size. When doing Big-O analysis, "input" can mean a lot of different things depending on the problem being solved. In our examples above, the input is the array that is passed into the different functions. But, input could also be the number of elements of a linked list, the nodes in a tree, or whatever data structure you are dealing with. Since input is of size n, and in our example the input is an array - we will say that the array is of size n. We will use the 'n' to denote input size in our Big-O analysis. So, the real question is how Big-O analysis measures efficiency. Basically, Big-O will want to express how many times the 'n' input items are 'touched'. The word 'touched' can mean different things in different algorithms - in some algorithms it may mean the number of times a constant is multiplied by an input item, the number of times an input is added to a data structure, etc. But in our functions CompareSmallestNumber and CompareToAllNumbers, it just means the number of times an array value is compared to another value.

In the function CompareSmallestNumber, the n (we used 10 items, but lets just use the variable 'n'

for now) input items are each 'touched' only once when each one is compared to the minimum value. In Big O notation, this would be written as O(n) - which is also known as linear time. Linear time means that the time taken to run the algorithm increases in direct proportion to the number of input items. So, 80 items would take longer to run than 79 items or any quantity less than 79. You might also see that in the CompareSmallestNumber function, we initialize the curMin variable to the first value of the input array. And that does count as 1 'touch' of the input. So, you might think that our Big O notation should be O(n + 1). But actually, Big O is concerned with the running time as the number of inputs - which is 'n' in this case - approaches infinity. And as 'n' approaches infinity the constant '1' becomes very insignificant - so we actually drop the constant. Thus, we can say that the CompareSmallestNumber function has O(n) and not O(n + 1). Also, if we have n 3 + n, then as n approaches infinity it's clear that the "+ n" becomes very insignificant - so we will drop the "+ n", and instead of having O(n3 + n), we will have O(n3). Now, let's do the Big O analysis of the CompareToAllNumbers function. Let's just say that we want to find the worst case running time for this function and use that as the basis for the Big O notation. So, for this function, let's assume that the smallest integer is in the very last element of the array. Since we are taking each element in the array and comparing it to every other element in the array, that means we will be doing 100 comparisons, if we are assuming our input size is 10 (10 * 10 = 100). Or, if we use a variable that will n2 'touches' of the input size. Thus, this function uses a O(n2 ) algorithm.

Big O analysis measures efficiency

Now, let's compare the 2 functions: CompareToAllNumbers is O(n2) and CompareSmallestNumber is O(n). So, let's say that we have 10,000 input elements, then CompareSmallestNumber will 'touch' on the order of 10,000 elements, whereas CompareToAllNumbers will 'touch' 10,000 squared or 100,000,000 elements. That's a huge difference, and you can imagine how much faster CompareSmallestNumber must run when compared to CompareToAllNumbers - especially when given a very large number of inputs. Efficiency is something that can make a huge difference and it's important to be aware of how to create efficient solutions. In an interview, you may be asked what the Big-O of an algorithm that you've come up with is. And even if not directly asked, you should provide that information in order to show that you are well aware of the need to come up with an efficient solution whenever possible.

Whats the difference between DFS and BFS?

DFS (Depth First Search) and BFS (Breadth First Search) are search algorithms used for graphs and trees. When you have an ordered tree or graph, like a BST, its quite easy to search the data structure to find the node that you want. But, when given an unordered tree or graph, the BFS and DFS search algorithms can come in handy to find what youre looking for. The decision to choose one over the other should be based on the type of data that one is dealing with.

In a breadth first search, you start at the root node, and then scan each node in the first level starting from the leftmost node, moving towards the right. Then you continue scanning the second level (starting from the left) and the third level, and so on until youve scanned all the nodes, or until you find the actual node that you were searching for. In a BFS, when traversing one level, we need some way of knowing which nodes to traverse once we get to the next level. The way this is done is by storing the pointers to a levels child nodes while searching that level. The pointers are stored in FIFO (First-In-First-Out) queue. This, in turn, means that BFS uses a large amount of memory because we have to store the pointers.

Subscribe to our newsletter on the left to receive more free interview questions!

An example of BFS
Heres an example of what a BFS would look like. The numbers represent the order in which the nodes are accessed in a BFS:

In a depth first search, you start at the root, and follow one of the branches of the tree as far as possible until either the node you are looking for is found or you hit a leaf node ( a node with no children). If you hit a leaf node, then you continue the search at the nearest ancestor with unexplored children.

An example of DFS
Heres an example of what a DFS would look like. The numbers represent the order in which the nodes are accessed in a DFS:

Comparing BFS and DFS, the big advantage of DFS is that it has much lower memory requirements than BFS, because its not necessary to store all of the child pointers at each level. Depending on the data and what you are looking for, either DFS or BFS could be advantageous. For example, given a family tree if one were looking for someone on the tree whos still alive, then it would be safe to assume that person would be on the bottom of the tree. This means that a BFS would take a very long time to reach that last level. A DFS, however, would find the goal faster. But, if one were looking for a family member who died a very long time ago, then that person would be closer to the top of the tree. Then, a BFS would usually be faster than a DFS. So, the advantages of either vary depending on the data and what youre looking for.

What are the differences between a hash table and a binary search tree? Suppose that you are trying to figure out which of those data structures to use when designing the address book for a cell phone that has limited memory. Which data structure would you use?
A hash table can insert and retrieve elements in O(1) (for a big-O refresher read here). A binary search tree can insert and retrieve elements in O(log(n)), which is quite a bit slower than the hash table which can do it in O(1).

A hash table is an unordered data structure

When designing a cell phone, you want to keep as much data as possible available for data storage. A hash table is an unordered data structure which means that it does not keep its elements in any particular order. So, if you use a hash table for a cell phone address book, then you would need additional memory to sort the values because you would definitely need to display the values in

alphabetical order it is an address book after all. So, by using a hash table you have to set aside memory to sort elements that would have otherwise be used as storage space.

A binary search tree is a sorted data structure

Because a binary search tree is already sorted, there will be no need to waste memory or processing time sorting records in a cell phone. As we mentioned earlier, doing a lookup or an insert on a binary tree is slower than doing it with a hash table, but a cell phone address book will almost never have more than 5,000 entries. With such a small number of entries, a binary search trees O(log(n)) will definitely be fast enough. So, given all that information, a binary search tree is the data structure that you should use in this scenario, since it is a better choice than a hash table.

Suppose that you are given a linked list that is either circular or or not circular (another word for not circular is acyclic). Take a look at the figures below if you are not sure what a circular linked list looks like. Write a function that takes as an input a pointer to the head of a linked list and determines whether the list is circular or if the list has an ending node. If the linked list is circular then your function should return true, otherwise your function should return false if the linked list is not circular. You can not modify the linked list in any way.
This is an acyclic (non-circular) linked list:

This is a circular linked list:

You should start out this problem by taking a close look at the pictures of a circular linked list and a singly linked list so that you can understand the difference between the 2 types of lists.

The difference between the 2 types of linked lists is at the very last node of the list. In the circular linked list, you can see that the very last node (37) links right back to the first node in the list which makes it circular. However, in the acyclic or non-circular linked list you can see that the end node does not point to another node in the linked list and just ends. It is easy enough to know when you are in an acyclic linked list the pointer in the end node will just be pointing to NULL. However, knowing when you are in a circularly linked list is more difficult because there is no end node, so your function would wind up in an infinite loop if it just searches for an end node. So, there has to be a solution other than just looking for a node that points to NULL, since that clearly will not work. Take a closer look at the end node (37) in the circular linked list. Note that it points to the head node (12), and that there must also be a head pointer that points to the head node. This means that 12 is the only node that has 2 pointers pointing to it. And, suppose 37 pointed to 99, then this would still be a circularly linked list, and 99 would be the only node that has 2 pointers pointing to it. So, clearly, in a circular linked list there will always be a node that has 2 pointers pointing to it. Now the question is whether we can use that property to somehow provide a solution to this problem by traversing the list and checking every node to determine whether 2 nodes are pointing at it? If true, the list is circular. If not true, the list is acyclic and you will eventually run into a NULL pointer at the last element.

Too difficult a solution

This sounds like a good solution, but unfortunately it is difficult to find out how many pointers each element has pointing to it. So, before we even try to implement this solution, we should eliminate it since it will be very messy and difficult to implement. Another way of looking at a circular linked list is to say that if you are traversing the list and you have visited the same node twice, then you know that the linked list is circular. So, another possible algorithm is this: if we have already encountered a node, then we know the linked list is circular, but if we encounter a NULL pointer then we know that the linked list is acyclic. But, the big question here is how do we determine whether or not we have already visited a node in an algorithm? One way of doing this would be to mark each node after visiting it and then check to see if a node that you are visiting already has a mark. But, we can not modify the list in any way. Another option is to put all the nodes that we have already visited in a separate data structure, and then we would compare the current node to all of the nodes in that separate data structure. If we have a match, then we know that we are in a circular linked list. This would definitely work, but in the worst case-scenario (where the last node points to the head node) it would require as much memory as the linked list that you are given. Can we somehow minimize the memory requirement? Well, why bother replicating the linked list we are given, when we can just use the linked list itself to compare to. All we have to do is compare the current nodes pointer to the previous nodes directly. So, for the nth node, we just compare its next pointer to see if it points to any nodes from 1 to n 1. If any of those nodes are equal then we know that we have a circular linked list.

Lets figure out what the Big-O is of our algorithm is (for a refresher on Big-O Notation read this:Big O Notation Explained). If we are at the first node, then we examine 0 nodes; if we are at the 2nd node then 1 node is examined, if we are at the 3rd node then 2 nodes are examined. This means that the algorithm examines 0 + 1 + 2 + 3 + . + n nodes. The sum of 0 + 1 + 2 ++ n is equal to (n 2)/2 + n/2. And when calculating the Big-O of an algorithm we take off everything except the highest order term, so this means our algorithm is O(n2).

Try using 2 pointers

Can we do better than this solution? Well, yes there is a better solution although it is very difficult to have come up with unless you were given some sort of hint. Suppose we had 2 pointers instead of just one. Then what we could do is advance the pointers at different speeds. What happens if we advance 2 pointers at different speeds so that one pointer is traversing the list faster than the other pointer? So, we can have one slower pointer just going through every item in the linked list, and the other faster pointer traversing every other item in the linked list. In an acyclic (non-circular) list, the faster pointer will eventually hit a null pointer. But in a circular list, the faster pointer will eventually catch up to the slower pointer because the pointers are going in a circle. Take a few seconds and think about that to make sure it makes sense to you just imagine 2 pointers going in a circle, one faster and one slower. The faster one will eventually catch up to the slower one. So, this is another solution, and here is the pseudocode for this problem:

2 pointers travelling at different speeds start from the head of the linked list Iterate through a loop If the faster pointer reaches a NULL pointer then return that list is acyclic and not circular If the faster pointer is ever equal to the slower pointer or the faster pointer's next pointer is ever equal to the slower pointer then return that the list is circular Advance the slower pointer one node Advance the faster pointer by 2 nodes

If we write the actual code for this, it would look like this:

bool findCircular(Node *head) { Node *slower, * faster; slower = head; faster = head; while(true) {

// if the faster pointer encounters a NULL element if( !faster || !faster->next) return false; //if faster pointer ever equals slower or faster's next //pointer is ever equal to slow then it's a circular list else if (faster == slower || faster->next == slower) return true; else{ // advance the pointers slower = slower->next; faster = faster->next->next; } } }

And there is our solution. What is the Big-O of our solution? Well, whats the worst case if we know that the list is circular? In this case, the slower pointer will never go around any loop more than once so it will examine a maximum of n nodes. The faster pointer, however, will traverse through 2n nodes and examine half of those nodes (n nodes). The faster pointer will pass the slower pointer regardless of the size of the circle which makes it a worse case of n +n, which equals 2n nodes. This is O(n).

The Big O of the worst case when list is acyclic

And what about the worst case when the list is not circular acyclic? Then the faster pointer will have come to the end after traversing through n nodes and examining n/2 nodes since the faster pointer skips every other node. The slower pointer will have examined n/2 nodes, which means that combined the faster and slower pointer will have examined n/2 + n/2 nodes, which is also O(n). Thus, the algorithm is O(n) for both worst case scenarios.

Suppose that you are given a binary tree like the one shown in the figure below. Write some code in Java that will do a preorder traversal for any binary tree and print out the nodes as they are encountered. So, for the binary tree in the figure below, the algorithm will print the nodes in this order: 2, 7, 2, 6, 5, 11, 5, 9, 4

When trying to figure out what the algorithm for this problem should be, you should take a close look at the way the nodes are traversed a preorder traversal keeps traversing the left most part of the tree until a leaf node (which has no child nodes) is encountered, then it goes up the tree, goes to the right child node (if any), and then goes up the tree again, and then as far left as possible, and this keeps repeating until all of the nodes are traversed. So, it looks like each sub-tree within the larger tree is being traversed in the same pattern which should make you start thinking in terms of breaking this problem down into sub-trees. And anytime a problem is broken down into smaller problems that keep repeating, you should immediately start thinking in recursion to find the most efficient solution. So, lets take a look at the 2 largest sub-trees and see if we can come up with an appropriate algorithm. You can see in the figure above that the sub-trees of 7 and 5 (child nodes of the root at 2) are the 2 largest subtrees. Lets start by making observations and see if we can convert those observations into an actual algorithm. First off, you can see that all of the nodes in the subtree rooted at 7 (including 7 itself) are printed out before the subtree rooted at 5. So, we can say that for any given node, the subtree of its left child is printed out before the subtree of its right child. This sounds like a legitimate algorithm, so we can say that when doing a preorder traversal, for any node we would print the node itself, then follow the left subtree, and after that follow the right subtree. Lets write that out in steps:

1. Print out the root's value, regardless of whether you are at the actual root or just the subtree's root. 2. Go to the left child node, and then perform a pre-order traversal on that left child node's subtree. 3. Go to the right child node, and then perform a pre-order traversal on that right child node's subtree. 4. Do this recursively.

This sounds simple enough. Lets now start writing some actual code. But first, we must have a Node class that represents each individual node in the tree, and that Node class must also have some methods that would allow us to go to the left and right nodes. This is what it would look like in Java psuedocode:

public class Node { private Node right; private Node left; private int nodeValue; public Node ( ) { // a Java constructor } public Node leftNode() {return left;} public Node rightNode() {return right;} public int getNodeValue() {return nodeValue;} }

Given the Node class above, lets write a recursive method that will actually do the preorder traversal for us. In the code below, we also assume that we have a method called printNodeValue which will print out the Nodes value for us.

void preOrder (Node root) { if(root == null) return; root.printNodeValue(); preOrder( root.leftNode() ); preOrder( root.rightNode() ); }

Because every node is examined once, the running time of this algorithm is O(n).

Suppose that you are given a binary tree like the one shown in the figure below. Write some code in Java that will do an inorder traversal for any binary tree and print out the nodes as they are encountered. So, for the binary tree in the figure below, the algorithm will print the nodes in this order: 2, 7, 5, 6, 11, 2, 5, 4, 9 note that the very first 2 that is printed out is the left child of 7, and NOT the 2 in the root node

When trying to figure out what the algorithm for this problem should be, you should take a close look at the way the nodes are traversed. In an inorder traversal If a given node is the parent of some other node(s) then we traverse to the left child. If there is no left child, then go to the right child, and traverse the subtree of the right child until you encounter the leftmost node in that subtree. Then process that left child. And then you process the current parent node. And then, the traversal pattern is repeated. So, it looks like each sub-tree within the larger tree is being traversed in the same pattern which should make you start thinking in terms of breaking this problem down into sub-trees. And anytime a problem is broken down into smaller problems that keep repeating, you should immediately start thinking in recursion to find the most efficient solution. So, lets take a look at the 2 largest sub-trees and see if we can come up with an appropriate algorithm. You can see in the figure above that the sub-trees of 7 and 5 (child nodes of the root at 2) are the 2 largest subtrees.

1. Go the left subtree, and perform an inorder traversal on that node. 2. Print out the value of the current node.

3. Go to the right child node, and then perform an inorder traversal on that right child node's subtree.

This sounds simple enough. Lets now start writing some actual code. But first, we must have a Node class that represents each individual node in the tree, and that Node class must also have some methods that would allow us to go to the left and right nodes, and also a method that would allow us to print a Nodes value. This is what it would look like in Java psuedocode:

public class Node { private Node right; private Node left; private int nodeValue; public Node ( ) { // a Java constructor } public Node leftNode() {return left;} public Node rightNode() {return right;} public int getNodeValue() {return nodeValue;}

Given the Node class above, lets write a recursive method that will actually do the inorder traversal for us. In the code below, we also assume that we have a method called printNodeValue which will print out the Nodes value for us.

void inOrder (Node root) { if(root == null) return; inOrder( root.leftNode() ); root.printNodeValue(); inOrder( root.rightNode() ); }

Because every node is examined once, the running time of this algorithm is O(n).

Suppose that you are given a binary tree like the one shown in the figure below. Write some code in Java that will do a postorder traversal for any binary tree and print out the nodes as they are encountered. So, for the binary tree in the figure below, the algorithm will print the nodes in this order: 2, 5, 11, 6, 7, 4, 9, 5, 2 where the very last node visited is the root node

When trying to figure out what the algorithm for this problem should be, you should take a close look at the way the nodes are traversed there is a pattern in the way that the nodes are traversed. If you break the problem down into subtrees you can see that these are the operations that are being performed recursively at each node:

1. 2. 3.

Traverse the left subtree Traverse the right subtree Visit the root

This sounds simple enough. Lets now start writing some actual code. But first, we must have a Node class that represents each individual node in the tree, and that Node class must also have some methods that would allow us to go to the left and right nodes. This is what it would look like in Java:

public class Node { private Node right; private Node left; private int nodeValue; public Node ( ) { // a Java constructor } public Node leftNode() {return left;} public Node rightNode() {return right;} public int getNodeValue() {return nodeValue;} }

Given the Node class above, lets write a recursive method that will actually do the postorder traversal for us. In the code below, we also assume that we have a method called printNodeValue which will print out the Nodes value for us.

void postOrder (Node root) { if(root == null) return; postOrder( root.leftNode() ); postOrder( root.rightNode() ); root.printNodeValue(); }

Because every node is examined once, the running time of this algorithm is O(n).

Whats the difference between a stack and a heap?

The differences between the stack and the heap can be confusing for many people. So, we thought we would have a list of questions and answers about stacks and heaps that we thought would be very helpful.

Where are the stack and heap stored?

They are both stored in the computers RAM (Random Access Memory). For a refresher on RAM and virtual memory, read here: How Virtual Memory Works

How do threads interact with the stack and the heap? How do the stack and heap work in multithreading?
In a multi-threaded application, each thread will have its own stack. But, all the different threads will share the heap. Because the different threads share the heap in a multi-threaded application, this also means that there has to be some coordination between the threads so that they dont try to access and manipulate the same piece(s) of memory in the heap at the same time.

Can an object be stored on the stack instead of the heap?

Yes, an object can be stored on the stack. If you create an object inside a function without using the new operator then this will create and store the object on the stack, and not on the heap. Suppose we have a C++ class called Member, for which we want to create an object. We also have a function called somefunction( ). Here is what the code would look like:

Code to create an object on the stack:

void somefunction( ) { /* create an object "m" of class Member this will be put on the stack since the "new" keyword is not used, and we are creating the object inside a function */ Member m; } //the object "m" is destroyed once the function ends

So, the object m is destroyed once the function has run to completion or, in other words, when it goes out of scope. The memory being used for the object m on the stack will be removed once the function is done running. If we want to create an object on the heap inside a function, then this is what the code would look like:

Code to create an object on the heap:

void somefunction( ) { /* create an object "m" of class Member this will be put on the heap since the "new" keyword is used, and we are creating the object inside a function */ Member* m = new Member( ) ; /* the object "m" must be deleted otherwise a memory leak occurs */ delete m; }

In the code above, you can see that the m object is created inside a function using the new keyword. This means that m will be created on the heap. But, since m is created using the new keyword, that also means that we must delete the m object on our own as well otherwise we will end up with a memory leak.

How long does memory on the stack last versus memory on the heap
Once a function call runs to completion, any data on the stack created specifically for that function call will automatically be deleted. Any data on the heap will remain there until its manually deleted by the programmer.

Can the stack grow in size? Can the heap grow in size?
The stack is set to a fixed size, and can not grow past its fixed size (although some languages have extensions that do allow this). So, if there is not enough room on the stack to handle the memory being assigned to it, a stack overflow occurs. This often happens when a lot of nested functions are being called, or if there is an infinite recursive call. If the current size of the heap is too small to accommodate new memory, then more memory can be added to the heap by the operating system. This is one of the big differences between the heap and the stack.

How are the stack and heap implemented?

The implementation really depends on the language, compiler, and run-time thesmall details of the implementation for a stack and a heap will always be different depending on what language and compiler are being used. But, in the big picture, the stacks and heaps in one language are used to accomplish the same things as stacks and heaps in another language.

Which is faster the stack or the heap? And why?

The stack is much faster than the heap. This is because of the way that memory is allocated on the stack. Allocating memory on the stack is as simple as moving the stack pointer up.

How is memory deallocated on the stack and heap?

Data on the stack is automatically deallocated when variables go out of scope. However, in languages like C and C++, data stored on the heap has to be deletedmanually by the programmer using one of the built in keywords like free, delete, or delete[ ]. Other languages like Java and .NET use garbage collection to automatically delete memory from the heap, without the programmer having to do anything..

What can go wrong with the stack and the heap?

If the stack runs out of memory, then this is called a stack overflow and could cause the program to crash. The heap could have the problem of fragmentation, which occurs when the available memory on the heap is being stored as noncontiguous (or disconnected) blocks because used blocks of memory are in between the unusedmemory blocks. When excessive fragmentation occurs, allocating new memory may be impossible because of the fact that even though there is enough memory for the desired allocation, there may not be enough memory in one big block for the desired amount of memory.

Which one should I use the stack or the heap?

For people new to programming, its probably a good idea to use the stack since its easier. Because the stack is small, you would want to use it when you know exactly how much memory you will need for your data, or if you know the size of your data is very small. Its better to use the heap when you know that you will need a lot of memory for your data, or you just are not sure how much memory you will need (like with a dynamic array).

Provide an explanation of recursion, including an example.

Here we will try to explain recursion as simply as possible, without getting into the extremely theoretical and mathematical aspect of it. We want to provide a practical explanation that would be easy to understand. Defining recursion is easy any routine that calls itself is a recursive routine. But, using and understanding recursion is difficult. Recursion is best used for problems where a large task can be broken down into a repetitive subtask. Because a recursive routine calls itself to perform those sub-tasks, eventually the routine will come across a sub-task that it can handle without calling itself. This is known as a <bbase case and it is needed to prevent the routine from calling itself over and over again without stopping. So, one can say that the base case stops the recursion.</b

Base cases and Recursion

In the base case, the routine does not call itself. But, when a routine does have to call itself in order to complete its sub-task, then that is known as the <brecursive case. So, there are 2 types of cases when using a recursive algorithm: <bbase cases and recursive cases. This is very important to remember when using recursion, and when you are trying to solve a problem you should ask yourself: What is my base case and what is my recursive case?.</b</b

Example of recursion in Java the factorial

Lets start out with a simple example of recursion to best illustrate how it works: the factorial is probably the most commonly used example. What is a factorial? Well, any number written like this: x! is said to be x factorial. A factorial of a number is just the product of all integers between 1 and x. So, if x was the number 5, then the factorial would be 5*4*3*2*1, which equals 120. So, the factorial of any number x could also be defined as:

x! = x * (x - 1)!

and 0! = 1! = 1

Note how we defined the factorial of a number as that number multiplied by the factorial of the integer that is 1 less than the number (x * (x-1)! ). So, what we have done is essentially break the problem down into a sub-task, and in order to find the factorial of a number we just keep finding the factorials of the integers below that number and multiplying. So, the factorial of 3 is equal to 3 multiplied by the factorial of 2 and the factorial of 2 is equal to 2 multiplied by the factorial of 1. And that is what recursion is all about finding repetitive patterns, and breaking a problem down into repetitive subtasks. But, there is still one issue we seem to have found a recursive case, in which the routine will call itself, but what about the base case? Remember that we must have both a recursive case and a base case. The base case is what will stop the routine from calling itself infinitely, and will stop the recursion. Think about this what do you think would be a good base case for this problem? Well, it turns out that the base case would be when the factorial function hits a value of 1 because at that point we know the factorial of 1 is 1, so we should stop right there. And, it doesnt make sense to allow the function find the factorial of numbers less than 1, since the factorial is defined for integers between x and 1. So, heres what the Java code for our recursive factorial method would look like:

public int factorial (int x) { if (x > 1) { //recursive case: return factorial(x-1) * x; } else /*base case*/ return 1; }

Call Stacks, Recursion, and Stack Frames

A call stack is a data structure used by the program to store information about the active subroutines (like functions in C++ or methods in Java) in a program. The main reason for having a call stack is so that the program can keep track of where a subroutine should return control to once it finishes

executing. For example, suppose we have a method CreateBox which calls another method CreateLine in 4 different places. If the program has finished executing the method CreateLine, then it needs to know where in the CreateBox method it needs to return to. This is why the program uses a call stack so that it can keep track of these details.

A call stack is composed of stack frames

A stack frame is a part of the call stack, and a new stack frame is created every time a subroutine is called. So, in our recursive Factorial method above, a new stack frame is created every time the method is called. The stack frame is used to store all of the variables for one invocation of a routine.

Stack Frames in Recursion

A diagram of how the stack frames work in recursion will really help to clarify things so lets take a look at one. Lets suppose that we try to find the factorial of 3 using the function that we created above (so x is equal to 3), this is how it would look with the stack frames:

You can see that the first stack frame is created with x equal to 3. And then a call to Factorial(2) is made so the 1st call to Factorial does not run to completion because another call is made before the current call to Factorial can run to completion. A stack frame is used to hold the state of the first call to Factorial it will store the variables (and their values) of the current invocation of Factorial, and it will also store the return address of the method that called it. This way, it knows where to return to when it finishes running.

Finally, in the 3rd stack frame, we run into our base case, which means the recursive calls are finished and then control is returned to the 2nd stack frame, where Factorial(1) * 2 is calculated to be 2, and then control is returned to the very first stack frame. Finally, our result of 6 is returned.

Without a base case in recursion the stack overflows

What would happen if there were no base case in our example above? Well, recursive calls will be made continuously, and each time a recursive call is made a new stack frame is created. Every new stack frame created needs more memory, which then means that there is less memory on the call stack. The call stack has limited memory, which is usually determined at the start of the program and when that limited memory is exceeded then the stack is said to overflow, which will usually results in the program crashing. So, if we did not have a base case, then the stack would overflow.

Can every recursive function be converted into an iterative function?

Yes, any problem that can be solved recursively can also be solved through the use of iteration. In case you dont know, iteration is the use of a looping construct like a while loop, for loop, etc in order to solve a problem, whereas recursion is the use of a function that calls itself to solve a problem. The reason that iteration can be used to solve any problem that recursion solves is because recursion uses the call stack behind the scenes, but a call stack can also be implemented explicitly (in code) and iteratively (through a for loop, while loop, or any kind of loop). So, whatever recursion gives us behind the scenes can be also be done by implementing a call stack directly in the code itself. This is true even for problems that seem to be fundamentally recursive. The factorial function which we discussed here can also be implemented iteratively.

What are some of the differences between using recursion to solve a problem versus using iteration? Which one is faster? Which one uses more memory?
The fact is that recursion is rarely the most efficient approach to solving a problem, and iteration is almost always more efficient. This is because there is usually more overhead associated with making recursive calls due to the fact that the call stack is so heavily used during recursion (for a refresher on this, read here). This means that many computer programming languages will spend more time maintaining the call stack then they will actually performing the necessary calculations.

Does recursion use more memory than iteration?

Generally speaking, yes it does. This is because of the extensive use of the call stack.

Should I use recursion or iteration?

Recursion is generally used because of the fact that it is simpler to implement, and it is usually more elegant than iterative solutions. Remember that anything thats done in recursion can also be done iteratively, but with recursion there is generally a performance drawback. But, depending on the problem that you are trying to solve, that performance drawback can be very insignificant in which case it makes sense to use recursion. With recursion, you also get the added benefit that other programmers can more easily understand your code which is always a good thing to have.

Write a method in Java that will print out all the possible combinations (or permutations) of the characters in a string. So, if the method is given the string dog as input, then it will print out the strings god, gdo, odg, ogd, dgo, and dog since these are all of the possible permutations of the string dog. Even if a string has characters repeated, each character should be treated as a distinct character so if the string xxx is input then your method will just print out xxx 6 times.
Finding an algorithm to answer this question may seem challenging because finding all the different permutations of a string is something that you just do naturally without really thinking about it. But, try to figure out what algorithm you are implicitly using in your mind whenever you write down all the different permutations of a string. Lets use the word dogs as an example and see what different permutations we get. Here is what comes up when we list out all the possible permutations of the letters in dogs:

dogs dosg dsgo dsog dgso dgos

sgod sgdo sdgo sdog sodg sogd

gsod gsdo gdso gdos gods gosd

ogsd ogds osdg osgd odgs odsg

So, when we came up with the permutations of dogs above, how did we do it? What were we

implicitly thinking? Well, it looks like what we did in each column was choose one letter to start with, and then we found all the possible combinations for the string beginning with that letter. And once we picked a letter in the 2nd position, we then found all the possible combinations that begin with that 2 letter sequence before we changed any of the letters in the first 2 positions. So, basically what we did was choose a letter and then peformed the permutation process starting at the next position to the right before we come back and change the character on the left.

Finding the permutations with recursion

Our description of the process that we followed sounds a lot like something that could be solved with recursion. How can we change our description so that its easier to write out in a recursive method? Lets phrase it like this: In order to find all possible combinations for a given string, then start at a position x, then find and place all possible letters in position x. Every time we put a new letter in position x, we should then find all the possible combinations at position x+1 this would be the recursive call that we make. How do we know when to print out a string? Well, when we are at a position x that is greater than the number of letters in the input string, then we know that we have found one valid combination/permutation of the string and then we can print it out and return to changing letters at positions less than x. This would be our base case remember that we always must have a recursive case and a base case when using recursion.

Which letters can we use in a given position?

Another big part of this problem is figuring out which letters we can put in a given position. Using our sample string dogs, lets say that we are going through all the permutations where the first 2 letters are gs. Then, it should be clear that the letters in the 3rd or 4th position can only be either d or o, because g and s were already used. As part of our algorithm, we have to know which letters can be used in a given position because we cant use the letters that were used in the earlier positions. And in order to do this we can simply have an array of Boolean values that correspond to the positions of the letters in the input string so if a certain character from the input string has already been used, then its position in the array would be set to true. Now, here is what the Java method would like for our algorithm:

void permute( String input) { int inputLength = input.length(); boolean[ ] used = new boolean[ inputLength ]; StringBuffer outputString = new StringBuffer(); char[ ] in = input.toCharArray( ); doPermute ( in, outputString, used, inputLength, 0 ); } void doPermute ( char[ ] in, StringBuffer outputString,

boolean[ ] used, int inputlength, int level) { if( level == inputLength) { System.out.println ( outputString.toString()); return; } for( int i = 0; i < inputLength; ++i ) { if( used[i] ) continue; outputString.append( in[i] ); used[i] = true; doPermute( in, outputString, used, length, level + 1 ); used[i] = false; outputString.setLength( outputString.length() - 1 ); } }

Provide the Java code that would be used find the factorial of a number using iteration and not recursion in other words use a loop to find the factorial of a number.
Earlier we had discussed how to find the factorial of a number using recursion. Now, if we want to find the factorial of a number using iteration instead of recursion, how would we do that? Its actually pretty simple, and its something you should try to figure out on your own. A factorial of a number x is defined as the product of x and all positive integers below x. To calculate the factorial in a for loop, it seems like all we would have to do is start from x and then multiply by all integer values below x, and just hold that value until we are done iterating. And that is exactly what needs to be done. Here is what the code will look like in Java:

Java code for Iterative Factorial

int factorial ( int input ) { int x, fact = 1; for ( x = input; x > 1; x--)

fact *= x; return fact; }