Professional Documents
Culture Documents
TABLE OF CONTENTS
NEED LITERATURE SURVEY HDFS HADOOP MAP REDUCE FORMAT CONVERSION
NEED
THE MANY TO MANY MAPING:
NEED:
File playable in one device cannot be played in other
device. It has to be converted to other formats to be played. Hence we go for format conversion.
LITERATURE SURVEY
Existing tools:
Any to any convertor Format Factory Total video Convertor
LITERATURE SURVEY
Conversion tools basic requirement is to provide
Quality in conversion Higher rate of conversion
The existing tools provides the first requirement but not the second.
LITERATURE SURVEY
To convert larger files at a faster rate,we go for cluster
environment.
Existing System in cluster Environment:
MAV GRID
LITERATURE SURVEY
This mav grid focuses on resource sharing but not on
10
HDFS
Master/slave architecture HDFS cluster consists of a single Namenode, a master
server that manages the file system namespace and regulates access to files by clients. There are a number of DataNodes usually one per node in a cluster.
12
HDFS Architecture
13 Namenode Client Block ops Read Datanodes
replication
Datanodes
B Blocks
Rack1
Write Client
Rack2
13
HDFS-DATA NODE
The DataNodes manage storage attached to the nodes that they run on. A file is split into one or more blocks and set of blocks are stored in DataNodes. DataNodes serves read, write requests, performs block creation, deletion, and replication upon instruction from Namenode.
14
HDFS-NAME NODE
Namenode maintains the file system Any meta information changes to the file system
recorded by the Namenode. An application can specify the number of replicas of the file needed: replication factor of the file. This information is stored in the Namenode.we take this as 1.
15
HDFS
The Namenode receives a Heartbeat and a BlockReport from each DataNode in the cluster. BlockReport contains all the blocks on a Datanode. Receipt of a Heartbeat implies that the DataNode is functioning properly.
16
18
19
20
21
22
23
25
MAPCONVE RT
3GP FILE
REDUCE COMBIN E
26
REFERENCES
[1] Hadoop 0.20 Documentation [2] http://hadoop.apache.org/mapreduce/ [3]IEEE paper on A Design of Grid Supported Services for
Mobile Learning System by M. Norazizi Sham Mohd Sayuti,Universiti Sains Islam Malaysia (USIM)
27