Introduction to High Performance Computing / Parallel Computing

SYED HASIBUR RAHMAN

Who am I?
Syed Hasibur Rahman (Ananno)
M.Sc. 2008 Department of Physics
Thesis : Setup a prototype PC Cluster for High Performance Computing

And now?
Sr . Software Engineer Research and Development Brac IT Services Ltd.

http://www.ananno-bd.com # shr.ananno@outlook.com # +8801713334553

What is HPC / Parallel Computing ?

Hardware:

Good quality computer usually Servers interconnected via High Speed Network are working together on one particular job in parallel delivering very high calculations per unit time
Software:

A computer programming that allows a particular task to be split into parts in such a way that each parts can be assigned to one of the several different processing units available in a distributed system i.e. HPC facility

What is HPC / Parallel Computing ?

Powerful
Can

compute much more then PC

Much

bigger Memory

E.g. : 1024x1024x1024 x 4 bits (Integer) = 4294967296 bits > 4 GBytes E.g. : 1024x1024x1024 x 64 bits (Double) = 68719476736 bits > 68 GBytes

Very

Big in Size
of Sq. Ft. of floor space

Thousands

Power

Hungry
Hundreds of Kilowatts / Several Megawatts of Electricity

Requires

Each PC ~ 400Watt Few Gigaflops Few GB of RAM x 1000 = Teraflop scale ~ Megawatt power Server ~ 600Watt 7~12 Gigaflops 8~128 TB RAM x 100 = Teraflop scale ~ 30 KW

Types of HPC
Supercomputers
One

very large machine thousands of CPUs, Petabytes of Memory

Clusters
Hundreds/Thousands

of relatively small machine grouped together with high speed interconnectivity Hundreds/Thousands of Node Each node has several CPUs and each CPU has multiple processing Core, few GB of RAM for each Core or Clusters geographically distributed are grouped together to sharing resources several clusters / supercomputers Infrastructure As A Service Virtual Clusters consist of several machines

Grid
Supercomputers

Cloud
IAAS

Computation Need in Scientific Research Problem Type Calculations Large

Very complex equation solving, Handling polynomials, Higher order of precision

Large number of occurrence of relatively small calculations

Generating large quantities of Random Numbers, System involving Large number of particles, Genome decoding, Nano-magnetism

Parametric calculations Analyzing very large data Analyzing Huge number of relatively small data

Same equation but different value of input variables

LHC

Statistical analysis, Stock market analysis

Computation Need in Scientific Research - Fields

Genome Code Pharmaceuticals

Oil and Gas Mapping Atmospheric Mapping

Physics and Math

Particle Interaction Nano Physics

SETI
Stock Market Analysis Cyber forensic

Astrophysics
Bio Informatics Aerodynamics

Nuclear & Particle Experiment Simulation

Energy and Defense Lab

Satellite Data

And many more are joining .

Why simulation is preferred before actual experiments No high-tech expensive lab setup is needed
High-tech

HPC facility might be required and computation time depends of number of independent

Higher degrees of freedom

Complexity

variables

Much convenient for parametric research

Some

programming skills might be needed of them are pretty expensive programming skills is needed

Mostly open source software

Some

Redesign and Rerun an experiment virtually costless

Some

Better visualization

Prospect of HPC in a Developing country - Bangladesh

Supercomputers are very much expensive and strictly task oriented machine

Clusters can be built using even Desktop PCs which may not be run continuously for very long time
No special hardware Commodity Hardware Can be scaled very easily Relatively less power hungry Can be contribute to existing Grid / Cloud Technology Can be connected via fiber-optics within the country using govt. infrastructures to establish grid network Not much sophisticated support systems i.e. cooling, power etc. are required

Not much special and expensive manpower is needed

Pre-requisites to work on HPC

Basic requirement of all researchers

Ability

to ask question WHAT, WHY , WHEN, WHERE and most importantly HOW open source and very powerful and most importantly - Free sense

Linux of course
Being Common

Very basic Programming skills

C

should be and practically enough sense

Common

Patience of Bug-Fixing
Parallel

code can be little bit messy

Again Common sense

Concept of Parallel Programming

SISD

Single Instruction Single Data Serial Processing

SIMD

Single Instruction Multiple Data Instruction Parallel

Parametric

MISD

Multiple Instruction Single Data Data Parallel Calculations Multiple Instruction Multiple Data

Recursive MIMD

Both

Example of Parallel Computing Formula 1 Pit Stop

Tasks

Refueling Oil Change The four tire changes

Eight tire carriers

The front and rear jack men

Cooling Refreshing Driver Electronic System Check

LESS THEN 10 SECOND

Example of Parallel Computing Formula 1 Pit Stop

Instruction Parallel
Same

Instruction

Multiple

Data (Tires)

LESS THEN 10 SECOND

Example of Parallel Computing Formula 1 Pit Stop

Data Parallel
Same

Data

Multiple
Each

Instruction (Crews)

Crew has his won job (Instructions)

LESS THEN 10 SECOND

Future of HPC - GPGPU

General Purpose Graphics Processing Unit

Massively

Multicore

CUDA core - 3072

Base Clock (MHz) 915~1019 Memory Speed (Gbps) - 6 4096 MB (2048 MB per GPU) GDDR5 4 GB ~ 5 Teraflop performance

Embarrassingly

parallel / SIMD

Last Supercomputer without GPU ~ Hundreds of Teraflops

Hundreds

of thousands of Node / Core, ~50 MW

Todays
6/2012)
16.32

Fastest Supercomputer with GPU - IBM Sequoia (Top500

PFLOPS, 1.6 PB, 3,000 square feet, 7.9 MW

Thank You

QA