Distributed Tuple-space Server for

Ubiquitous Video Conferencing

Faculty Advisors Student Assistants:

Dr. Charles Liu JohnPaul Adigwu

Adrienne Lam
Dr. Helen Boussalis Sergio Beltran
Aleksander Milshteyn
Jorge Estrada
Alexander Alegre

7/01/2010 NASA Grant URC NCC NNX08BA44A 1

 Overview
Hardware Layout
•Distributed server network
•Multi-platform compatibility

Application Layout
Synchronizing data across
servers

Organizing and relating nodes of information

based on their meaning

Connecting users across different

platforms & network access capabilities

7/01/2010 NASA Grant URC NCC NNX08BA44A 2

 Progress

7/01/2010 NASA Grant URC NCC NNX08BA44A 3

 Progress

7/01/2010 NASA Grant URC NCC NNX08BA44A 4

 Semantic Network

Active
Active Semantic
Semantic Scraping
Scraping Ubiquitous
Ubiquitous
Video
Video
Technology Conferencing
Conferencing
Technology
INTERNE
T
Directory Network INTERNE
T

Directory Network
5/27/2010 NASA Grant URC NCC NNX08BA44A 5
 Overview
Server 1 Server 3
DB3
DB1 Tuple Tuple
Space Space
Server 2
Tuple
Applications Space
Applications
DB2

Applications
Client-Server Communications
 Considerations
• Organized storage of information
• Ease of access
• Handle large amounts of users
• Heavy information traffic
• Client hardware limitations
 Prior Work on Semantic
Network
• Developed a basic model for storing information across a
network
• Storing of information locally
• Add, delete and edit of objects within the tree structure.
• Embedded browser and Webpage parsing
 How do we actually organize
the information on the device?

Can we just store information on a device

with a unique id and put into a directory?

Maybe, lets consider the scenarios….

Embedded
On a Laptop
A Desktop Systems
? Likely.
? No ?
Not without overwhelming the device!
Problem.
 Solution: A Database
• A database consists of an organized collection of data
for one or more multiple uses.

• Indexing allows a set of table rows matching some

criterion to be quickly located. (keywords, location,
trigger algorithms)

• Primary key functions ensure information is uniquely

identified in the system, but also related to other pieces
of information through reference of primary-keyed
parent and child tables.

7/01/2010 NASA Grant URC NCC NNX08BA44A 11

 Database Criteria
Max Max Max Max
Max table Max row Max CHAR Min DATE Max DATE
  Max DB size columns Blob/Clob NUMBER column
size size size value value
per row size size name size

524,258 TB
Microsoft (32,767 files * 16 524,258 TB Unlimited 30000 2 GB 2 GB 6 126 bits 2 0001 9999 128
SQL Serv TB max file size)
er

MyISAM
storage limits: 4 GB
MySQL 5 Unlimited 256TB; Innodb 64 KB  4096  4
(longtext, 64 KB (text) 64 bits 1000 9999 64
storage limits: longblob)
64TB

4 GB block
Unlimited (4 GB
size (with
Oracle block size per
BIGFILE
8KB 1000 Unlimited 4000 B 126 bits -4712 9999 30
tablespace)
tablespace)
 MySQL
• MySQL “My S-Q-L” is a open source
relational database management
system

•Developed in 1995 by Michael Widenius

• Currently used by:

 Why MySQL?

• Cost - The MySQL Community Server operating costs are

significantly less then other commercial database products, most
notably Oracle.

• MySQL database server has the capacity to run massive data

warehouses holding terabytes of information

• Reliability - Features include distributed transaction

capability(where readers never block writers and vice-versa)

• Full data integrity is also assured through server-enforced referential

integrity, specialized transaction isolation levels, and instant
deadlock detection
 How do we deal with
concurrency issues ?
• Concurrency refers to the sharing of resources by multiple interactive users or application programs at the same time.

– Lost updates. Two users logged in to the network, A and B, might both read the same row* (or file) from the database and both calculate new values. If A updates the row with its new value and B then also updates the row, the update performed by A is lost.
– Access to uncommitted data. System of user A might update a value in the database, and system of user B might read that value before it was committed. Then, if the value of A is not later committed, but backed out, the calculations performed by B are based on uncommitted (and presumably invalid) data.
– Non-repeatable reads. Application A reads a row from the database, then goes on to process other SQL requests. In the meantime, application B either modifies or deletes the row and commits the change. Later, if application A attempts to read the original row again, it receives the modified row or discovers that the original row has
been deleted.
– Phantom Read Phenomenon.
 Concurrency Solution

• DB table file “stack”

Unique File Stack Id
Unique File Id
Owner Id#*
Timestamp
Server path
Security level
Image id # 0099
0101010
Image id # 0099 1010101
0101010
1010101
0101010 0101
1010101
0101010
1010101
Stack_File_Id
0101
#0002
Stack_File_Id
#0001
 Concurrency Solution (cont.)

Sys_Content Table Sys_db_content_stack Table

Image id # 0099
0101010
Image id # 0099 1010101
0101010
1010101
0101010 0101
1010101
0101010
1010101
Stack_File_Id
0101
#0002
Stack_File_Id
#0001
Database Working Model
 Qt SQL Module
• A compilation of code which provides operability
between client application and the MySQL database

• Multi-layer library core consists of Drivers, API layers,

and a User Interface Layer.

Client Side MYSQL DB

Application QT SQL MODULE
ENGINE
 Embedded Device
Considerations

Sem_DB_1 Information
DB via server sends
Client Request
Rendered
xml code…to Client

7/01/2010 NASA Grant URC NCC NNX08BA44A 22

 MySQL Login Code

// Select Portion of Login function for semantic network. Nothing unusual.

try
{
Connection conn(false);
conn.connect("DB NAME", "DB _HOST ", "DB USER", "DB_PASS");
Query query = conn.query();
}
catch (const Exception& er)
{
// Catch-all for any other MySQL++ exceptions
cerr << "Error: " << er.what() << endl; return -1;
} 
return (EXIT_SUCCESS);
}
 Client-Server: Current Work

• Working on piecing the SQL framework extension to link

with Qt source code

• Testing and Refining Database information model

• Formulating code to leverage the table content stack to

deal with concurrency issues in multi-user environment
 Ubiquitous Video Conferencing

Active Semantic Scraping Ubiquitous

Video
Conferencing
Technology

INTERNE
T

Directory Network
7/01/2010 NASA Grant URC NCC NNX08BA44A 26
 Purpose of Ubiquitous
Video Conferencing

• Interaction that goes beyond desktop environment in

everyday life
• Visual and Audio collaboration in real time between
two or more locations

7/01/2010 NASA Grant URC NCC NNX08BA44A 27

 Bandwidth and Memory
Control
- User Adjustments:
• Resolution
• Video Window Resizing
• Close video portion. Keep audio/text intact
• Video Encoding Selection (MPEG-2)

7/01/2010 NASA Grant URC NCC NNX08BA44A 29

 Video Encoding Selection
- Motion JPEG - MPEG-2 (H.262)
• Maximum compression rate – 1:20 • Support for interlaced video
• Low computational complexity • MP3 audio support

7/01/2010 NASA Grant URC NCC NNX08BA44A 30

 Motion JPEG in depth

• Series of “still” pictures

• At low bandwidth, priority is given to image resolution
• Minimum latency in image processing
• Images have a constant file size
• Still the most widespread picture compression format used today
• Captures at maximum rate of 30 frames per second
 MPEG-2 Algorithm

• Color Conversion
• Motion Estimation
• Discrete Cosine Transform
• Quantization
• Run-length Encoding
• Variable-length Coding
• IDCTQ/Correlation for Reference Frame

7/01/2010 NASA Grant URC NCC NNX08BA44A 33

 MPEG-2 Algorithm

7/01/2010 NASA Grant URC NCC NNX08BA44A 34

 Color Space Conversion

7/01/2010 NASA Grant URC NCC NNX08BA44A 35

 Color Space Conversion

• Involves conversion of images that are represented in one color

space to another color space

• RGB and YUV(YCrCb)

7/01/2010 NASA Grant URC NCC NNX08BA44A 36

 RGB

• Original video files or camera input are RGB coded

• Uses additive color mixing
• Stores individual values for red, green and blue
• Three color channels are digitized with a certain precision (bit depth)
• Pixel = (x0, y0) = (R, G, B, A) = 32 bits

7/01/2010 NASA Grant URC NCC NNX08BA44A 37

 YUV (Y'CbCr)

• Transformation of the RGB information

• Y’ is the Luma component
• CB and CR are the blue-difference and red-difference chroma
components
• Colors are approximately calculated from RGB values

7/01/2010 NASA Grant URC NCC NNX08BA44A 38

 Purpose of Colorspace
Conversion
• Some of the information can be discarded in order to
reduce bandwidth by reducing the size of the UV or
CrCb channels
• JPEG, MPEG, DIVX, WMV utilize this compression
method
• Chroma Subsampling of uncompressed YUV(Y’CrCb)
files have 50% size of uncompressed RGB files.

7/01/2010 NASA Grant URC NCC NNX08BA44A 39

 Chroma Subsampling

• Separates the luma (Y’) signal and the chroma signal

(CrCb)
• Encoding of images by elimination of redundant chroma
(CrCb) information to achieve more efficient data
compression

7/01/2010 NASA Grant URC NCC NNX08BA44A 40

 RGB Y’CrCb (via YUV)

• Weighted values of R,G, B added to produce Y signal

(brightness or luminance)
• U(Cr) signal created by subtracting the Y’ from original
Blue signal (B)
• V(Cb) signal created by subtracting the Y’ from original
Red signal (R)

Source: Intel® Image Processing Library

7/01/2010 NASA Grant URC NCC NNX08BA44A 41

 Mathematical Model
RGB  Y’CrCb (Encoder)

7/01/2010 NASA Grant URC NCC NNX08BA44A 42

 Mathematical Model
RGB  Y’CrCb (Decoder)

7/01/2010 NASA Grant URC NCC NNX08BA44A 43

 RGB Decomposition

Original RGB image of Crab Nebula

Intensity of R Intensity of G Intensity of B

7/01/2010 NASA Grant URC NCC NNX08BA44A 44
 YCrCb Decomposition

YCrCb image of Crab Nebula

Intensity of Y Intensity of Cr Intensity of Cb

7/01/2010 NASA Grant URC NCC NNX08BA44A 45
 MPEG-2 Motion Estimation

7/01/2010 NASA Grant URC NCC NNX08BA44A 46

 MPEG-2 Structure

I-frames  least compressible, don't require other frames to decode

P-frames  use data from previous I-frames to decompress
B-frames  use both previous and forward frames for data reference
to get the highest amount of data compression
 MPEG-2 Motion Estimation

Applying MPEG-2 video compression results in lesser amounts of

data transmitted over the network, than the Motion JPEG
 Main Challenge

• Compilation of MPEG Libraries within Qt

framework
• Compatibility of MPEG frame format with OpenCV
(Open Computer Vision software)

Compatible

Compiler Conversion

7/01/2010 NASA Grant URC NCC NNX08BA44A 49

 Current and Future Work
- UVC:
• MPEG-4 Streams (MPEG-2) In Progress
• Chat Translator
• Embedded Video Player

- Scraping Tool:
• Scrape Web Pages Content In Progress
• Drag and Drop of File Objects
• Network Search by Objectized Tags

7/01/2010 NASA Grant URC NCC NNX08BA44A 50

 2009 – 2010 Timeline

7/01/2010 NASA Grant URC NCC NNX08BA44A 51

 2010-2011 Timeline

7/01/2010 NASA Grant URC NCC NNX08BA44A 52