A Thesis for the Bachelor of Computer Systems

Engineering
Development of a Vision System for a Humanoid
Robot

Andrew Blower 33539973

19 October 2001
 89 Trevallyan Drive,
Daisy Hill, Qld, 4127,
19 October 2001.
Prof. Simon Kaplan
Head of School Information Technology
and Electrical Engineers
University of Queensland
St. Lucia, Old, 4072.

Dear Professor Kaplan,

In accordance with the requirements of the degree of Bachelor of Engineering in the divi-
sion of Computer Systems Engineering, I present the following thesis entitled “Development
of a Vision System for a Humanoid Robot”. This work has been performed under the super-
vision of Dr Gordon Wyeth.

I declare that the work submitted in this thesis has not been previously submitted for a degree
at the university of Queensland or any other institution. To the best of my knowledge and
belief, this thesis contains no material previously published or written by any other person,
except where reference is made in the text.

Yours sincerely,

Andrew Blower.
 Acknowledgements
The following people I would like to thank:

• Gordon Wyeth: my supervisor, for driving the entire project and providing advice on
how to get the job done.

• Mark Chang: for his help with the old vision system and assistance with the specifica-
tion and current design.

• David Prasser: for his help with image processing and vision software.

• Mark Wagstaff and Damien Kee: for being one of the ones to keep me sane during this
thesis .

• Rest of the Robotics team: for providing a great team environment in which to develop
this project.

• To my parents for proof reading the many drafts churned out as a part of this thesis.

A.R. Blower Development of a Vision System for a Humanoid Robot ii

 Abstract

This thesis investigates the design of a vision system for the "Guroo" a robot humanoid soccer
player. The aim was to create a vision system that was able to view and track objects in the
three dimensional world. Vision hardware is presented for the CMOS Digital Camera and
Interface software for a field programmable gate array included on the main vision board.
The design and implementation of an aesthetically pleasing head is also presented.
Beginning in 2000 the project has moved on from the development of the design of the
original vision system for the "Viperoo" robot soccer team to designing a new system for the
use in the Guroo project and the 2001 upgrade for the Viperoos small soccer league.
The major achievements in the redesign of the vision system are the use of the OV7620
CMOS digital camera chip which has trebled the frame rate of the digital camera used and
the implementation of a dual port ram system for the field programmable gate array. These
combine with the developments in the vision board to produce an improved local vision
system.
The result of this development was a camera that passed data from the three dimensional
world surrounding it to the main vision board.
Future work on the design of a vision system will involve optimising the vision code and
low level hardware code. Future students could also port areas of the object detection code
to the field programmable gate array as another area of vision development.
Contents

1 Introduction 1
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 RoboCup Soccer Tournament . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 The Guroo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Importance of Vision to a Humanoid . . . . . . . . . . . . . . . . . . . . . 3
1.5 Expected Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Outline Of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Literature review 7
2.1 CMOS Camera Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Omnidirectional Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3 Stereo Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Vision System 2000 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.5 Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.5.1 Current Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.5.2 Megapixel Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.6 Vision Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Problem Specification 13
3.1 Importance of specifications . . . . . . . . . . . . . . . . . . . . . . . . . 13

i
CONTENTS CONTENTS

3.2 Development of Specifications . . . . . . . . . . . . . . . . . . . . . . . . 13

3.3 Digital Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3.1 Old Camera Design . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3.2 Human Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3.3 Outline of Digital Camera Specifications . . . . . . . . . . . . . . 15
3.4 FPGA Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4.1 Old Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.4.2 Outline of Field Programmable Gate Array Interface . . . . . . . . 16
3.5 Specification Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Digital Camera Design 18

4.1 CCD Vs CMOS Digital Cameras . . . . . . . . . . . . . . . . . . . . . . . 18
4.1.1 CCD or CMOS Camera? . . . . . . . . . . . . . . . . . . . . . . . 20
4.2 Previous Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
4.3 Selection of the CMOS Camera . . . . . . . . . . . . . . . . . . . . . . . 21
4.4 Progressive and Interlaced Scanning Method . . . . . . . . . . . . . . . . . 21
4.5 YUV . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.6 Clocking System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.6.1 Crystal Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.6.2 Programmable Oscillator . . . . . . . . . . . . . . . . . . . . . . . 23
4.6.3 FPGA Connection . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.7 Digital Camera Windowing . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.8 Camera Lens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.9 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5 Interface Design 27
5.1 Interface Decision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
5.2 Interface Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

A.R. Blower Development of a Vision System for a Humanoid Robot ii

CONTENTS CONTENTS

5.3 Test Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5.4 Dual Port Ram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
5.4.1 Problems with Memory Access . . . . . . . . . . . . . . . . . . . 29
5.5 Camera Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.6 Dual Port Ram for the Hitachi SH4 . . . . . . . . . . . . . . . . . . . . . 32
5.7 DMA Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
5.8 Chapter Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

6 Aesthetic Head Design 34

6.1 Sources of Inspiration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
6.2 Head Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.3 Head Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
6.4 Final Head Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.5 Design for Manufacture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.6 Camera Housing and Bracket . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.7 Remainder of the Humanoid . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.8 Summary of Chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

7 Results 41
7.1 Product Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
7.1.1 Field Programmable Gate Array Interface . . . . . . . . . . . . . . 41
7.1.1.1 Testing Procedure . . . . . . . . . . . . . . . . . . . . . 42
7.1.2 CMOS Digital Camera . . . . . . . . . . . . . . . . . . . . . . . . 42

8 Future Work 45
8.1 CMOS Digital Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
8.2 FPGA Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
8.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

A.R. Blower Development of a Vision System for a Humanoid Robot iii

CONTENTS CONTENTS

A Camera Schematic 52

B Camera Printed Circuit Board 54

C Dual Port Ram 56

D Test Code Counter 62

E Camera Housing 74

F Camera Bracket 76

G Head Original 79

H Head Final 81

I Head Assembly 84

A.R. Blower Development of a Vision System for a Humanoid Robot iv

List of Figures

1.1 Sony and Honda Humanoids . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Vision’s Place within the Humanoid . . . . . . . . . . . . . . . . . . . . . 4

2.1 CMOS Camera Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Omnidirectioal Camera and lens . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Urbie by NASA and JPL . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Viperoos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.5 PB-159 CMOS Camera . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.6 Photobit Megapixel CMOS Camera Chip . . . . . . . . . . . . . . . . . . 11
2.7 SH3 Vision Board . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.1 Human Eye . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.1 Digital Camera Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.2 Interlaced and Progressive Scan . . . . . . . . . . . . . . . . . . . . . . . 22
4.3 RGB and YUV Colour Space . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.4 Camera Clocking Systems . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.1 Development Board and Spartan 2 FPGA . . . . . . . . . . . . . . . . . . 28

5.2 Dual Port Ram VHDL Architecture . . . . . . . . . . . . . . . . . . . . . 30
5.3 Camera Interface Declaration . . . . . . . . . . . . . . . . . . . . . . . . . 31

v
LIST OF FIGURES LIST OF FIGURES

6.1 C3PO and R2D2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

6.2 Initial Head Design Amish . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6.3 Final Head Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.4 Manufacturable Head . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
6.5 Camera Bracket . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
6.6 The Guroo Mechanical Design [22] . . . . . . . . . . . . . . . . . . . . . 39

7.1 FPGA Dual Port Ram Simulation . . . . . . . . . . . . . . . . . . . . . . . 42

7.2 Hitachi Dual Port Ram . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

A.R. Blower Development of a Vision System for a Humanoid Robot vi

List of Tables

4.1 Feature of CCD Vs CMOS Cameras[12] . . . . . . . . . . . . . . . . . . . 19

4.2 Performance of CCD Vs CMOS Cameras[12] . . . . . . . . . . . . . . . . 19

vii
Chapter 1

Introduction

“Nothing can stop us, we’re on a mission from Gordo”.David Prasser May 2001.

1.1 Introduction
This thesis investigates vision hardware for use in object detection with application to a
humanoid robot. Development of a digital camera and software for a field programmable gate
array is discussed. The objective of this thesis is to design a vision system for a humanoid
robot so as to allow the robot to view a soccer ball in order to be involved in the RoboCup
soccer tournament. This is only the beginning, vision systems of this design will allow
humanoid robots to interact in increasingly complex situations with humans and the outside
world.

1.2 RoboCup Soccer Tournament

The RoboCup soccer tournament is an international event held annually in different coun-
tries each year showcasing the best efforts of the world’s research and educational institu-
tions. The mission statement of RoboCup is “by 2050 develop a team of fully autonomous

1
CHAPTER 1. INTRODUCTION 1.3. THE GUROO

Figure 1.1: Sony and Honda Humanoids

humanoid robots that can win against a human world champion soccer team”. To that end
there are several different leagues, the small soccer league which comprises teams of 5 robots
using either a global vision system or a local vision system. Global vision is the position-
ing of an image sensor above the field of play, this sensor has a global view of the playing
surface. Local vision is a system where every robot has its own individual vision system.
The simulation league is a network version which is purely a software problem. The latest
league and at this time exhibition only is the humanoid league, this consists of humanoid
biped robots playing at this stage of development as individual against individual.
Two humanoid designs that have come to prominence of late are the SDR-3X and Asimo,
designed by Sony and Honda respectively. These are the most advanced humanoid bipeds
operating at present. These designs display the full range of movement for a humanoid biped
and are able to successfully locate and kick a soccer ball.

1.3 The Guroo

The Guroo or the “Grossly Under-funded Roo” is the creation of the University of Queens-
land robotics team and has been designed to compete in the world Robocup soccer tourna-
ment. The name Grossly Under-Funded comes from the fact that the team’s initial budget of
six thousand dollars and final budget of around sixteen thousand dollars was in the order of
three orders of magnitude less than that of either of the other main humanoid teams of Sony

A.R. Blower Development of a Vision System for a Humanoid Robot 2

CHAPTER 1. INTRODUCTION 1.4. IMPORTANCE OF VISION TO A HUMANOID

or Honda. The term “Roo” is a part of the tradition of Australian soccer teams originating
with the Australian soccer team called the Socceroos, this tradition has continued with all
UQ robotics soccer teams.
To meet the specifications of the tournament, the design must be a humanoid biped capa-
ble of playing soccer. The design must be under 1.2 meters tall and is based on the approx-
imate size of a seven year old child. The design of the humanoid has been divided amongst
thirteen people with the main electronic components being the vision system, the commu-
nications hardware, the IPAQ controller, motor controllers and the power supply. These
components are explained in other thesises written by other students of Dr Gordon Wyeth.
The diagram below displays the visions systems place in the overall design of the Guroo.

1.4 Importance of Vision to a Humanoid

Vision to the humanoid is essential, it is the predominant sensor that will allow the robot
to interact with the real world, without its sensor the robot would only be able to react to
commands entered by the user. With the addition of a vision system the robot can be made
to act autonomously, it can actively react to its surroundings making decisions based on the
objects it encounters. Although a humanoid biped robot requires many sensors to allow
accurate actuation of its limbs, without a vision sensor the system can never truly interact
with its surrounds.
As stated above the aim of this thesis is to design a vision system to allow the robot to
participate in soccer games against other robots. Work for this thesis has been developed
from a vision system designed by Mark Chang and Ben Brown[24] for use in the Viperoos
small league robots. These machines have their own local vision systems that allow them
all to individually view the world and make decisions based on this information. This thesis
will extend upon the current design with the implementation of a new digital camera sensor
and interface software for a field programmable gate array operating on the main vision

A.R. Blower Development of a Vision System for a Humanoid Robot 3

CHAPTER 1. INTRODUCTION 1.4. IMPORTANCE OF VISION TO A HUMANOID

Figure 1.2: Vision’s Place within the Humanoid

A.R. Blower Development of a Vision System for a Humanoid Robot 4

CHAPTER 1. INTRODUCTION 1.5. EXPECTED OUTCOMES

processing board.

1.5 Expected Outcomes

The expected outcome from this research is the development of an updated vision system
for both the humanoid biped and the Viperoos small league soccer robots. This will enable
them to have the best vision possible for playing soccer at the next Robocup tournament. The
design will need to be compatible with the main vision processing board designed by Mark
Chang [24] and be able to be used by the Viperoo robots and the biped Humanoid robot. The
main outcomes to be developed are

• An increased frame rate.

• Greater accuracy in vision measurements.

• Simplification of the digital camera hardware.

• An elegant portable solution between the Viperoos and the humanoid.

1.6 Outline Of Thesis

This thesis has been broken down into six chapters The first two, outlining an initial intro-
duction as to why the thesis is being written and what the current technologies are. Chapter
three deals with a review of the problem while chapters four and five outline the design pro-
cess used to solve this problem. Chapter six discusses the final performance of the system
and what future developments could be made to improve the design.

Chapter 1 - Introduction This chapter will outline the aim of this thesis, the robot for
which the vision system is being implemented, how the vision system relates to the humanoid
robot and why the vision system is so important to the soccer playing robot.

A.R. Blower Development of a Vision System for a Humanoid Robot 5

CHAPTER 1. INTRODUCTION 1.6. OUTLINE OF THESIS

Chapter 2 - Literature Review Recent developments in the design of vision systems

are discussed and examples explained as to how the basic principles of a vision system op-
erate.

Chapter 3 - Outline of Objectives This chapter sets out the tasks and design goals to
be achieved in the thesis toward the development of the humanoid vision system.

Chapter 4 - CMOS Digital Camera Design The design implementation of the hu-
manoid’s eye is outlined, the problems encountered and the discriminators used in the deci-
sion as to which digital camera chip was the most suitable.

Chapter 5 - Field Programmable Gate Array Design The interfacing used to pass
the data from the CMOS Camera to the Vision Processing Board through the use of a field
programmable gate array is explained. The design is outlined and the code used to program
the FPGA is explained.

Chapter 6 - Design of an Aesthetically Pleasing Head The design and consideration

of an aesthetically pleasing head and the design process involved not only in the head design
but the mechanical design required to allow the camera to operate effectively.

Chapter 7 - Future Work Work yet to be completed and what could be done to im-
prove the implementation of the humanoids vision system in the future will be discussed as
well as what should be areas of design should be focussed on in the next redesign.

Chapter 8 - Results and Conclusion The final chapter discusses what was achieved
throughout the thesis project, what outcomes were successfully accomplished and what fur-
ther developments can be made to the humanoid vision system.

A.R. Blower Development of a Vision System for a Humanoid Robot 6

Chapter 2

Literature review

“The danger from computers is not that they will eventually get as smart as men,
but that we will meanwhile agree to meet them halfway”. Bernard Avishai

This chapter will explain the current work developed in this field and in particular the design
from which the inspiration for the new design came. The original vision system will be
discussed as well as the new innovation in vision system technology.

2.1 CMOS Camera Operation

CMOS or complementary metal oxide semiconductor digital cameras operate similar to CCD
or charge coupled device cameras in that both CMOS and CCD cameras capture the image
data in exactly the same method however from that point on the two are very different.
CCD cameras produce an analogue output that requires extra hardware to allow the images
produced to be analysed. CMOS cameras use on board analogue to digital converters to
convert the data into a digital signal.

7
CHAPTER 2. LITERATURE REVIEW 2.2. OMNIDIRECTIONAL VISION

Figure 2.1: CMOS Camera Operation

2.2 Omnidirectional Vision

Another form of vision being developed by teams involved in the robotic soccer competi-
tion is omnidirectional vision where a parabolic or conical mirror is placed above the CCD
or CMOS camera to produce a 360°image of the surrounding world. Teams at the 2001
Robocup tournament used this technology in the small size league. The main drawback with
this technology is the amount of processing power required to process the amount of data
produced from the digital camera. This is because by using an omnidirectional camera a
high amount of resolution is required and the image can’t be subsampled as this would result
in a loss of data that reduces the visions effectiveness.

2.3 Stereo Vision

Stereo vision is another concept being used by some organisations around the world. NASA
is using this concept on its “Urbie” project being conducted in conjunction with JPL’s Ma-
chine Vision Group to develop an urban tactical vehicle capable of moving over urban ter-
rain. This vehicle has incorporated stereo vision to give it depth perception and a greater
ability to handle the terrain surrounding it. In the future stereo scopic vision may be a vi-

A.R. Blower Development of a Vision System for a Humanoid Robot 8

CHAPTER 2. LITERATURE REVIEW 2.4. VISION SYSTEM 2000

Figure 2.2: Omnidirectioal Camera and lens

Figure 2.3: Urbie by NASA and JPL

able adaptation for humanoid robots to develop however it is not a viable concept with the
amount of processing power required for the size of the robot whereas the “Urbie” being a
vehicle can contain and carry the processing power required. The “Urbie” is also fitted with
omnidirectional vision as described above amongst its other sensors.

2.4 Vision System 2000

The design for the new vision board is a development from the Viperoos design from the 2000
RoboCup tournament in Melbourne. The design used by the Viperoos will be explained in

A.R. Blower Development of a Vision System for a Humanoid Robot 9

CHAPTER 2. LITERATURE REVIEW 2.5. CAMERA

Figure 2.4: Viperoos

further detail in the following sections. This vision system worked effectively in the 2000
RoboCup Soccer Tournament with the team placing third in their group

2.5 Camera
All vision systems require a sensor to collect the data about the world surrounding the system.
In the case of a vision system, this sensor is a camera or image sensor. There are many
options for the choice of the camera these include CCD and CMOS digital cameras.

2.5.1 Current Camera

The camera used in the 2000 vision system design uses a PB-159 Photobit CMOS digital
camera. This camera puts out a RGB colour spectrum and operates at around 12 frames per
second. The camera passes the RGB colour data to the SH3 vision board through the use of
an 8 bit serial connection and the camera is controlled through a I2C command connection.
The camera receives the command line data from the SH3 main vision board and then returns
the RBG data from the camera.

A.R. Blower Development of a Vision System for a Humanoid Robot 10

CHAPTER 2. LITERATURE REVIEW 2.6. VISION BOARD

Figure 2.5: PB-159 CMOS Camera

Figure 2.6: Photobit Megapixel CMOS Camera Chip

2.5.2 Megapixel Cameras

The latest innovation in CMOS camera technology is the megapixel camera. These cameras
have greater than one megapixel resolution and frame rates of 500 frames per second and
greater. These sensors will be the future of vision systems however today’s technology is not
able to accommodate the amount of information that these CMOS cameras provide, future
designs need to consider these sensors when future redesigns are being completed.

2.6 Vision Board

The Vision Board used by the Viperoos uses the Super H architecture from Hitachi as its
main processor. This chip is a microcontroller not a digital signals processing chip and as
such has a few different requirements. The SH7709 is a RISC processor operating at 80MHz

A.R. Blower Development of a Vision System for a Humanoid Robot 11

CHAPTER 2. LITERATURE REVIEW 2.6. VISION BOARD

Figure 2.7: SH3 Vision Board

and 104 MIPS. This board takes the data from the digital camera and determines the location
of the ball and other obstacles in relation to the robot and is an essential part of the hardware
for the local vision robots.

A.R. Blower Development of a Vision System for a Humanoid Robot 12

Chapter 3

Problem Specification

Lunkwill: “O great computer, the task we have designed you to perform is this:
We want you to tell us the Answer”.The Hitchhiker’s Guide to the Galaxy.

The specification of a design is important to the successful implementation of the final de-
sign. This chapter sets out to specify what the design should be able to do to allow the Guroo
to play soccer. It is from these specifications that the development of the whole project can
begin to take shape.

3.1 Importance of specifications

The importance of specification to any project is to allow the design and the client or team to
work cohesively and include all relevant ideas involved in the completion of the design.

3.2 Development of Specifications

The specifications for this project were developed whilst the project was constantly develop-
ing to produce the final set of specifications. This was due to the fact that this was a team

13
CHAPTER 3. PROBLEM SPECIFICATION 3.3. DIGITAL CAMERA

project and people were at various levels of knowledge in vision design and some parts of
the project had to be completed before others could be finalised.

3.3 Digital Camera

The digital camera design is intrinsic to the complete operation of the vision system. This
“eye” to the real world for the computer design is essential if it is ever to be given real world
tasks. Since the “Guroo” is being designed to play soccer, a local vision system is required
if the humanoid is to be autonomous.
The camera system operates similar to that of a human eye. Both systems have a lens to
focus the image onto a light sensitive area being the retina in a human and the CMOS digital
camera photo plate in the robots vision system.

3.3.1 Old Camera Design

The current camera design implements the PB-159 CMOS digital camera by Photobit, this
camera operates in the red, green and blue colour spectrum and operates at 12 frames per
second at a resolution of 480 x 320. This CMOS camera requires the designer to set the gain
levels for the system to operate correctly requiring extra hardware.

3.3.2 Human Vision

The ultimate goal of the robot soccer competition is the construction of a humanoid robot.
In order to achieve this, the vision system should be as close to that of a human beings. The
first step in that goal is to give the robot’s ’eye’ a frame rate similar to that of a real person.

A.R. Blower Development of a Vision System for a Humanoid Robot 14

CHAPTER 3. PROBLEM SPECIFICATION 3.3. DIGITAL CAMERA

Figure 3.1: Human Eye

3.3.3 Outline of Digital Camera Specifications

The CMOS camera has been designed to the following specifications to produce the best
possible vision system for the GUROO humanoid robot. These specifications have been
developed from past experiences using vision systems.
The specifications for the digital camera are as follows:

• Higher frame rate.

• vision equivalent to that of human beings (frame rate of around 30 fps).

• The ability to do subsampling on board the chip. This allows some of the processing
that would normally be done on the vision board to be done on the camera chip prior
to vision processing.

• The ability to do automatic gain calibration. Some of the new CMOS camera chips do
this on chip meaning that extra hardware is not required.

• That the output colour spectrum of the camera be YUV instead of RGB to allow the
vision processing to be simpler as YUV is invariant to the light intensity of the sur-
rounding conditions.

A.R. Blower Development of a Vision System for a Humanoid Robot 15

CHAPTER 3. PROBLEM SPECIFICATION 3.4. FPGA INTERFACE

• Simplification of the Vision Hardware.

3.4 FPGA Interface

The new vision board design by Mark Chang[24] utilises a Hitachi Super H4 architecture
microcontroller to do the image recognition required by the Guroo to play soccer and do
additional tasks. To connect the digital camera to the vision board a suitable interface is
required to do some of the image processing and set the various clocking signals required to
run the digital camera.

3.4.1 Old Interface Design

The previous design implemented the Hitachi Super H3 microcontroller as the vision pro-
cessor and received its vision data from the Photobit PB300 CMOS digital camera. The SH3
board required no subsampling interface to operate the camera at 32 x 128 resolution at 12
frames per second. That system operated in the RGB colour space and required gain control
hardware. This system is limited to the amount of data that can be analysed by the SH3
board.

3.4.2 Outline of Field Programmable Gate Array Interface

The field programmable gate array is required to buffer the input signal from the digital
camera to the vision board through the use of a dual port ram implementation. It should also
make use of the delay locked loops in the field programmable gate arrays to generate any
clock rate and signals required by the CMOS digital camera. The basic specifications are
displayed below:

• Implement a dual port ram system for a field programmable gate array. This is to
control the flow of data to the field programmable gate array.

A.R. Blower Development of a Vision System for a Humanoid Robot 16

CHAPTER 3. PROBLEM SPECIFICATION 3.5. SPECIFICATION SUMMARY

• Use the delay locked loops to implement any clocking signals required.

3.5 Specification Summary

The specifications above have been developed to produce part of the vision system. The
other components of this system are the vision board designed by Mark Chang[24] and the
development of object recognition software by David Prasser[20].

A.R. Blower Development of a Vision System for a Humanoid Robot 17

Chapter 4

Digital Camera Design

The design and development of the digital camera is the basis of the “Guroo’s” vision system.
It is the robots ’eye’ and as “eyes are the windows to the souls”(traditional proverb) for
people they draw the attention of any person interacting with the machine as well as being
the main sensor for the robots interaction with the outside world.

4.1 CCD Vs CMOS Digital Cameras

These were the two different types of cameras considered when it came to choosing the dig-
ital camera chip to implement in the vision system design. CCD cameras or charge coupled
device cameras and CMOS complementary metal oxide semiconductor cameras have some
similarities. Both types of camera operate by converting light into electric charge and process
it as an electronic signal, however the main difference between the two camera types is that
the CCD cameras have an analogue output whereas the output of CMOS cameras are digital
in bits. The main features and performance indicators for both CCD and CMOS cameras are
displayed in table 4.1.

18
CHAPTER 4. DIGITAL CAMERA DESIGN 4.1. CCD VS CMOS DIGITAL CAMERAS

Feature CCD CMOS

Signal out of Pixel Electron packet Voltage
Signal out of chip Voltage (analog) Bits (digital)
Signal out of camera Bits (digital) Bits (digital)
Fill factor High Moderate
Amplifier mismatch N/A Moderate
System Noise Low Moderate to High
System Complexity High Low
Sensor Complexity Low High
Camera components PCB + multiple chips + lens Chip + Lens
Relative R&D cost Depends on Application Depends on Application
Relative system cost Depends on Application Depends on Application
Table 4.1: Feature of CCD Vs CMOS Cameras[12]

Performance CCD CMOS

Responsivity Moderate Slightly better
Dynamic range High Moderate
Uniformity High Low to Moderate
Uniform Shuttering Fast, common Poor
Speed Moderate to High Higher
Windowing Limited Extensive
Antiblooming High to none High
Biasing and clocking Multiple, higher voltage Single, low voltage
Table 4.2: Performance of CCD Vs CMOS Cameras[12]

A.R. Blower Development of a Vision System for a Humanoid Robot 19

CHAPTER 4. DIGITAL CAMERA DESIGN 4.1. CCD VS CMOS DIGITAL CAMERAS

Figure 4.1: Digital Camera Operation

4.1.1 CCD or CMOS Camera?

The decision whether to use a CMOS or CCD is of great importance to the overall design
as there are significantly different hardware requirements, output data types and other con-
siderations described above. The final decision was to use a CMOS digital camera as the
remainder of the vision system is digital and the requirement for extra hardware combined
with the lower frame rate and limited windowing led to the decision that a CMOS digital
camera would be optimal for this application.

A.R. Blower Development of a Vision System for a Humanoid Robot 20

CHAPTER 4. DIGITAL CAMERA DESIGN 4.2. PREVIOUS DESIGN

4.2 Previous Design

The previous design implemented by Mark Chang[24] using the photobit PB-159 chip the
specifications and basic design parameter have been explained in the specifications section.
The camera is a CMOS digital camera that utilises the RGB colour space and requires colour
gain circuitry. The schematic for this design can be seen below.

4.3 Selection of the CMOS Camera

Once the decision that the CMOS digital camera was the type of camera chip to be used,
research began into which chip would be the most suitable for a humanoid vision system.
Various camera chips were considered, these are described in the table below. The megapixel
cameras were considered however the amount of data produced from these cameras would
overwhelm the vision board and be counter productive. The chip that was finally decided
upon was the OV7620 chip by OmniVision. This chip has a resolution similar to that of the
PB-300. The OmniVision chip has many advantages that made it very attractive, these being
the optional eight or sixteen bit data output, the options for output type in RGB or YUV.
These combined with the ability to subsample the image on chip made this chip ideal for
vision system applications.

4.4 Progressive and Interlaced Scanning Method

There are the two methods of operation for the OV7620 digital camera progressive scan and
interlaced scan methods to capture images. These methods are used for different applica-
tions, progressive scan reads each line of pixels and sends the data to the output one line at a
time while interlaced scan reads every second line and sends that to the output. This results
in all the odd pixel lines sent in the first sweep and all the even lines sent in the second sweep.

A.R. Blower Development of a Vision System for a Humanoid Robot 21

CHAPTER 4. DIGITAL CAMERA DESIGN 4.5. YUV

Figure 4.2: Interlaced and Progressive Scan

Figure 4.3: RGB and YUV Colour Space

The digital camera system operates using the progressive scan method as it is faster than in-
terlaced scanning and as the image is to be subsampled or “windowed” the progressive scan
method will be better with the smaller image area.

4.5 YUV
The YUV colour spectrum as compared to the RGB (red, blue and green) spectrum involves
the use of brightness blue and red values. This can be seen in the two pictures below. The
picture on the right is in the RGB spectrum while the one on the left has been converted
into the YUV colour space using red blue and light intensity. It can be seen that the areas of
the greatest light intensity being the sky and the deck of the boat have become pink or red
while other areas have become green when mixed with blue. Further information on object
detection using the YUV colour space can be found in Prasser[20].

A.R. Blower Development of a Vision System for a Humanoid Robot 22

CHAPTER 4. DIGITAL CAMERA DESIGN 4.6. CLOCKING SYSTEM

4.6 Clocking System

The clocking system is fundamental to the successful operation of the CMOS digital camera
as it controls the reading of the pixels and data to the outputs. As this circuit is import it
was decided to place redundancies in the circuit, these include three different ways to clock
the digital camera. The use of a standard 24 MHz crystal oscillator, a programmable crystal
oscillation and the ability to use a clock signal generated by a field programmable gate array
were placed in the circuit to ensure the success of this important hardware. The decision to
use each of the different clocking mechanisms can be decided through the use of the jumpers
to connect each clocking method to the camera chip. The clocking schematic can be seen
below.

4.6.1 Crystal Circuit

The crystal circuit used implemented a 24 MHz HC19 crystal with two 22 pF capacitors.
When this circuit was built it was found to contain too much inductance in the wires due
to their length and the distance of the crystal from the camera chip. Subsequently the pro-
grammable oscillator had to be used for the clocking circuit on the digital camera.

4.6.2 Programmable Oscillator

The programmable oscillator used in this design was the EXO-3C programmable oscillator
an integrated circuit that produces a 24 MHz clock output that can be reduced by factors of
two by setting bits at the input to the chip. This was the clocking system used by the camera
and proved to be successful.

A.R. Blower Development of a Vision System for a Humanoid Robot 23

CHAPTER 4. DIGITAL CAMERA DESIGN 4.7. DIGITAL CAMERA WINDOWING

4.6.3 FPGA Connection

The field programmable gate array being used by the vision board contains a delay locked
loop capable of producing various clock signals, if needed this connection was placed in
the circuit as an alternate means of applying a clock signal to the camera chip but was not
required.

4.7 Digital Camera Windowing

For the image processing to operate, the digital camera needs to subsample the image taken
as the amount of data to be processed by the digital camera at full resolution is far in excess of
what the Hitachi SH4 vision board can process if thirty images per second is to be achieved.
The OV7620 has facilities on board the camera to enable this to be done prior to passing
the image data to the vision board. The OV7620 has two modes of operation to reduce the
resolution of the image to accommodate a higher frame rate. The first is the enabling of the
QVGA mode which reduces the resolution from 640 x 480 to 320 x 240 pixels however even
with this lower frame rate the amount of data to be transferred to the vision board is still
in excess of what can be handled. The other method for windowing or subsampling is to
control the Href and Vsync horizontal and vertical inputs to the camera in order to limit the
resolution to 240 x 180 which the vision coded has been designed to process.

4.8 Camera Lens

The camera employed in this design is only as good as the lens that is placed in front of it.
For this project the lens chosen was the same as the one used last year by the Viperoos[26]
, a CCTV lens by AVENIR 2.8mm F 1.4. This lens was chosen as it has an 80 degree angle
of vision which is better that many other lenses on the market and the university has these
lenses, reducing the cost of the vision upgrade.

A.R. Blower Development of a Vision System for a Humanoid Robot 24

CHAPTER 4. DIGITAL CAMERA DESIGN 4.8. CAMERA LENS

Figure 4.4: Camera Clocking Systems

A.R. Blower Development of a Vision System for a Humanoid Robot 25

CHAPTER 4. DIGITAL CAMERA DESIGN 4.9. CHAPTER SUMMARY

4.9 Chapter Summary

This chapter has discussed the design of the CMOS digital camera being implemented by the
humanoid robot vision system. The hardware for this system operates using the OmniVision
OV7620 CMOS digital camera. As has been discussed in this chapter, this chip is capable
of YUV data output, image subsampling or windowing and interlaced or progressive image
scanning. These combined with the clocking systems operating on the camera board make
this a highly successful device and essential to the complete operation of the vision system.

A.R. Blower Development of a Vision System for a Humanoid Robot 26

Chapter 5

Interface Design

“A computer terminal is not some clunky old television with a typewriter in front
of it. It is an interface where the mind and body can connect with the universe
and move bits of it about”. Douglas Adams(Mostly Harmless)

The implementation of a successful interface between the digital camera and the Hitachi
SH4 vision board is essential to the operation of the humanoid vision system. To that end
the interface chosen for this design is the Spartan 2 field programmable gate array.

5.1 Interface Decision

The interface to be used in the Guroo and Viperoos was the decision of Mark Chang[24]
the designer of the vision processing board. The specification for the design of the interface
has been outlined in chapter three and won’t be discussed here, however the design called
for a system that controlled all the data signals required by the digital camera and the flow
of data from the camera. A suitable way to implement this design was through the use of
a field programmable gate array enabling the use of a softcore with a hardware design, the
combination of which, would enable future development in the optimisation of the vision

27
CHAPTER 5. INTERFACE DESIGN 5.2. INTERFACE DESIGN

Figure 5.1: Development Board and Spartan 2 FPGA

hardware.

5.2 Interface Design

The interface design for the field programmable gate array (FPGA) has been written in
VHDL (Very High Speed Integrated Circuit Hardware Description Language) using the Xil-
inx foundation suite to development and simulate the design of the Interface for the Vision
system. The chip chosen for the interface was the Spartan2 xc2s100 field programmable
gate array by Xilinx, this FPGA of the spartan2 family has 100 000 gates. Development of
the design was done on the spartan2 development kit from insight electronics. Using this
development board and the Xilinx foundation software suite software to develop the softcore
that is the advantage of an FPGA. This provided a software solution to a hardware problem.

A.R. Blower Development of a Vision System for a Humanoid Robot 28

CHAPTER 5. INTERFACE DESIGN 5.3. TEST SOFTWARE

5.3 Test Software

With the development board came some test software to enable the user to test if the board
was operating correctly, this code became the software used to learn and operate the devel-
opment board prior to the design and implementation of the vision software interface. The
sample coded operated a counter operating between 00 and 99 on two seven segment displays
on board the development board.

5.4 Dual Port Ram

Dual port ram is a system of buffering data through memory where the data can be prepared
for image analysis. The Spartan 2 FPGA has sets of bock ram surrounding the chip totalling
40K bites of RAM as well as what can be designed in the combinational logic blocks(CLB’s).
Dual port ram operates through the use of two sets of data lines, the first is the read address
and the other the write address. The reading is controlled by a chip select control(CS) bit
and the writing is controlled by a write enables(WE) bit. The two other busses involved in
dual port ram are obviously the data in and data out buses. The data is written into the on
board ram of the FPGA and combinational logic blocks, the address is dependent on the input
address bus, as long as the WE is high the data from the indata bus will be written into the
memory array. The read operation is very similar to the write, the read address bus provides
the address and as long as the CS is low the data will be produced at the outdata bus.

5.4.1 Problems with Memory Access

The main problem that can arise with a dual port ram implementation on a field programmable
gate array is that the same memory address can be read as well as written. If memory is ac-
cessed by both read and write at the same time, timing problems may occur and memory
overwrites may cause the data to become corrupted and produce erroneous reading when

A.R. Blower Development of a Vision System for a Humanoid Robot 29

CHAPTER 5. INTERFACE DESIGN 5.4. DUAL PORT RAM

architecture data of dual_port_ram is

–memory allocation of 256 memory locations
type mem_type is array (127 downto 0) of
STD_LOGIC_VECTOR (7 downto 0);
signal mem : mem_type;
begin
process(clk, we, waddr)
begin
if (rising_edge(clk)) then
if (we = ’1’) then
– write data into memory
mem(conv_integer(waddr)) <= indata;
end if;
end if;
end process;
process(raddr, clk)
begin
if(rising_edge(clk))then
if(CS = ’0’)then
– read data from memory
outdata <= mem(conv_integer(raddr));
end if;
end if;
end process;
end data;

Figure 5.2: Dual Port Ram VHDL Architecture

A.R. Blower Development of a Vision System for a Humanoid Robot 30

CHAPTER 5. INTERFACE DESIGN 5.5. CAMERA INTERFACE

entity camera_ram is

port ( clk : in STD_LOGIC;

we : in STD_LOGIC; – write enable
– single address bus
address : in STD_LOGIC_VECTOR(6 downto 0);
– data write
dataw: in STD_LOGIC_VECTOR(15 downto 0);
– data read
datar: out STD_LOGIC_VECTOR(15 downto 0);
PCLK: in STD_LOGIC; – signal inputs
HREF: in STD_LOGIC;
VSYNC: in STD_LOGIC;
SI0: in STD_LOGIC;
SI1: in STD_LOGIC;
XCLK: in STD_LOGIC;
opclk: out STD_LOGIC; – signal outputs
ohref: out STD_LOGIC;
ovsync: out STD_LOGIC;
oxclk: out STD_LOGIC;
osi0: out STD_LOGIC;
osi1: out STD_LOGIC);
end camera_ram;

Figure 5.3: Camera Interface Declaration

analysed.

5.5 Camera Interface

The CMOS digital camera requires certain signals to allow the camera to operate effectively,
generally these signals have been routed through the FPGA apart from the data signals which
have been implemented using the dual port ram. These signals have been passed straight
through the FPGA in VHDL for the Hitachi SH4.

A.R. Blower Development of a Vision System for a Humanoid Robot 31

CHAPTER 5. INTERFACE DESIGN 5.6. DUAL PORT RAM FOR THE HITACHI SH4

5.6 Dual Port Ram for the Hitachi SH4

The dual port ram implementation for the Hitachi SH4 is different from that of the conven-
tional dual port ram implementations in that the data buses are not of a single direction, the
data is written in and read from the same bus into and out of the field programmable gate
array. The data written in from the camera is a sixteen bit bus. The data is again controlled
by the write enabled bit if the bit is high then the data is written however if write enabled is
low the data is read from the field programmable gate array to the Hitachi SH4 for analysis.
Extra data associated with the operation of the FPGA is also included in the design, these
include the control line interface SI0 and SI1 the Pixel clock Pclk the horizontal and vertical
reference signals to determine the size of the window to be sampled. These signal are routed
straight through the FPGA providing straight through connection for the signals. This allow
at a later stage the implementation of a command interface involving these signals. The sig-
nificant differences between the original dual port ram and the final version are the different
data, address and control buses, these are modified to conform to the hardware.

5.7 DMA Code

Vision data transfered in the RGB colour space is transmitted back through the FPGA through
the use of direct memory access, information on this part of the design can be gained from
Hosking[21].

5.8 Chapter Summary

This chapter has outlined the design and implementation of the interface for the Hitachi SH4
vision board and the CMOS digital camera being used in this vision system. The design of a
dual port ram system and how it operates has been explained, these all combine to produce a

A.R. Blower Development of a Vision System for a Humanoid Robot 32

CHAPTER 5. INTERFACE DESIGN 5.8. CHAPTER SUMMARY

vision system for playing soccer in both the “Guroo” and ’Viperoo’.

A.R. Blower Development of a Vision System for a Humanoid Robot 33

Chapter 6

Aesthetic Head Design

“A good scientist is a person with original ideas. A good engineer is a person

who makes a design that works with as few original ideas as possible. There are
no prima donnas in engineering”. Freeman Dyson, English physicist, educator
"Disturbing the Universe," pt. 1, ch. 10, 1979.

The development of an aesthetically pleasing head is fundamental to the successful design

of not only the mechanical design of the humanoid but is the focus point for all humans
viewing the ’Guroo’ in action, this particular part impacts on opinion that people will have
of the design as a whole.

6.1 Sources of Inspiration

Sources of inspiration for the development of the head came from many sources predomi-
nantly science fiction films such as Starwars and the many manga films. The design calls
for a cyclops and there are very few humanoid robots in any film or cartoon or film that are
cyclopses. The main inspiration for the head and the overall design of the humanoid must be
Starwars C3PO[4].

34
CHAPTER 6. AESTHETIC HEAD DESIGN 6.2. HEAD REQUIREMENTS

Figure 6.1: C3PO and R2D2

6.2 Head Requirements

The head has many requirements to enable the humanoid robot to operate effectively at
soccer and be aesthetically pleasing. This design must also be simple to construct and be
able to hold whatever vision hardware is to be placed inside the head. The heads mass must
also be less than 0.5 kg due to the inertia on the servos which control the movement of the
head.

6.3 Head Design

The initial design of the head called for a swept back look similar to many of the charac-
ters in manga films and cartoons such as “dragon ball z” This design can be seen below.
The inherent problems with this design are finding a head suitable for the Guroo and the
manufacturing aspects of building this design.

A.R. Blower Development of a Vision System for a Humanoid Robot 35

CHAPTER 6. AESTHETIC HEAD DESIGN 6.4. FINAL HEAD DESIGN

Figure 6.2: Initial Head Design Amish

6.4 Final Head Design

After many revisions and many different shapes and ideas some resembling an amish nun
according to fellow thesis students, a head design was decided upon. Based upon the initial
design, the final design that was decided upon was one that was both of a science fiction
background and soccer, one of the fundamental reasons for the robot. The soccer ball type
head not only gives the design a humanoid type shape but provides the camera hardware
with a good amount of protection from any impacts the robot might suffer from other robots
or hitting the ground. This design gives the humanoid a basic human appearance while
providing a very functional shape for the vision hardware to be housed in. The whole design
was developed in the solid edge CAD package using the sheet metal tool so as to be extremely
simple to manufacture and to ensure that the material for the design was readily available.
The design is to be cut from a sheet of 2mm aluminium and folded into shape and welded to
hold the shape, allowing for a simple and easily manufacturable design.

6.5 Design for Manufacture

The Guroo’s head has been designed to be manufactured as easily as possible from 2mm
aluminium from the 5005 series metal. It is to be manufactured from a single sheet of alu-
minium and simply cut and bent as can be seen from the diagram below. The design of the

A.R. Blower Development of a Vision System for a Humanoid Robot 36

CHAPTER 6. AESTHETIC HEAD DESIGN6.6. CAMERA HOUSING AND BRACKET

Figure 6.3: Final Head Design

Figure 6.4: Manufacturable Head

soccer ball head can be folded and then welded to complete the final head.

6.6 Camera Housing and Bracket

The mechanical design developed to directly develop maintain and operate the vision system
and in particular the digital camera is the camera housing. The mechanical design and elec-
tronic design have been developed in such a way as to create the most elegant and compact
solution to the design of the humanoid eye as possible. This housing has been developed for
the Viperoo team as well as the humanoid. The camera is completely contained within the
housing, a vast improvement of the original design by Mark Chang that had only the digital
camera chip inside the housing and much of the accompanying hardware outside exposed.

A.R. Blower Development of a Vision System for a Humanoid Robot 37

CHAPTER 6. AESTHETIC HEAD DESIGN 6.7. REMAINDER OF THE HUMANOID

Figure 6.5: Camera Bracket

This design was approximately two and half time the size of the new design. The housing
has also been designed to hold and focus the light from the image through the lens and onto
the photo active area on the digital camera chip. Along with the camera housing the bracket
for the camera housing has been designed simply to hold the camera in position and to be as
light as possible.

6.7 Remainder of the Humanoid

The remainder of the mechanical design was completed by Mark Wagstaff and can be found
in his thesis. This involved the design of a humanoid torso and limb with consideration to
walking and the overall goal of playing soccer.

6.8 Summary of Chapter

This chapter has outlined the requirements and design for the aesthetic design for the head
of the humanoid. The development of the digital camera housing and how the designs fit
together to produce the aesthetic head for the humanoid and camera housing for the Viperoos

A.R. Blower Development of a Vision System for a Humanoid Robot 38

CHAPTER 6. AESTHETIC HEAD DESIGN 6.8. SUMMARY OF CHAPTER

Figure 6.6: The Guroo Mechanical Design [22]

A.R. Blower Development of a Vision System for a Humanoid Robot 39
CHAPTER 6. AESTHETIC HEAD DESIGN 6.8. SUMMARY OF CHAPTER

small league robots has been explained.

A.R. Blower Development of a Vision System for a Humanoid Robot 40

Chapter 7

Results

“Results! Why, man, I have gotten a lot of results. I know several thousand
things that won’t work”. Thomas Edison

The results of any project are the important determinants as to whether a success or failure
has been recorded in the implementation of any design.

7.1 Product Evaluation

The evaluation of the project has been outlined below.

7.1.1 Field Programmable Gate Array Interface

The operation of the field programmable gate array using a dual port ram implementation
proved to be successful in simulation as can be seen from the flow graphs below. The first
timing diagram displays the inputs and outputs for a standard dual port ram implementation.
The signals above the compressed address lines are the input clocking and set state signals.
These can be seen to be reflected in the output signals below the compressed address buses.
This all displays a fully operational dual port ram system.

41
CHAPTER 7. RESULTS 7.1. PRODUCT EVALUATION

Figure 7.1: FPGA Dual Port Ram Simulation

7.1.1.1 Testing Procedure

The test procedure used on this part of the design involved the use of the Xilinx Foundation
Software to simulate the designs and produce timing diagrams. This software allows the user
to collect the various signals within the FPGA of interest and set them high or low as well as
clocking function and stimulus to added to each of the inputs or clocking signals. Once the
initial signal were in place the design was simulated for a preset length of time after which
the control signals were changed to test all possible signal possibilities.

7.1.2 CMOS Digital Camera

The digital camera clocking system operation was originally implemented using the 24MHz
crystal however when this design was built it was found that there was too much inductance
in the circuit due to the length of the connection between the crystal and the clock pins on
the OmniVision OV7620. The second option of the crystal oscillator was then placed on

A.R. Blower Development of a Vision System for a Humanoid Robot 42

CHAPTER 7. RESULTS 7.1. PRODUCT EVALUATION

Figure 7.2: Hitachi Dual Port Ram

A.R. Blower Development of a Vision System for a Humanoid Robot 43

CHAPTER 7. RESULTS 7.1. PRODUCT EVALUATION

the board, this small integrated circuit operated correctly allowing the clocking system to
operate effectively.
Currently the OV7620 CMOS digital camera is not operational, this may have been
caused by the component being destroyed whilst population of the board was being com-
pleted. Once power is supplied the chip will output data from the YUV output pins and the
pixel clock will produce a measurable clocking signal.

A.R. Blower Development of a Vision System for a Humanoid Robot 44

Chapter 8

Future Work

“The best way to predict the future is to invent it”. Alan Kay

Future work on the design of a vision system will have to include the complete and oper-
ational interface implementing not only the dual port ram system but a control interface to
allow the I2C command registers to be placed in the field programmable gate array. This will
allow more of the processing to be done outside of the Hitachi SH4 vision board.
In addition to this the parts of the vision object detection code can be ported onto the field
programmable gate array to allow further processing to be done prior to reaching the Hitachi
SH4 allowing higher resolution to be processed at a faster rate.

8.1 CMOS Digital Camera

Future work on the CMOS digital camera will not be necessary for some time as there are
problems in analysing the image data. A megapixel camera could have been implemented
operating at up to 500 frames per second however the whole vision system would have to
be redesigned with a faster processor and at this stage that processor does not exist for this
application. The OmniVision camera has an acceptable resolution of 640 x 480 pixels and the

45
CHAPTER 8. FUTURE WORK 8.2. FPGA INTERFACE

ability to do onboard subsampling or windowing of the image taken. All of these advantages
mean that the digital camera or “eye” of the humanoid will not need a redesign for some
time. Until the image processor becomes powerful enough to process the data no further
redesign of the digital camera is required.
Another consideration for the future generation of digital cameras design is the intro-
duction of bead inductors to reduce noise in the circuit which is a major problem in CMOS
digital cameras.

8.2 FPGA Interface

Future work on the development of the interface will be quite extensive with not only the
control I2C interface to be implemented but part of the image analysis software. The control
interface involves the copying of the registers in the OV7620 digital camera on the FPGA.
Once these registers are in the FPGA design all control of the camera control registers is to
be done by the FPGA and not the Hitachi SH4.
The other modification that can be made is porting part of the image analysis software
across to the field programmable gate array. This will involve using the FPGA as a read only
memory to compare colour from the camera to set values held in memory in the FPGA. For
further information on this refer Prasser[20].

A.R. Blower Development of a Vision System for a Humanoid Robot 46

CHAPTER 8. FUTURE WORK 8.3. CONCLUSION

8.3 Conclusion
Over the last 12 months the “Guroo” humanoid project has put together a complete design
for the development of a robot capable of playing soccer in the RoboCup soccer tournament.
To that end the design for the vision hardware and interface software has been developed. A
significant amount of learning has taken place during the project especially in the areas of
field programmable gate array design. The aim of this thesis was to design a vision system
capable of providing external data from the real world. In particular the head design of a
digital camera to act as the “eye” to the robot, design of interface software for a dual port
ram system for the control of data from the camera to the main vision board and the design
of an aesthetically pleasing head as the centre of attention for the humanoid. Part of this aim
was completed in this thesis, the simulation of the dual port ram and the design and choice
of a suitable digital camera for use in this design.
The major achievements for this thesis are listed below:

• Design of an aesthetically pleasing head.

• Choice of a suitable CMOS digital camera with characteristics that make it by far the
best component available.

• The simulation of an effective dual port ram system to control the data from the cam-
era.

• The simulation of the complete interface for the camera using the simulation packages
in the Xilinx foundation suit.

The major concern in the design of the digital camera was the differences in the data sheets
which in some cases contradicted each other, these data sheets came out at different times,
this attributed to the non operational status of the digital camera to the conclusion of this
thesis.

A.R. Blower Development of a Vision System for a Humanoid Robot 47

CHAPTER 8. FUTURE WORK 8.3. CONCLUSION

Future work should focus on the operation of the camera and continuing development of
the software to reduce the processing required to be done by the Hitachi SH4 to increase the
frame rate that can be analysed. From now on software will be the major concern for this
project.

A.R. Blower Development of a Vision System for a Humanoid Robot 48

Bibliography

[1] Brown Saddtocontents{toc}{Bibliography}, ”Fundamental of Digital Logic with

VHDL Design”,

[2] Data Sheet OV7620 Product Specification - Rev 1.3(5/3/00), "OV7620 Single Chip
CMOS VGA Colour Digital Camera", Omnivision, 2000.

[3] BEC Manufacturing Pty Ltd, "Engineering Guidelines", PCB Guidelines, http://
www.becman.com/BEC.htm (current APR. 26. 2001).

[4] C3PO, www.haight-ashbury.com/c3po.htm(current OCT 18. 2001)

[5] Gollegde Electronics Ltd., "Programmable Oscillator EXO - 3C", http://www.

golledge.com/pdf/products/osc_ld/exo3c.pdf (current APR. 26. 2001)

[6] Duncan Technologies, Inc, "Overview of Digital Camera Technology", http://www.

duncantech.com/Dig_Camera_OV.htm#Camera%20Resolution (current APR. 26.
2001).

[7] Dr Semel, “How Eyes Work”, http://drsemel.net.eyeworks1.htm(current OCT.

05.2001).

[8] PeaCock, Craig, “CMOS DIgital Image Sensors”, http://www.beyondlogic.org/

imaging/camera.htm (current OCT. 05. 2001).

49
BIBLIOGRAPHY BIBLIOGRAPHY

[9] Ziff Davis Media Inc, “Anatomy of a Digital Camera: Image Sensors”,
http://www.extremetech.com/article/0,3396,apn%253D5%2526s%253D200%
2526a%253D2036%2526app%253D3%2526ap%253D4,00.asp (current OCT. 05. 2001).

[10] Dr. Mark S. Drew, “Basics of Color”, http://www.cs.sfu.ca/cs/CC/365/mark/

material/notes/Chap3/Chap3.3/Chap3.3.html (current OCT. 05. 2001).

[11] Denyer, Peter B,” CMOS vs CCD”, http://www.vvl.co.uk/whycmos/whitepaper.

htm (current OCT. 05. 2001).

[12] Dalsa, “CCD vs. CMOS”, http://www.dalsa.com/markets/ccd_vs_cmos.asp

(current OCT. 05. 2001).(current OCT. 05. 2001).

[13] OmniVision, “OmniVision”, http://www.ovt.com/ (current OCT. 05. 2001).

[14] Bourke, Paul, “RGB colour space”, http://astronomy.swin.edu.au/pbourke/

colour/colourspace/ (current OCT. 05. 2001).

[15] United Visual, Inc, “Interlaced and progressive scanning”, http://www.

lcdprojector.net/2tips/2tDTV104.asp (current OCT. 05. 2001).

[16] Maller, Joe, “RGB and YUV Color”, http://www.joemaller.com/fcp/fxscript_

yuv_color.shtml (current OCT. 05. 2001).

[17] Accowle CO. LTD, “Omnidirectional Vision Sensors”,www.pluto.dti.ne.jp/

~accowle1 (current OCT. 05. 2001).

[18] VHDL and AHDL digital systems implementation

[19] Fryer Thesis, “Automated design of specialized FPGA processor memory intensive
applications”.

A.R. Blower Development of a Vision System for a Humanoid Robot 50

BIBLIOGRAPHY BIBLIOGRAPHY

[20] Prasser D, “Vision Software for a Humanoid Soccer Robot”, University of Queensland,
2001.

[21] Hosking S, “High Speed Pheipherial Interface”, Univeristy of Queensland, 2001.

[22] Wagstaff M,”Mechanical Design for a Humanoid Robot”, University of Queensland,

2001.

[23] Kee D, “Drive System Selection and Simulation for a Humanoid”, University of
Queensland, 2001.

[24] Chang M, Browning B,”Viper Roos”, http://www.itee.uq.edu.au/~chang/

ViperRoos/index.html (current OCT.16.2001).

[25] Horn, “Robot Vision”

[26] c3po, www.haight-ashbury.com/c3po.htm(current OCT. 18. 2001).

A.R. Blower Development of a Vision System for a Humanoid Robot 51

Appendix A

Camera Schematic

52
APPENDIX A. CAMERA SCHEMATIC

A.R. Blower Development of a Vision System for a Humanoid Robot 53

Appendix B

Camera Printed Circuit Board

54
APPENDIX B. CAMERA PRINTED CIRCUIT BOARD

A.R. Blower Development of a Vision System for a Humanoid Robot 55

Appendix C

Dual Port Ram

56
APPENDIX C. DUAL PORT RAM

– VHDL Dual Port Ram

– This program allow reading and writting to ram simultaneously using
– a dual prot ram implementation.
– Author: Andrew Blower
– Student Number: 33539973

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_unsigned.all;

entity dual_port_ram is
generic( d_width : integer := 4;
mem_depth : integer := 256); – memory array size

port ( clk : in STD_LOGIC;

CS : in STD_LOGIC; – chip select
we : in STD_LOGIC; – write enable
indata : in STD_LOGIC_VECTOR(7 downto 0); – data input
outdata : out STD_LOGIC_VECTOR(7 downto 0); – data output
raddr, waddr : in STD_LOGIC_VECTOR(7 downto 0)); – read address and
write address
end dual_port_ram;

architecture data of dual_port_ram is

–memory allocation of 256 memory locations

type mem_type is array (127 downto 0) of

A.R. Blower Development of a Vision System for a Humanoid Robot 57

APPENDIX C. DUAL PORT RAM

STD_LOGIC_VECTOR (7 downto 0);

signal mem : mem_type;

begin

process(clk, we, waddr)

begin
if (rising_edge(clk)) then
if (we = ’1’) then
– write data into memory
mem(conv_integer(waddr)) <= indata;
end if;
end if;
end process;

process(raddr, clk)
begin
if(rising_edge(clk))then
if(CS = ’0’)then
– read data from memory
outdata <= mem(conv_integer(raddr));
end if;
end if;
end process;

end data;

A.R. Blower Development of a Vision System for a Humanoid Robot 58

APPENDIX C. DUAL PORT RAM

– VHDL Dual Port Ram

– This program allow reading and writting to ram simultaneously
– Author: Andrew Blower
– Student Number: 33539973

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_unsigned.all;

entity camera_ram is

port ( clk : in STD_LOGIC;

we : in STD_LOGIC; – write enable
– single address bus
address : in STD_LOGIC_VECTOR(6 downto 0);
– data write
dataw: in STD_LOGIC_VECTOR(15 downto 0);
– data read
datar: out STD_LOGIC_VECTOR(15 downto 0);
PCLK: in STD_LOGIC; – signal inputs
HREF: in STD_LOGIC;
VSYNC: in STD_LOGIC;
SI0: in STD_LOGIC;
SI1: in STD_LOGIC;
XCLK: in STD_LOGIC;
opclk: out STD_LOGIC; – signal outputs
ohref: out STD_LOGIC;

A.R. Blower Development of a Vision System for a Humanoid Robot 59

APPENDIX C. DUAL PORT RAM

ovsync: out STD_LOGIC;

oxclk: out STD_LOGIC;
osi0: out STD_LOGIC;
osi1: out STD_LOGIC);
end camera_ram;

architecture camera_data of camera_ram is

type mem_type is array (128 downto 0) of –memory allocation of 256 memory

locations
STD_LOGIC_VECTOR (15 downto 0);
signal mem : mem_type;

begin

process(we, address)
begin
if (we = ’1’) then
mem(conv_integer(address)) <= dataw(15 downto 0);
end if;

if(we = ’0’)then
datar(15 downto 0) <= mem(conv_integer(address));
end if;

end process;

A.R. Blower Development of a Vision System for a Humanoid Robot 60

APPENDIX C. DUAL PORT RAM

opclk <= PCLK;

ohref <= HREF;
ovsync <= VSYNC;
oxclk <= XCLK;
osi0 <= SI0;
osi1 <= SI1;
end camera_data;

A.R. Blower Development of a Vision System for a Humanoid Robot 61

Appendix D

Test Code Counter

62
APPENDIX D. TEST CODE COUNTER

-- module: counter.vhd

-- This is the top level module that ties all sub-modules together

library IEEE;
use IEEE.std_logic_1164.all;

entity counter is
port ( clk : in STD_LOGIC;
reset : in STD_LOGIC;
lcd_com : out STD_LOGIC;
one_dp : out STD_LOGIC;
ten_dp : out STD_LOGIC;
one_out : out STD_LOGIC_VECTOR(6 downto 0);
ten_out : out STD_LOGIC_VECTOR(6 downto 0));
end counter;

architecture counter_arch of counter is

component div_20
port ( clk : in STD_LOGIC;
clk_2k : out STD_LOGIC);
end component;

component div_24
port ( clk : in STD_LOGIC;
tc_1s : out STD_LOGIC);
end component;

component cnt_10
port ( ce : in STD_LOGIC;
clk : in STD_LOGIC;
clr : in STD_LOGIC;
tc : out STD_LOGIC;
qout : out STD_LOGIC_VECTOR(3 downto 0));
end component;

component hex2lcd
port ( hex : in STD_LOGIC_VECTOR(3 downto 0);
lcd : out STD_LOGIC_VECTOR(6 downto 0));

A.R. Blower Development of a Vision System for a Humanoid Robot 63

APPENDIX D. TEST CODE COUNTER

end component;

component lcd_mux
port ( clk : in STD_LOGIC;
cnt : in STD_LOGIC;
data_in : in STD_LOGIC_VECTOR(6 downto 0);
lcd_seg : out STD_LOGIC_VECTOR(6 downto 0);
lcd_com : out STD_LOGIC;
lcd_dp : out STD_LOGIC);
end component;

signal nreset : STD_LOGIC;

signal lcd_clk : STD_LOGIC;
signal ones_en : STD_LOGIC;
signal tens_en : STD_LOGIC;
signal tc_ones : STD_LOGIC;
signal cnt_1s : STD_LOGIC_VECTOR(3 downto 0);
signal cnt_10s : STD_LOGIC_VECTOR(3 downto 0);
signal lcd1_out: STD_LOGIC_VECTOR(6 downto 0);
signal lcd10_out: STD_LOGIC_VECTOR(6 downto 0);

begin

DIV20:div_20 port map(clk=>clk,

clk_2k=>lcd_clk);

DIV24:div_24 port map(clk=>clk,

tc_1s=>ones_en);

ONES:cnt_10 port map(ce=>ones_en,

clk=>clk,
clr=>nreset,
tc=>tc_ones,
qout=>cnt_1s);

TENS:cnt_10 port map(ce=>tens_en,

clk=>clk,
clr=>nreset,
qout=>cnt_10s);

A.R. Blower Development of a Vision System for a Humanoid Robot 64

APPENDIX D. TEST CODE COUNTER

ONES_LCD:hex2lcd port map(hex=>cnt_1s,

lcd=>lcd1_out);

TENS_LCD:hex2lcd port map(hex=>cnt_10s,

lcd=>lcd10_out);

ONES_MUX:lcd_mux port map(clk=>clk,

cnt=>lcd_clk,
data_in=>lcd1_out,
lcd_seg=>one_out,
lcd_com=>lcd_com,
lcd_dp=>one_dp);

TENS_MUX:lcd_mux port map(clk=>clk,

cnt=>lcd_clk,
data_in=>lcd10_out,
lcd_seg=>ten_out,
lcd_dp=>ten_dp);

tens_en<=tc_ones and ones_en;

nreset<=not(reset);

end counter_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 65

APPENDIX D. TEST CODE COUNTER

-- Module: lcd_mux.vhd

-- This module controls the bias reversal for a single 7-segment

-- display. The cnt input xor’s the segment data to cause the
-- segment biasing to be swapped. The cnt signal should be about
-- 20-30Hz for the LCD to function correctly. The common signal is
-- also bias reversed at this same frequency.

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_arith.all;
use IEEE.std_logic_misc.all;
use IEEE.std_logic_unsigned.all;

entity lcd_mux is
port ( clk : in STD_LOGIC;
cnt : in STD_LOGIC;
data_in : in STD_LOGIC_VECTOR(6 downto 0);
lcd_seg : out STD_LOGIC_VECTOR(6 downto 0);
lcd_com : out STD_LOGIC;
lcd_dp : out STD_LOGIC);
end lcd_mux;

architecture lcd_mux_arch of lcd_mux is

begin

process(clk)
begin

if(clk’event and clk=’1’) then

lcd_seg(0) <= data_in(0) xor cnt;
lcd_seg(1) <= data_in(1) xor cnt;
lcd_seg(2) <= data_in(2) xor cnt;
lcd_seg(3) <= data_in(3) xor cnt;
lcd_seg(4) <= data_in(4) xor cnt;
lcd_seg(5) <= data_in(5) xor cnt;
lcd_seg(6) <= data_in(6) xor cnt;
lcd_com <= ’0’ xor cnt;
lcd_dp <= ’0’ xor cnt;

A.R. Blower Development of a Vision System for a Humanoid Robot 66

APPENDIX D. TEST CODE COUNTER

end if;

end process;

end lcd_mux_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 67

APPENDIX D. TEST CODE COUNTER

-- module: div_20.vhd

-- This module divides the incoming clock by 2^20 and outputs the
-- 20th bit of the counter as clk_2k.

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_arith.all;
use IEEE.std_logic_misc.all;
use IEEE.std_logic_unsigned.all;

entity div_20 is
port ( clk : in STD_LOGIC;
clk_2k : out STD_LOGIC);
end div_20;

architecture div_20_arch of div_20 is

signal count : STD_LOGIC_VECTOR(19 downto 0);

begin

process(clk)
begin

if(clk’event and clk=’1’) then

count<=count + "00000000000000000001";
end if;

end process;

clk_2k<=count(19);

end div_20_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 68

APPENDIX D. TEST CODE COUNTER

-- module: div_24.vhd

-- This module divides the incoming clock by 2^24 and generates a

-- terminal count pulse of one clock width during the counter rollover.

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_arith.all;
use IEEE.std_logic_misc.all;
use IEEE.std_logic_unsigned.all;

entity div_24 is
port ( clk : in STD_LOGIC;
tc_1s : out STD_LOGIC
);
end div_24;

architecture div_24_arch of div_24 is

signal count : STD_LOGIC_VECTOR(23 downto 0);

begin

process(clk)
begin

if(clk’event and clk=’1’) then

count<=count + 1;
end if;

if(count="111111111111111111111111") then
tc_1s <= ’1’;
else
tc_1s <= ’0’;
end if;

end process;

end div_24_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 69

APPENDIX D. TEST CODE COUNTER

-- Module: cnt_10.vhd

-- This module counts from 0 to 10 and then rolls over.

library IEEE;
use IEEE.std_logic_1164.all;
use IEEE.std_logic_arith.all;
use IEEE.std_logic_misc.all;
use IEEE.std_logic_unsigned.all;

entity cnt_10 is
port ( ce : in STD_LOGIC;
clk : in STD_LOGIC;
clr : in STD_LOGIC;
tc : out STD_LOGIC;
qout : out STD_LOGIC_VECTOR(3 downto 0));
end cnt_10;

architecture cnt_10_arch of cnt_10 is

signal count : STD_LOGIC_VECTOR(3 downto 0);

begin

process(ce,clk,clr)
begin

if(clr=’1’) then
count <="0000";
elsif(ce=’1’) then
if(clk’event and clk=’1’) then
if(count="1001") then
count<="0000";
else
count<=count + "0001";
end if;
end if;
end if;

end process;

A.R. Blower Development of a Vision System for a Humanoid Robot 70

APPENDIX D. TEST CODE COUNTER

qout<=count;

tc<=’1’ when count="1001" else ’0’;

end cnt_10_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 71

APPENDIX D. TEST CODE COUNTER

-- module: hex2lcd.vhd

-- This module converts a 4 digit binary number into the LCD format
-- required by the 7-segment LCD display.

library IEEE;
use IEEE.std_logic_1164.all;

entity hex2lcd is
port (
hex: in STD_LOGIC_VECTOR (3 downto 0);
lcd: out STD_LOGIC_VECTOR (6 downto 0)
);
end hex2lcd;

architecture hex2lcd_arch of hex2lcd is

begin

--HEX-to-seven-segment decoder
--
-- segment encoding
-- 0
-- ---
-- 5 | | 1
-- --- <- 6
-- 4 | | 2
-- ---
-- 3

with hex select

lcd<= "0000110" when "0001", --1

"1011011" when "0010", --2
"1001111" when "0011", --3
"1100110" when "0100", --4
"1101101" when "0101", --5
"1111101" when "0110", --6
"0000111" when "0111", --7
"1111111" when "1000", --8
"1101111" when "1001", --9

A.R. Blower Development of a Vision System for a Humanoid Robot 72

APPENDIX D. TEST CODE COUNTER

"1110111" when "1010", --A

"1111100" when "1011", --b
"0111001" when "1100", --C
"1011110" when "1101", --d
"1111001" when "1110", --E
"1110001" when "1111", --F
"0111111" when others; --0

end hex2lcd_arch;

A.R. Blower Development of a Vision System for a Humanoid Robot 73

Appendix E

Camera Housing

74
APPENDIX E. CAMERA HOUSING

A.R. Blower Development of a Vision System for a Humanoid Robot 75

Appendix F

Camera Bracket

76
APPENDIX F. CAMERA BRACKET

A.R. Blower Development of a Vision System for a Humanoid Robot 77

APPENDIX F. CAMERA BRACKET

A.R. Blower Development of a Vision System for a Humanoid Robot 78

Appendix G

Head Original

79
APPENDIX G. HEAD ORIGINAL

A.R. Blower Development of a Vision System for a Humanoid Robot 80

Appendix H

Head Final

81
APPENDIX H. HEAD FINAL

A.R. Blower Development of a Vision System for a Humanoid Robot 82

APPENDIX H. HEAD FINAL

A.R. Blower Development of a Vision System for a Humanoid Robot 83

Appendix I

Head Assembly

84
APPENDIX I. HEAD ASSEMBLY

A.R. Blower Development of a Vision System for a Humanoid Robot 85