Slide 1

PERCEPTION OF
COLOR AND DEPTH
An Empirical View
Slide 2

Questions
How do we see?
Why is context important?
How do we see color?
How do we see depth?
How does this all relate?

How do we see?
Why is context important?
How do we see color?
How do we see depth?
How does this all relate/why should I care  tie to purves, or should
this be done at the beginning?
Slide 3

How do we see?
Jacob talks about brain mechanisms
of seeing color

JACOBS SHIT GOES HERE!

Slide 4

How do wesee, really?

But I don’t know Jacob, that might be good description of some of the
mecahanisms behind vision, but I want to know how we really see.

Everyone think back to when you were a little kid, and try to remember
the first idea you had about our eyes, and how they see. What do you
remember?

When we thought of seeing, we thought of the retina like a camera,

taking a picture of the scene in front of us, and decoding wavelengths
of light to determine color. I guess this is more of a reconstructionist
point of view. But we are going to see that it does not work that way,
that context is important. So before we jump into color, I want to talk
about the first group of people to realize that context is important in
vision, the gestalt theorists.
Slide 5

Gestalt Theory
The first theory that recognized the
importance of context

This theory was developed in the 1920’s, and these guys were the first
group of psychologists to systematcially study perceptual organisation
. I think they were a bunch of German guys :P

So we’ll just run through these principles quickly because they serve as
a good starting base to show how important context is in deciphering
what we see, and then connecting this idea to color and depth. And I
think thee are fun. There are six principles
Slide 6

Principle 1: Proximity

Principle one relates to proximity, Who can tell me what different

groups they see on this screen?

1 and 2
3 and 4
3 groups dots in three lines
What happens with the evenly spaced dots?
So on the left we see horizontal groups, and on the right we see
vertical groups.

So the first principle of proximity or contiguity says that things that are
closer together are seen as belonging together.
Slide 7

Principle 2: Similarity

Principle 2 relates to similarity. Now where do we see the groups?

THat’s riiight! We tend to see groups that have similar characteristics,
so blacks and white. And how do we see these groups arranged? In
lines… The principle of similarity says that things that share visual
characteristics like shape, size, color, texture, value or orientation are
seen as belonging together.
Slide 8

Principle 3: Common Fate

Principle three is basically combining the first two, it’s called common
fate. This is a subtle illusion, but cool nonetheless. When both the
principles of proximity and similarity are used, then we actually see a
little bit of movement/. Do you guys see that?
Why do you think that is?

I think it’s because we jump back and forth from grouping based on
proximity, and grouping based on similarity. So I want to group the
ones on the top based on proximity, but not this guy down here, but
then my mind shifts to wanting to group based on similarity, which
includes them all.
Slide 9

Principle 4: Good Continuation

The fourth principle relates to the idea that we prefer to see things as
unified. So in this picture we actually have 4 lines, but instead of
seeing A to O and O to D, we just see A to D, and the same with
instead of seeing C to O and O to B we see C to B
lines.

So this is called the principle of continuity, and it predicts our

preference for continuous figures. Theres no way that there are 4
crosses lines here, there are two crossed lines.
Slide 10

Principle 5: Closure

Principle five is related to the last one in the idea that not only to we
want continuity, we want to see closed, simple figures. So our visual
system fills in the gaps.

So the circle at the top its easy to see this, especially if you look
slightly away. In the second figure we see two overlapping rectangles
as opposed to two rectangles with chunks cut out touching corners, or
you could just see three shapes touching, which I actually had a hard
time seeing.

The third can be looked at like a curve with three squares, or just as
three random, irregular shapes touching. What do you guys see?
Slide 11

Principle 6: Area and Symmetry

The sixth and last principle, the principle of area and symmetry, says
that the smaller of two overlapping figures is perceived as a figure
while the bigger one is seen as ground.

And with symmetrical figures we tend to see closed figure, and the
Symmetrical contours tend to separate the defined figure from the
ground. And we will be looking at some cool examples of this a bit
later.
Slide 12

Color Perception
An Empirical View

So let’s apply this stress on context first to color!

some of the demonstrations I am going to show you today, and asked
us to think a little outside of the box, so to speak.
Slide 13

HUE

SATURATION

BRIGHTNESS

Before we jump in I want to refresh your memory and define some

terms that are important to know when talking about color… hue,
saturation, and brightness.

Hue is like is it red, is it blue, etc.

Saturation is the INTENSITY of a specific color, which may look familiar

to those of you that like to play around in photoshop, the more highly
saturated is towards the top, and brightness is the luminesence of a
visual target, which is a little hard to show on a computer screen.
Slide 14

EMPIRICAL STRATEGY OF
PERCEPTION

thecolor an observer sees is entirely determined by the

probabilitydistribution of the possiblesources of the
stimulus

Context is important!

EMPIRICAL STRATEGY OF PERCEPTION: So purves applied this empirical

strategy of vision on color, luminesence, and suggested that it could be
applied to all aspects of perception. Im going to focus exclusively on
color, color constancy and color contrast and try to argue that the color
an observer sees is entirely determined by the probability distribution
of the possible sources of the stimulus

So let’s dumb this down a bit to a jargon that I can understand. we

have a scene, and we have light, and that makes us see color, right?
Well one thing that makes this perception so hard is the fact that they
are influenced by the context of the scene. So let’s first look at a
broad example.
Slide 15

Why is context important?

So first I want to try and convince you that context is important. This
picture answered that really well for me. So these stimuli, pointed out
in the diagram at the bottom, appear to differ in respect to distance
from the viewer, luminesence, surface reflectivity, and obviously, color.
But guess what… we use a little photoshop magic to get rid of the rest
of the stuff and… they really are the same
Slide 16

Color Contrast

So let’s try to explain this phenomenon… and I’m going to

start by talking about color contrast. Here you see two targets
on an orangish and a nasty yellow background, and how do
they look to you in terms of color? So I know that you know
that althugh they look different, I am going to reveal to you
that we are actually looking at the same color. Ta da! They
only look difference is in the context of the two targets.
Differences aren’t in the color, but in the perceived hue and
saturation of the targets in the context of the background. I
chose this example in particular because these colors arent
different in terms of brightness and luminesence, so we are
seeing a phenomenon just related to hue and saturation, color
contrast.
Slide 17

Color Constancy

So that was color contrast, conversely we have color constancy. So id

like to draw your attention to these squares on the left, and these on
the right. What color do they look like to you guys? (I guess that is sort
of leading you on by saying color… because when we add a mask, we
see that they are very different colors, so this Is called color contansy,
and that is taking different colors and putting them in different
contexts so they look similar (example)

So This presents an obvious problem for explanations of color vision in

physiological terms.
Obviously whatever is there isnt just taken a shapshot of and uploaded
into our brains.
Slide 18

So when we have a situation of the same spectral targets eliciting

different color sensations Land, who came up with the first theory of
color perception called retinex theory in the 1980’s, said that this could
be ascribed to 'adaptation' of the color system to the average spectral
content of the overall stimulus or to computations of spectral ratios
across chromatic contrast boundaries. And here is an example of that.
Focus your eyes on this black dot, and Ill time 30 seconds. So what did
you guys see??

That’s right, so you saw a lack of green in the picture on the left and a
lack of red in the picture on the right, and then it took you a bit a
staring to realize that they are actually exactly the same. The idea is
that our visual system adapted to the red and the green at the input
stages.
(show counterexample picture with t t)

So great, Land say that we adapt, but this idea doesn’t provide a good
explanation, or biological rationale really.
Slide 19

Color Contrast and constancy: an empirical explanation

Phylogenetic

So let’s bring a cool guy named Purves into the picture, and give an
explanation in empirical terms for color constancy and color contrast.
Now the idea behind the empirical strategy of vision is that everything
that we see is dependent on the historical success or failure of the
interactions between what I see and me. So when I look at a scene the
way that I see it is dependent on my past perceptions. So you can
think about it as your brain seeing what probabilistically makes most
sense in the environment. So it isn’t that my mind breaks it up into
little pieces and shoots everything out like a camera, but that what we
see is literally and figuratively colored by what we’ve percevied in the
past.

So let’s give an emprical explanation of color contrast (and constancy).

So Id like to argue that the visual stimuli that I’ve shown you up to thi
point, are actually pretty ambiguous in the sene that many
combinations of reflectances, conditions of illumination and influences
of transmittance could generate the same perception. It is the job of
the visual system to take in all of these patterns that are returned to
the eye and create a behavioral response that makes sense. What we
see is based on the successes and failures of what we have seen
before. I like to think of the idea like a neural network, that you have
back propogration and the weights of the nodes are adjusted to
promote more appropriate reactinos to the stimuli.
We can call this a phylogenetic process, phylogenetic meaning there is
evolutionary relatedness amongst members of a species., you can sort
of think of it like lineage. You experience a visual stimuluus, there is a
certain neuronal response, and those responses get linked with specific
spectral profiles so they have significance. So this means that then,
the pattern of neuronal activity that you experience in response to a
visual stimulus, the strength of that network, is determined by the
number of occurrences of the combinations of colors, reflectances,
illuminants and transmittances that have lead to successful behavior in
response to that visual stimulus in the past. Ok, methinks we need
more examples!
Slide 20

So Jacob is going to talk about brightness and luminesence in a

second, and I think I’d like to use an example with color and some
brightness to explain this idea of how past experiences influence what
we see.

So if this is true, then when we put targets of the same color on two
contexts, two different colors, we might experience them differently
based on their context.

So this is very cool, here we have similar surfaces under similar

illuminants, and you can see that the purple target in the middle looks,
purple in both cases.
Slide 21

Here we have the exact same surfaces BUT they are seen under
different illuminants, the left one a blue light, and the right one a
purple light. What do you guys see now? Yeah I see purple on the left
and more of a blue on the right.

So as human beings we have to be able to make appropriate

responses, behavior wise, to many thing that we see. And of course
this isn’t something that you consciously think about. And what a
challenge that is because the variety of different combinatinos of color,
brightness, the number of different contexts that we can see is
basically infinite. So the only way to do this is to use successes and
failures of the past, and then adjust the weights in this neural network.
This is the main idea of the empirical strategy of vision. This idea
applys to brightness, luminence, and reflectivity as well, which Jacob is
going to cover next.
Slide 22

Depth Perception
How do we see depth?
Slide 23

Four Key Ideas

• Binocular Disparity
• Neural correlates
• Ventral vs. dorsal streams
• Amblyopia, squint, and awareness
Slide 24

What is binocular disparity?

• Each eye obtains a different image of the world

• Difference between the two images is called
binocular disparity
• Requires high accuracy from CNS to register
differences between the two images
– Differenceis usually smaller than the width of a
singlephotoreceptor

-each eye has separate image due to their horizontal separation

Slide 25

cont. binocular disparity

• How does the brain extract these differences?

– Individual features in the left eye’s image must be
matched with the correspondingimage on the right
eye
– Ex. For object recognition, eye’s must match up and
also match sensory input with what is sored in
memory

-complexity deals with not only matching visual stimuli, but also
making sense of that stimuli
Slide 26

Neural Correlates
• Absolute vs. relative disparity
• Binocular anti-correlation
• Ambiguity in stereomatching
Slide 27

AbsoluteDisparity
• Suppose two eyes looking at one point
• The angular difference in the projections of that
point onto the left and right eyes with reference
to each eye’s fovea
Slide 28

Relative Disparity
• Takes into account two points
• Difference in the absolute disparity of both eyes
with respect to both points
– Eliminates fovea as areference point
– Differencebetween the angle subtended by the two
points on the left eye and the angle between those
point on theright eye
Slide 29

-point to the Alpha minus Beta

-explain that fovea is not reference point, but rather r.d. is a
reflection of the spacing of objects with varying depth
Slide 30

• Human stereoscopic system relies more on

relative disparity when assessing depth
• V2 areas show more sensitivity to complex
relative disparity; V1 shows no sensitivity to
relative disparity
• Different spatial configurations result in the
same numerical relative disparity
– MT shows some sensitivity for relative disparity for
certain stimuli configurations
– Spatial layout for relative disparities is important

-MT shows sensitivity to rel. disp. When a single plane is tiled to the
observer and has a relative disparity between the nearer and further
edges of the plane; in same region, neurons show no snesitivity to
relative disparity when center-surround configuration is used
-also highly responsive to introduction of a relative disparity in a
rotating transparent cylinder stimulus
-it is impossible to apply such a label as ‘relative disparity’ to a
particular brain region without being more specific about the kinds of
relative disparity referred to
Slide 31
Slide 32

Binocular anti-correlation
• Stimuli presented visual features of opposite
contrast to the left and right eyes
• Random dot stereogram:
• Pair of pictures, one for each eye; randomly composed of
black and white elements
• Stereopsisreveals previously hidden figure (think: magic
eyebooks)

-stereopsis: perception of depth that occurs when information from the

right and left eyes are encoded
Slide 33

With stereoscopic vision,

oneimage can be
presented to your left
eye, and the other to the
right eye
Anticorrelation prevents
theperception of
stereoscopic depth

-by either converging or diverging the eyes

-three possibilities:
-dot patterns presented to the left and right eyes are based on
same sequence; correlation = 1; circle defined by depth should be
seen
-dot patterns based on independent sequences, correlation = 0;
no circle is defined
-dot patterns are based on same sequence only for every dark
circle in one eye, theree is a light circle on the other eye and vice
versa; stereogram results in anticorrelation (-1)
Slide 34

Magic Eye

-supposed to be a locomotive
Slide 35

Ambiguity in Stereomatching
• When looking at two or more vertical bars,
subjects report seeing the expected number of
bars in the same depth field
– Each bar has same zero disparity
– Correctly appear in same picture plane
• So what is the problem?
Slide 36

• Each eye receive identical image of bars

• Binocular system sorts which bar as seen by left
eye belongs with which bar seen by the right eye
• Brain region that responds to a match generates
“false matches”
• Their disparity with respect fo fixation point is
such that they would signladifferent binocular
depth in front of or behind the picture plane
• But this doesn’t actually happen….
Slide 37

• V1 neurons fire just as strongly for “false

matches” as for genuine ones
• However…. In V2, most neurons sfirestronger
for genuine matches
• As a result, we don’t actually see these “false
matches”
Slide 38

Dorsal and Ventral Streams

• Old theories rejected:
– Stereoscopic depth exlusiveto ventral or dorsal
stream
– Simpledivision of:
• Coarse stereopsisin dorsal stream
• finestereopsisin ventral stream
Slide 39

• New theory: the two streams carry out

fundamentally different types of stereo
communication
– Each streamis specialized and contributes to
stereoscopic vision in a unique way
– Dorsal: simple computation of both relative and
absolute disparity
– Ventral: sophisticated communication of relative
disparity including 3-D texture

-dorsal: relies on direc5t computation of the bin. Correlation between

the left and right eye images’; relative disparity is only shown in
processing of spatially extended surfaces and the segregation in depth
of one surface to another; suggests that bin. Depth in dorsal stream
helps the individual to navigate the world
-ventral: full resolution of bin. Matching problem and neurons in this
strean appear to ve specifically sensitive to relative deoth between
different features located at nearby positions; sensitivity to shape and
curvature of three-dimensional figures
Slide 40

Stereovision in dorsal and ventral pathways

(monkey cortical areas)
Slide 41

Problems with binocular vision

• Complexity of binocular vision oeaves room
for developmental complications
• Amblyopia
• Squint
Slide 42

Amblyopia
• Poor vision as a result of a healthy eye having
faulty connection with the rest of the brain
• Disrupted transmission of the visual image
• Not corrected by lenses
• Incorrect linage of one eye cause loss of
binocular disparity
Slide 43

GREG’S SHIT…
• Here!
Slide 44

What we’ve talked about…

• Gestalt Theory
• Mechanics of Vision
• Color Perception
• Depth Perception

So what we’ve covered today is a little bit of the history of the study of
vision, related to Gestalt theory and the first theory that recognized
that context is important, and then some mechanics of vision, and tied
that to color and depth perception. So before we get into a break and
then the next activity, I’d like to reveal to you that next week, the big
man, Purves himself, is actually coming to our class to have a
conversation with us. And I have no authority to make an assignment,
but I’ve heard he loves challenges, he loves questions, and just getting
people talking, so I thought it would be cool for everyone to come up
with maybe a question for him.

I’ll explain the activity after the break!!

So we are going to do a little activity to get you guys looking at and

playing with some illusions, so since working alone is scary for some
but we don’t want to have HUGE group dynamics so some people talk
a lot and others no so much, we want you to split into groups of two.
We are going to give each group a cool illusion to play with, then you
get to present your illusion to the class, and talk about it in terms of
some of the ideas we just outlined. And after you do that, in
celebration of valentines day, or single awareness day, you get candy!
Sound cool?

----------------------------------
EXTRA STUFF

When surface illuminated by spectrum similar the intensity is

INCREASED and the width of distribution narrows, as in 1

When surface is illuminated by a spectrum that is opposite in its

distribution of power, intensity is decreases and width of distribution is
broadened and flattened

When surface is illuminated by spectrum that is neither the same nor

opposite,
the spectral return shifts along the x-axis in the direction of the
spectral
profile of the illuminant

And we can measure the contrast effect by placing a stimulus on top of

a background and having subjects adjust contrast, hue, and britness
until it matches a side color, and I don’t need to get into the specifics,
but its always the case that the color contrast theory applies!

Empirical explanation color contrast:

this effect can be understood
as the natural outcome of a visual strategy in which color percepts are
generated
according to the relative contribution of illumination, reflectance and
transmittance
to similar stimuli experienced in the past

But when we look at a scene there is always always more than one
thing, so our visual system has to decide soo am I looking at
reflectances under the same illuminant (Fig.
4B), in which case they should appear similarly colored, or different
reflectances
under different illuminants

If
perceptions of color are indeed generated in a wholly empirical way,
then identical
targets presented on differently chromatic backgrounds should give
rise to different
chromatic sensations, as indeed they do.
the chromatic appearance of two identical targets will always follow
the
probability distribution of the possible sources of the target spectra,
given the
constellation spectral returns from the rest of the scene.