L4 Vision

一、Why Study Vision

We are highly visual animals.

Vision occupies a large percent of the cortex (Approx. 5 billon neurons)

Vision is the best understood system in the brain, in part because of prior neurophysiology work in monkeys

• Until very recently, no AI system could touch us, now they can in some respects (pattern recognition), but there is still a great deal we can do that AI systems cannot understanding human vision should enable better AI

How Can We See

Different aspects of what the eyes sees, like form, color, depth and motion, are transmitted to different areas of visual cortex, via different pathways from the retina. It is the continuous interaction of various computations in the visual cortex that stitch those different aspects together and culminate in the perception.

With each eye fixation, we experience a richly detailed visual world.

The richness of our visual experience leads us to believe that our visual representations will include and preserve the same amount of detail (Levin et al 2000).

二、The Visual System

The Anatomy

Coding of Visual Information in the Retina

1. The Eyes
2. Photoreceptors
3. Coding of Light and Dark
4. Coding of Color
5. Connections Between Eye and Brain

Analysis of Visual Information: Role of the Striate Cortex

1. Anatomy of the Striate Cortex
2. Orientation and Movement
3. Spatial Frequency
4. Retinal Disparity
5. Color
6. Modular Organization of the Striate Cortex

Analysis of Visual Information: Role of the Visual Association Cortex

1. Two Streams of Visual Analysis
2. Perception of Color
3. Perception of Form
4. Perception of Movement
5. Perception of Spatial Location

The Stimulus

Our eyes detect the presence of light.

For humans light is a narrow band of the spectrum of electromagnetic radiation.

Electromagnetic radiation with a wavelength of between 380 and 760 nm is visible to humans.

Other animals can detect different ranges of electromagnetic radiation.

The perceived color of light is determined by three dimensions:

1. hue : dominant wavelength
2. brightness: intensity
3. saturation: purity

• This figure shows examples of colors with the same dominant wavelength (hue) but different levels of saturations or brightness.

The Eyes

• The Human Eye

• Retina: The neural tissue and photoreceptive cells located on the inner surface of the posterior portion of the eye

1. Retina

Retina: The neural tissue and photoreceptive cells located on the inner surface of the posterior portion of the eye

• Photoreceptor: Receptor cells of the retina; transduces photic energy into electrical potentials.
  • Cone: One of the receptor cells of the retina; maximally sensitive to one of three different wavelengths of light and thus encodes color vision.
  • Rod: One of the receptor cells of the retina; sensitive to light of low intensity.
• Fovea: in the middle of the retina. This region mediates the most acute vision of birds and higher mammals.
  • Color-sensitive cones constitute the only type of photoreceptor found in the fovea.
• Optic disk: the location of the exit point from the retina of the fibers of the ganglion cells that form the optic nerve; responsible for the blind spot.

	Cones	Rods
Location	Most prevalent in the central retina (fovea); found in the fovea	Most prevalent in the peripheral retina; NOT found in fovea
Light Sensitivity	Sensitive to moderate-to high levels of light	Sensitive to low levels of light
Function	Provide information about the hue (color)	Provide only monochromatic information
Acuity	Provide excellent acuity	Provide poor acuity

Blind Spot

Foveal Versus Peripheral Acuity

• Ganglion cells in the fovea receive input from a smaller number of photoreceptors than in the periphery and hence provide more acute visual information.

Cones and Rods

The human retina contains approximately 120 million rods and 6 million cones.

• Although they are greatly outnumbered by rods, cones provide us with most of the visual information about our environment.
  • In particular, they are responsible for our daytime vision. They provide us with information about small features in the environment and thus are the source of vision of the highest sharpness, or acuity (from acus, “needle”).

The fovea , or central region of the retina, which mediates our most acute vision, contains only cones. Cones are also responsible for color vision—our ability to discriminate light of different wavelengths.

Although rods do not detect different colors and provide vision of poor acuity, they are more sensitive to light. In a very dimly lighted environment we use our rod vision; therefore, in very dim light we are color-blind and lack foveal vision.

Coding of Color: Photoreceptors: Trichromatic Coding

In 1802, Thomas Young, a British physicist and physician, proposed that the eye detected different colors because it contained three types of receptors, each sensitive to a single hue.

His theory was referred to as the trichromatic (three-color) theory

• Three different types of photoreceptors (three different types of cones) are responsible for color vision.
• Investigators have studied the absorption characteristics of individual photoreceptors, determining the amount of light of different wavelengths that is absorbed by the photopigments.
• These characteristics are controlled by the particular opsin a photoreceptor contains; different opsins absorb particular wavelengths more readily.

Absorbance of Light by Rods and Cones:

• The graph shows the relative absorbance of light of various wavelengths by rods and the three types of cones in the human retina.

Most people who are color blind only have two functioning types of cone cells, which is why they can only see around 10,000 shades – and almost all other mammals, including dogs and New World monkeys, are also dichromats.

Protanopia (red blindness)

• An inherited form of defective color vision in which red and green hues are confused; “red” cones are filled with “green” cone opsin.

Deuteranopia (green blindness)

• An inherited form of defective color vision in which red and green hues are confused; “green” cones are filled with “red” cone opsin.

Tritanopia (blue blindness)

• An inherited form of defective color vision in which hues with short wavelengths are confused; “blue” cones are either lacking or faulty.

Tetrachromat

• What is truly fascinating is that not everybody has just three variations of cone receptors, some have four and are tetrachromatic.

2. Details of Retinal Circuitry

• Bipolar cell: A bipolar neuron located in the middle layer of the retina, conveying information from the photoreceptors to the ganglion cells
• Ganglion cell: A neuron located in the retina that receives visual information from bipolar cells; its axons give rise to the optic nerve
• Horizontal cell: A neuron in the retina that interconnects adjacent photoreceptors and the outer processes of the bipolar cells
• Amacrine cell: A neuron in the retina that interconnects adjacent ganglion cells and the inner processes of the bipolar cells

Coding of Light and Dark

(1) Receptive field

Receptive field

• That portion of the visual field in which the presentation of visual stimuli will produce an alteration in the firing rate of a particular neuron.
  • neuron 可以被刺激的视野内的范围/区域
  • If a neuron receives information from photoreceptors located in the fovea, its receptive field will be at the fixation point—the point at which the eye is looking.
  • If the neuron receives information from photoreceptors located in the periphery of the retina, its receptive field will be located off to one side.
• A description of the effective stimuli of a given neuron. For sensory receptor neurons, the receptive field is the type of effective stimulation (e. g., light, sound, mechanical pressure) and the range of sensitive locations (e.g., center of visual field, left auditory field, tip of right thumb).
• part of the retina of which the photoreceptors (rods and cones) pertain to a single optic nerve fiber

Kuffler (1952, 1953), recording from ganglion cells in the retina of the cat, discovered that their receptive field consists of a roughly circular center, surrounded by a ring

• Stimulation of the center or surrounding fields had contrary effects: ON cells were excited by light falling in the central field (center) and were inhibited by light falling in surrounding field (surround), whereas OFF cells responded in opposite manner.

ON/OFF ganglion cells were briefly excited when light was turned on or off.

In primates these ON/OFF cells project to the superior colliculus, which is primarily involved in visual reflexes in response to moving or suddenly appearing stimuli (Schiller and Malpeli, 1977), which suggests that they do not play a direct role in form perception.

(2) ON and OFF Ganglion Cells

• The figure shows responses of ON and OFF ganglion cells to stimuli presented in the center or the surround of the receptive field.

(3) Retinal Ganglion Cells: Opponent-Process Coding

Retinal Ganglion Cells: Opponent-Process Coding

• At level of retinal ganglion cell three-color code gets translated into opponent-color system.

Daw (1968) and Gouras (1968)

• Found that these neurons respond specifically to pairs of primary colors: red versus green and yellow versus blue.
• Retina contains two kinds of color-sensitive ganglion cells: red-green cells and yellow-blue cells.
• Some color-sensitive ganglion cells respond in center-surround fashion.

For example, a cell might be excited by red and inhibited by green in the center of their receptive field while showing the opposite response in the surrounding ring.

• When a portion of the receptive field is illuminated with the color shown, the cell’s rate of firing increases. When a portion is illuminated with the complementary color, the cell’s rate of firing decreases.

3. Movement of Eyes

Saccadic movement

• The rapid, jerky movement of the eyes used in scanning a visual scene. Saccades are fast eye movements that bring the image of an object of interest onto the fovea

Pursuit movement

• The movement that the eyes make to maintain an image of a moving object on the fovea

三、Connections Between Eye and Brain

Dorsal lateral geniculate nucleus (LGN)

Dorsal lateral geniculate nucleus (LGN)

• A group of cell bodies within the lateral geniculate body of the thalamus; receives inputs from the retina and projects to the primary visual cortex.

• 1 4 6: contralateral eye

• 2 3 5: ipsilateral eye

• 1 2: magnocellular layer (inner two layers)（巨细胞层）

  • transmits information necessary for the perception of form, movement, depth, and small differences in brightness to the primary visual cortex.

• 3 4 5 6: parvocellular layer (outer four layers)（细小细胞层）

  • transmits information necessary for perception of color and fine details to the primary visual cortex.

• Ventral of 1,2,3,4,5,6: koniocellular layer (sublayers)

  • transmits information from short-wavelength (“blue”) cones to the primary visual cortex.

• Layers 1, 4, and 6 of the lateral geniculate nucleus receive input from the contralateral eye, and layers 2, 3, and 5 receive input from the ipsilateral eye. Layers 1 and 2 are the magnocellular layers; layers 3–6 are the parvocellular layers. The koniocellular sublayers（角质细胞亚层） are found ventral to each of the parvocellular and magnocellular layers

Properties of the Magnocellular, Parvocellular, and Koniocellular Divisions of the Visual System：

Property	Magnocellular Division	Parvocellular Division	Koniocellular Division
Color	No	Yes (from “red” and “green” cones)	Yes (from “blue” cones)
Sensitivity to contrast	High	Low	Low
Spatial resolution (ability to detect fine details)	Low	High	Low
Temporal resolution	Fast (transient response)	Slow (sustained response)	Slow (sustained response)

The Primary Visual Pathway

• The lens inverts the image of the world projected on the retina (and similarly reverses left and right).
• The optic nerves join together at the base of the brain to form the X-shaped optic chiasm (khiasma means “cross”).
  • There, axons from ganglion cells serving the inner halves of the retina (the nasal sides) cross through the chiasm and ascend to the dorsal lateral geniculate nucleus of the opposite side of the brain.
  • The axons from the outer halves of the retina (the temporal sides) remain on the same side of the brain.
• Each hemisphere receives information from the contralateral half (opposite side) of the visual scene.
  • It is not correct to say that each hemisphere receives visual information solely from the contralateral eye
• That is, if a person looks straight ahead, the right hemisphere receives information from the left half of the visual field, and the left hemisphere receives information from the right.

1. Anatomy of the Striate Cortex (Primary Visual Cortex, V1)

• This photomicrograph of a small section of striate cortex shows the six principal layers. The letter W refers to the white matter that underlies the visual cortex; beneath the white matter is layer VI of the striate cortex on the opposite side of the gyrus

The striate cortex consists of six principal layers (and several sublayers), arranged in bands parallel to the surface

These layers contain nuclei of cell bodies and dendritic trees that show up as bands of light or dark in sections of tissue that have been dyed with a cell-body stain.

Orientation of Movement

Most neurons in the striate cortex are sensitive to orientation

That is, if a line or edge (the border of a light and a dark region) is positioned in the cell’s receptive field and rotated around its center, the cell will respond best when the line is in a particular position—a particular orientation.

1. Simple cell: An orientation-sensitive neuron in the striate cortex whose receptive field is organized in an opponent fashion.
2. Complex cell: A neuron in the visual cortex that responds to the presence of a line segment with a particular orientation located within its receptive field, especially when the line moves perpendicularly to its orientation
3. Hypercomplex cell: A neuron in the visual cortex that responds to the presence of a line segment with particular orientation that ends at a particular point within the cell’s receptive field

Types of Orientation-Sensitive Neurons

The figure illustrates the response characteristics of three types of orientation-sensitive neurons in the primary visual cortex: (a) simple cell, (b) complex cell, and (c) hypercomplex cell.

Orientation Sensitivity

An orientation-sensitive neuron in the striate cortex will become active only when a line of a particular orientation appears within its receptive field. For example, the neuron depicted in this figure responds best to a bar that is vertically oriented.

2. Modular Organization of the Striate Cortex

Most investigators believe that the brain is organized in modules, which probably range in size from a hundred thousand to a few million neurons.

Each module receives information from other modules, performs some calculations, and then passes the results to other modules

• Color: Cytochrome oxidase (CO) blob
• The central region of a module of the primary visual cortex, revealed by a stain for cytochrome oxidase; contains wavelength-sensitive neurons; part of parvocellular system
• Neurons located within the blobs have a special function: Most of them are sensitive to color, and all of them are sensitive to low spatial frequencies but relatively insensitive to other visual features
• Outside the CO blob, neurons show sensitivity to orientation, movement, spatial frequency, and binocular disparity, but most do not respond to color

• Patterns of orientation columns and long-range horizontal connections in the primary visual cortex

Parallel Gratings

• Two kinds of gratings are compared: (a) square-wave grating and (b) sine-wave grating

Sine-wave grating

• A series of straight parallel bands varying continuously in brightness according to a sinewave function, along a line perpendicular to their lengths

Visual Angle and Spatial Frequency

• Angles are drawn between the sine waves, with the apex at the viewer’s eye. The visual angle between adjacent sine waves is smaller when the waves are closer together.

Spatial frequency

• The relative width of the bands in a sine-wave grating, measured in cycles per degree of visual angle

Perceive Depth

1. Monocular Depth Cues

To perceive depth, we rely on a combination of monocular cues(i.e., depth cues that only require one eye)and binocular cues(i.e, depth cues that require both eyes)

1. Relative Size（相对大小）: All else being equal, more distant objects look smaller
2. Texture Gradient（纹理渐变）: Object textures become less apparent farther away
3. Interposition/Occlusion（干涉，遮挡）: If Object A is blocking our view of Object B, this must mean that Object A is closer to us than Object B: also known as”occlusion“
4. Linear Perspective（线性透视，直线透视）: Parallel lines converge as distance increases, eventually meeting at a vanishing point
5. Hight in Plane（平面高度）: Distant objects tend to appear higher than closer objects
6. Light and Shadow（光与阴影）: Objects cast shadows that tell us their 3-dimensional shape

2. Binopolar Depth Cues

Binocular Disparity 双目视差

Because our two eyes are offset, they produce two different images our brains use the disparity between these images to calculate depth, with larger disparities meaning closer objects

The fact that points on objects located at different distances from the observer will fall on slightly different locations on the two retinas; provides the basis for stereopsis（立体视觉）.

Binocular Convergence 双目收敛

Looking at closer objects causes our eyes to converge our brains use this information to calculate depth with more convergence meaning closer objects

Summary for L4

Important Points in L4

1. Describe the characteristics of light and color, outline the anatomy of the eye and its connections with the brain, and describe the process of transduction of visual information.
2. Describe the coding of visual information by photoreceptors and ganglion cells in the retina.
3. Describe the striate cortex and discuss how its neurons respond to orientation, movement, spatial frequency, retinal disparity, and color.
4. Describe the anatomy of the visual association cortex and discuss the location and functions of the two streams of visual analysis that take place there.
5. Discuss the perception of color and the analysis of form by neurons in the ventral stream.
6. Describe the role of the visual association cortex in the perception of objects, faces, body parts, and places.
7. Describe the role of the visual association cortex in the perception of movement.
8. Describe the role of the visual association cortex in the perception of spatial location.

Retina :

• Photoreceptors (cones and rods), Coding of color (three types of cones),
• Fovea, Optic disk (blind spot), Ganglion cells (Coding of light and dark, on center, off center)

LGN :

• Connections Between Eye and Brain
• 6 layers, contralateral eye layer , ipsilateral eye layer, magnocellular layer, parvocellular layer, koniocellular layer
• Cells in LGN have center-surround receptive fields just like retinal ganglion cells

Striate Cortex (V1)

• Orientation selectivity, Simple cell and Complex cell, Functional columns

How do we perceive depth:

• Monocular cues (Size, Texture, Occlusion, Perspective, Height in the plane, Light and shadows) Binocular cues (Binocular disparity, Binocular convergence)

---

四、Analysis pf Visual Information

Role of the Visual Association Cortex

Extrastriate Cortex (Visual Association Cortex):

• Association cortex that surrounds the striate cortex. (In this context, extra-means “outside of.”)
• Although the striate cortex is necessary for visual perception, perception of objects and of the totality of the visual scene does not take place there. Each of the thousands of modules of the striate cortex sees only what is happening in one tiny part of the visual field. Thus, for us to perceive objects and entire visual scenes, the information from these individual modules must be combined. That combination takes place in the visual association cortex.

These views of a human brain show (a) a nearly normal lateral view, (b) a nearly normal midsagittal view, (c) an “inflated” lateral view, (d) an “inflated” midsagittal view, and (e) an unrolling of the cortical surface caudal to the dotted red line and green lines shown in (c) and (d).

Topographic map of somatosensory and motor cortices

The Homunculus map:

• The more sensitive the body region, the more area is dedicated to it in the sensory cortex and motor cortex

Retinotopic organization of the primary visual cortex

• Image showing the visual field, the retina, and the cortical retinotopic map, with numbered sections showing how retinal space is projected onto brain space.

If two objects occupy the same size of visual field (the same distant to the fixation point), they should active the same size of area in the primary visual cortex.

• However, Same object size $\neq$ same area in brain is activated $\rarr$ illusion

1. Illusions

An illusion is not simply an error of perception, but an exaggeration of a normal perceptual process, which makes illusions very useful in studying perception.

For example:

Feedback from visual association cortex to the V1

• A functional-imaging (fMRI) study By Murray, Boyaci, and Kersten (2006)

Spheres positioned against a background in locations that made them look closer to or farther from the observer. Although the spheres were actually the same size, their location on the background made the one that was apparently farther away look larger than the other one.

• Murray and his colleagues used fMRI to record activation of the striate cortex while the subjects looked at the spheres. They found that looking at the sphere that appeared to be larger activated a larger area of the striate cortex.
• We know that perception of apparent distance in a background cannot take place in the striate cortex, but requires neural circuitry found in the visual association cortex. This fact means that computations made in higher levels of the visual system can act back on the striate cortex and modify the activity taking place there.

High-Level Visual Processing

Almost 30 visual areas have been identified in the cerebral cortex of the macaque monkey (Felleman, D. J. and Van Essen, D. C., 1991; Van Essen, D. C. et al., 1992).

These areas comprise two main pathways for visual processing: the ventral pathway leads from the primary visual area (V1) to the IT cortex, while the dorsal pathway leads from V1 to the inferior parietal lobule .

Areas in the ventral pathway are thought to analyze what an object is (object vision pathway), while those of the dorsal pathway are thought to analyze where the object is and where it is going (spatial vision pathway).

1. Dorsal vs. Ventral stream

Dorsal stream

• System of interconnected regions of visual cortex involved in perception of spatial location, beginning with striate cortex and ending with posterior parietal cortex（后顶叶皮质）.
• Uses for ‘Where’ Stream
  1. Used for visually guided behavior
  2. Sensitive to motion, fast processing
  3. Not conscious processing
  4. Neurons respond based on where visual attention is allocated
• Detect
  1. Location
  2. Movement
  3. Spatial Transformations
  4. Spatial Relations

Posterior parietal cortex

• Highest level of dorsal stream of visual association cortex.
• Involved in perception of movement and spatial location

Ventral stream

• System of interconnected regions of visual cortex involved in perception of form, beginning with striate cortex and ending with inferior temporal cortex （颞下皮质）
• Uses for ‘What’ stream
  1. Used for recognition/identification
  2. Captures fine details but is slow
  3. Conscious awareness and interactions with long term memories
  4. Notices fine details (fovea)
  5. Neurons respond to objects anywhere in the visual field
• Detect:
  1. Color
  2. Texture
  3. Pictorial Detail 图片细节
  4. Shape
  5. Size

Inferior temporal cortex

• Highest level of ventral stream of visual association cortex.
• Involved in perception of objects, including people’s bodies and faces

Perception of Color

As we saw earlier, neurons within the CO blobs in striate cortex respond to colors. Like the ganglion cells in the retina (and the parvocellular and koniocellular neurons in the LGN), these neurons respond in opponent fashion

This information is analyzed by the regions of the visual association cortex that constitute the ventral stream.

(1) Illusions for Color Perception

Color Constancy:

The relatively constant appearance of the colors of objects viewed under varying lighting conditions.

The Same Color Illusion:

(2) Color Perception in Brain

Damage to area V4 does disrupt color constancy, but not the ability to discriminate between different colors

Conway, Moeller, and Tsao (2007) performed a detailed analysis of the responsiveness of neurons in a large region of the visual association cortex in monkeys, including area V4. Using fMRI, the investigators identified color “hot spots”—small scattered regions that were strongly activated by changes in the color of visual stimuli. Next, they used microelectrodes to record the response characteristics of neurons inside and outside these spots, which they called globs. They found that glob neurons were indeed responsive to colors and had only weak sensitivity to shapes. In contrast, interglob neurons (those located outside globs) did not respond to colors, but were strongly selective to shape

The fact that color-sensitive globs are spread across a wide area of visual association cortex probably explains the fact that only rather large brain lesions cause severe disruptions in perception of color

(3) Cerebral achromatopsia (Discriminate hues)

Inability to discriminate among different hues; caused by damage to area V8 of the visual association cortex.

Neurons in color-sensitive area V8 responded to photographs of objects in their natural colors (a) but not to those of objects in unnatural colors (b).

Perception of Form

The analysis of visual information that leads to the perception of form begins with neurons in the striate cortex that are sensitive to orientation and spatial frequency

These neurons send information to area V2 that is then relayed to the sub-regions of the visual association cortex that constitute the ventral stream.

In primates recognition of visual patterns and identification of particular objects take place in the inferior temporal cortex, located on the ventral part of the temporal lobe

This region of the visual association cortex is located at the end of the ventral stream.

• It is here that analyses of form and color are put together and perceptions of three dimensional objects and backgrounds are achieved.

Damage to this region causes severe deficits in visual discrimination (Mishkin 1966; Gross 1973; Dean 1976)

Sensation and Perception

Sensation: the process by our sensory receptors and nervous system receive and represent which stimulus energies from our environment.

Perception: the process of organizing and interpreting sensory information, which allows us to make sense of meaningful objects and events

---

Bottom-up processing: analysis that begins with the sensory receptors and works up to the brain’s integration of sensory information

Top-Down processing: information processing guided by higher-level mental processes, as when we construct perceptions, drawing on our experience and expectations

(1) Perception organization

the process of how we organize and interpret our sensations so that they become significant and meaningful.

• This Gestalt law of perceptual organization suggests that elements that are grouped together within the same region of space tend to be grouped together

(2) Gestalt psychology

A gestalt is a pattern, thing, form, shape, or object: something whole.

The Gestalt laws of grouping are a set of principles in psychology that explain how humans naturally perceive stimuli as organized patterns and objects.

The law of similarity

The law of similarity suggests that things similar things tend to appear grouped together. Grouping can occur in both visual and auditory stimuli. In the image above, for example, you probably see the groupings of colored circles as rows rather than just a collection of dots.

---

Law of Pragnanz (Law of good figure)

The word pragnanz is a German term meaning “good figure.” The law of Pragnanz is sometimes referred to as the law of good figure or the law of simplicity. This law holds that objects in the environment are seen in a way that makes them appear as simple as possible. You see the image above as overlapping circles rather than an assortment of curved, connected lines.

---

Law of Proximity

According to the law of proximity, things that are near each other seem to be grouped together. In the above image, the circles on the left appear to be part of one grouping while those on the right appear to be part of another. Because the objects are close to each other, we group them together.

---

Law of Continuity

The law of continuity holds that points that are connected by straight or curving lines are seen in a way that follows the smoothest path. Rather than seeing separate lines and angles, lines are seen as belonging together.

---

Law of Closure

According to the law of closure, things are grouped together if they seem to complete some entity.

Our brains often ignore contradictory information and fill in gaps in information. In the image above, you probably see the shapes of a circle and rectangle because your brain fills in the missing gaps in order to create a meaningful image.

Law of Common Region

This Gestalt law of perceptual organization suggests that elements that are grouped together within the same region of space tend to be grouped together

---

(3) “The Whole is Other than the Sum of the Parts”

When the perceptual system forms a percept or gestalt, the whole thing has a reality of its own, independent of the parts.

The Gestalt psychol-ogist Kurt Koffka made a famous statement about this: “The whole is other than the sum of its parts.

(4) Figure-Ground Perception in Psychology

Figure-ground perception refers to the tendency of the visual system to simplify a scene into the main object that we are looking at (the figure) and everything else that forms the background (or ground).

Figure-ground perception occurs when two contiguous regions share a border; one is often perceived to be an entity (i.e., an object or a figure) shaped by the shared bo rder, whereas the other (the ground) appears to simply continue behind the figure near th eir shared border.

How Do People Distinguish Between Figure and Ground?

1. Blurriness: Objects in the foreground tend to be crisp and distinct while those in the background are blurry or hazy.
2. Contrast: The high contrast between objects can lead to the perception of figure and ground. The Rubin vase is one example.
3. Size: Images that appear to be larger will be perceived as closer and part of the figure while those that are smaller will seem further away and part of the background.
4. Separation: An object isolated from everything else in a visual scene is more likely to be seen as a figure versus background

Examples:

The Rubin Goblet illustrated a basic concept from Gestalt psychology: the figure-ground distinction. When a gestalt is formed (perceived) it becomes a figure (a thing apart, an entity or object).

Rubin’s Goblet: Rubin’s goblet is a popular optical illusion used to illustrate differences in perception of stimuli.

Neural Bases of Form detection

• V1 cells like Edges
• V2 cells exhibit responses to cognitive contours (also termed ‘illusory contours’)
• V4 cells tuned for complex shape

• IT (inferior temporal cortex) cells tuned for 3D shape
  • Viewpoint dependent

2. Inferior temporal (IT) cortex

The IT cortex is situated at the end of the ventral visual pathway. Via this pathway, retinal images are processed in areas V1, V2, and V4, after which the analyzed information is sent from V4 to the IT cortex

The IT cortex, in turn, projects to the limbic systems, such as the amygdaloid nuclei and the hippocampus via the entorhinal cortex; there also are direct routes to the hippocampus. The IT cortex also projects to the ventrolateral and orbital prefrontal cortices. Thus, the anatomical position of the IT cortex can be thought of as an interface between the visual cortices and the limbic systems and frontal lobe

Effects of Inferior Temporal Damage on Object Recognition

1. Patients with associative visual agnosia (关联性视觉失认) have difficulty recognizing objects (deficits in visual form perception) in the absence of blindness

2. Prosopagnosia is the inability to recognize faces, despite intact intellectual ability and even apparently intact visual recognition of most other stimuli. When looking in a mirror, these patients even misjudge themselves to be an unfamiliar person staring at them.

   • Studies with brain-damaged people and functional imaging studies suggest that these special face-recognizing circuits are found in the fusiform face area (FFA) , located in the fusiform gyrus on the base of the temporal lobe
   • Fusiform face area (FFA): Region of the visual association cortex located in the inferior temporal; involved in perception of faces.

Consistent with that idea are the findings of functional magnetic resonance imaging (fMRI) studies, which revealed that different regions of the human occipitotemporal cortex are activated specifically in response to faces (fusiform face area, FFA), places (parahippocampal place area, ), or body parts (extrastriate body area, EBA)

Fusiform face area (FFA)

• Region of the visual association cortex located in the inferior temporal; involved in perception of faces

Extrastriate body area (EBA)

• A region of the visual association cortex located in the lateral occipitotemporal cortex; involved in perception of the human body and body parts other than faces.

Parahippocampal place area (PPA)

• A region of the medial temporal cortex; involved in perception of particular places (“scenes”).

Responses to Categories of Visual Stimuli :

These functional MRI scans are of people looking at six categories of visual stimuli. Neural activity is shown on “inflated” ventral views of the cerebral cortex. The fusiform face area is shown as a black outline, derived from the responses to faces shown in the upper left scan

Perception of Faces and Bodies :

• The fusiform face area (FFA) and the extrastriate body area (EBA) were activated by images of faces, headless bodies, body parts, and assorted objects.

3. Perception of Movement

We need to know not only what things are, but also where they are and if they are moving, where they are going

Without ability to perceive the direction and velocity of movement of objects, we would have no way to predict where they will be. We would be unable to catch them (or avoid letting them catch us)

One of the regions of a monkey’s extrastriate cortex—**area V5**, also known as area MT, for medial temporal—contains neurons that respond to movement.

A region adjacent to area V5 (sometimes called V5a but more often referred to as MST, for medial superior temporal) receives information about movement from V5 and performs a further analysis. MST neurons respond to complex patterns of movement, including radial, circular, and spiral motion.

Location of Visual Area V5

• LOS = lateral occipital sulcus,
• IOS = inferior occipital sulcus.

The Posterior Parietal Cortex

• An “inflated” dorsal view of the left hemisphere of a human brain shows the anatomy of the posterior parietal cortex.

Livingstone and Hubel considered the directionally selective cells of V1 to be the basic units of motion perception.

Zeki was the first to describe that V5/MT (middle temporal) area neurons are especially sensitive to motion. Many cells responded vividly to stimuli (bright spots or dark/bright lines) moving in a given direction, while they failed to respond to movement in the opposite direction. The shape of the stimulus did not seem to matter, until the direction was optimal.

Eifuku and Wurtz pointed out that the MST(medial superior temporal) area in macaque monkey consists of a dorsomedial (MSTd) and a lateroventral (MSTl) part. According to the authors, MSTd processes optic flow information as in, information on the own motion of the observer, while MSTl is specialized for external motion.

Cells in the parietal lobe are sensitive to motion too, though these cells seem to foster movement regulation, rather than perception

Intraparietal sulcus (IPS)

• The end of the dorsal stream of the visual association cortex; involved in perception of location, visual attention, and control of eye and hand movements.
• In the intraparietal sulcus, five visual areas have been described, based on morphological and physiological criteria.

1. Lateral intraparietal (LIP): visual attention and control of saccadic eye movements.
2. Ventral intraparietal (VIP): visual attention and control of saccadic eye movements, visual control of reaching and pointing
3. Medial intraparietal (MIP): represents extra-personal space, visual control of reaching and pointing
4. Anterior intraparietal (AIP): responsible for the visual guidance of hand movements and grabbing
5. Caudal intraparietal (CIP): perception of depth from stereopsis

Optic flow

• The complex motion of points in the visual field caused by relative movement between the observer and environment; provides information about the relative distance of objects from the observer and of the relative direction of movement
• Analysis of the relative movement of the visual elements of your environment—the optic flow—will tell you where you are heading, how fast you are approaching different items in front of you, and whether you will pass to the left or right (or under or over) these items.
• Britten and van Wezel (1998) found that electrical stimulation of MSTd disrupted monkeys’ ability to perceive the apparent direction in which they were heading; thus, these neurons do indeed seem to play an essential role in heading estimations

Movement of Eyes

Saccadic movement

• The rapid, jerky movement of the eyes used in scanning a visual scene. Saccades are fast eye movements that bring the image of an object of interest onto the fovea

Pursuit movement

• The movement that the eyes make to maintain an image of a moving object on the fovea.

Form From Motion

• Perception of movement can even help us to perceive three-dimensional forms—a phenomenon known as form from motion
• A functional imaging study by Grossman et al. (2000) found that when people viewed a video that showed form from motion, a small region on the ventral bank of the posterior end of the superior temporal sulcus became active
• People with visual agnosia can often still perceive actions (such as someone pretending to stir something in a bowl or deal out some playing cards) even though they cannot recognize objects by sight. They may be able to recognize friends by the way they walk, even though they cannot recognize their faces.

Fraser-Wilcox or peripheral drift illusion:

• Fraser-Wilcox or peripheral drift illusion, fools your brain into thinking the circles are rotating even though you obviously are looking at a printed page. This is because the MT interprets the rapid high-contrast alternations between the yellow and blue, which are processed by the same ganglion cells, as being in motion. Finally, the cortex is responsible for combining features from the two processing streams into a singular experience.

4. Perception of Space Location

The parietal lobe is involved in spatial and somatosensory perception, and it receives visual, auditory, somatosensory, and vestibular information to perform these tasks. Damage to the parietal lobes disrupts performance on a variety of tasks that require perceiving and remembering the locations of objects and controlling movements of the eyes and the limbs

Relationship between spatial location and movement

Goodale and his colleagues suggested that the primary function of the dorsal stream of the visual cortex is to guide actions rather than simply to perceive spatial locations.

They cited the case of a woman with bilateral lesions of the posterior parietal cortex who had trouble picking up objects. The patient could easily perceive the difference in size of wooden blocks that were set out before her, but she failed to adjust the distance between her thumb and forefinger to the size of the block she was about to pick up

In contrast, a patient with profound visual agnosia caused by damage to the ventral stream could not distinguish between wooden blocks of different sizes but could adjust the distance between her thumb and forefinger when she picked them up.

5. Communication between the dorsal and ventral streams

If the primary role of the dorsal stream is to direct movement, it must be involved in the location of these objects, or else how could it direct movements toward them? (in fact, the dorsal stream is involved in perception of the location of an object’s space).

In addition, it must contain information about the size and shape of objects, or else how could it control the distance between the thumb and forefinger?

A fascinating (and delightful) study with young children demonstrates the importance of communication between the dorsal and ventral streams of the visual system (DeLoache, Uttal, and Rosengren, 2004).

The experimenters let children play with large toys: an indoor slide that they could climb and slide down, a chair that they could sit on, and a toy car that they could enter. After the children played in and on the large toys, the children were brought out of the room, the large toys were replaced with identical miniature versions, and the children were then brought back into the room. When the children played with the miniature toys, they acted as if they were the large versions: They tried to climb onto the slide, climb into the car, and sit on the chair.

A 2-year-old child trying to climb into the toy car. He says “In!” several times, and calls for his mother, apparently asking her to help him. The authors suggest that this behavior reflects incomplete maturation of connections between the dorsal and ventral streams. The ventral stream recognizes the identity of the objects and the dorsal stream recognizes their size, but the information is not adequately shared between these two systems.

---

Summary

Two stream of extrastriate Cortex (visual association cortex): high level visual processing

• Ventral stream (what stream): deal with object (color, form), inferior temporal cortex
• Dorsal stream (where stream): deal with motion (space, movement), posterior parietal cortex.

---

Gestalt Laws of Perceptual Organization $\rarr$ Face recognition

Region of human visual cortex	Name of the region	Function
V1	Striate Cortex	Small modules that analyze orientation, movement, spatial frequency, retinal disparity, and color
V2		Further analysis of information from V1
V3 and VIP		Further analysis of information from V2
V3A		Processing of visual information across the entire visual field of the contralateral eye
V4d/V4v	V4 dorsal/ventral	Analysis of form Processing of color constancy V4v = upper visual field, V4d = lower visual field
V8		Color perception
LO	Lateral occipital complex	Object recognition
FFA	Fusiform face area	Face recognition, object recognition by experts (“flexible fusiform area”)
Ventral Stream PPA	Parahippocampal place area	Recognition of particular places and scenes
EBA	Extrastriate body area	Perception of body parts other than face

---

Superior Colliculus: a shortcut in the visual system detecting motion

Region of Human Visual Cortex	Name of the Region	Function
V7		Visual attention Control of eye movements
V5 (Also called MT/MST or MT+)	Medial temporal/medial Superior temporal (named for locations in monkey brain)	Perception of motion Perception of biological motion and optic flow in specific subregions
LIP	Lateral Intraparietal area Visual attention C	Visual attention Control of saccadic eye movements
VIP	Ventral intraparietal area	Control of visual attention to particular locations Control of eye movements Visual control of pointing
AIP	Anterior intraparietal area	Visual control of hand movements: grasping, manipulation
MIP	Middle intraparietal area Parietal reach region (monkeys)	Visual control of reaching
CIP	Caudal intraparietal area Caudal parietal disparity region	Perception of depth from stereopsis