operation regime driven by the environment (e.g. after prolonged
exposure to specific moving patterns). When this enviroment changes
(e.g. when the motion stops) new stimuli (e.g. the static patterns) are
perceived in unusual way (e.g. illusory motion in reverse direction).
Movies below, generated with this
Matlab code, show new
and texture after-effects.
code allows you to modify the parameters of the examples (speed,
spatial frequency, contrast, luminance and tristimulus values...).
look at the fixation point (a cross in the center) and keep looking at
the cross after the change in the scene. The change will happen about
10 to 15 secs after starting the animation so that your visual system
allows an accurate stimuli control, and these illustrations are good to
point out the general trends of the effects, they are not meant
as monocular fixation,
of the images (or observation
(e.g. full screen versus
embedded reproduction), calibration
(your display is not calibrated according to the parameters used in the
generation), and even fixating
point, may help or impair the strength of the
Examples of STATIC MOTION AFTEREFFECT:
Example 1 (vertical motion, noise sequence) shows that
moving pattern generates a stronger reverse motion aftereffect.
The aftereffect comes from the fact that the sensors tuned to zero and
slow positive motion are attenuated by the adapting sequence.
Therefore, a static pattern presented afterwards (that stimulates slow
negative, zero and slow positive sensors) elicits a smaller-than-usual
response in zero and slow positive sensors. As a result, the stimulus
is seen as moving with slow negative motion. Attenuation in sensors
tuned to zero motion comes from the fact that
activity in certain sensors reduces the gain of other sensors [Morgan
et al. 06, Stocker and Simoncelli 09].
stronger aftereffect produced by the slow motion pattern reveals that
sensors tuned to slow motion attenuate more strongly the sensors tuned
to static patterns. The activity of sensors tuned to higher speeds does
not affect zero motion sensors as much leading to weaker aftereffect. The
where this neighborhood interaction comes from?. Why
slow motion sensors attenuate the zero motion sensor more than the
sensors tuned to high speed?
Example 1 shows
central part of the test image is achromatic (and not limited by any
contour), color and contours of the scene
change after adaptation: an illusory cyan/blueish patch emerges around
the fixation point.
This aftereffect arises because adaptation to a yellowish color leads
to shifts in the nonlinear responses of the Red-Green and Yellow-Blue
sensors [Krauskopf and Gegenfurtner 92, Fairchild 05]. After this
adaptation, stimuli that would be considered to be white in usual
unshifted curves) is perceived as greenish-blue. Equivalent shifts (but
in different chromatic directions) happen after adaptation to different
why the nonlinearities shift in this specific way?
differently after adaptation to the high contrast texture. Note that
the strength of the induced blindness around the fixation point depends
on the correspondence between frequency and orientation of the previous
and post scenes: if the frequency content is very similar the affected
zone virtually disappears. That is not the case in other situations.
This aftereffect arises because sensors tuned to certain frequency are
attenuated by the activity of sensors tuned to similar frequencies [Blakemore
90, Watson 97]. As a result, after the presentation of a high contrast
pattern, if a similar pattern is presented, the response of the sensors
tuned to this second pattern is weaker than usual (under no
adaptation). Therefore, this second pattern is seen as having less
contrast, or eventually, not seen at all. The
stronger aftereffect produced between patterns of similar frequency
sensors tuned to closer frequencies interact (attenuate each other)
more than sensors tuned to distant frequencies. The
question (as in the motion case) is:
where this local neighborhood interaction comes from?.
2 shows that
trends happen for the linear Principal Components of natural images:
even though they are statistically decorrelated, they are not
perceptually independent. Some additional processing is necessary to
make them perceptually independent.
do we have these
illusions?: Better use normative models based on
unsupervised nonparametric learning!
responses in the mechanisms tuned to motion [Morgan 06], texture
07]. For example, this
reproduces how divisive normalization [Simoncelli and Heeger 98,
Carandini and Heeger 12] leads to mutual attenuation of motion sensors
tuned to neighbor frequencies, and hence to misperceptions of motion of
static objects after adaptation to moving patterns (see section 1.2 of
empirical description does not explain why
the brain works in this
apparently dysfunctional manner. That is the classical limitation of descriptive models versus normative models (what versus why) [Dayan05].
maximization (a truly normative explanation),
multidimensional equalization has been proposed as a
convenient way to address adaptation and aftereffects
[Clifford00,Clifford02]. Unfortunately they did not propose
unsupervised techniques, but assumed specific parametric mechanisms
based on centering and scaling (a sort of divisive normalization).
using specific functional forms for the adaptation, in
this work we derive the behavior from a recently proposed
unsupervised non-parametric learning technique: The Sequential
Principal Curves Analysis (SPCA) [Laparra et al. 12]. SPCA effectively
performs the multidimensional equalization previously
suggested, but not implemented, in the statistically inspired
literature addressing aftereffects [Clifford00, Clifford02]. We
unsupervised learning is the appropriate way to focus on
the principle behind the aftereffects:
more clear that the behavior emerges from the specific optimization
strategies and not from an a priori
response model. Moreover, unlike other unsupervised learning
techniques, SPCA is more suited to answer the goal question because
it can be easily tuned to different principles such as
information maximization (as non-linear ICA) and also error
limited resolution scenarios (as in optimal Vector
Quantization). See Section 4 in
to see the equalization capabilities of SPCA on visual
textures. The illustrations below show how equalization leads to
attenuations and shifts in the responses that induce the aftereffects.
identifies relevant directions (curves) is datasets. SPCA is a
generalization of Principal Component Analysis (PCA) by allowing the
straight lines of PCA to be changed by non-Euclidean curves. As the
principal directions in PCA, the principal curves in SPCA can be
interpreted as sensors. Given a stimulus, the responses of SPCA sensors
are given by the projection of the stimulus onto the curves. The full
description on how these curves and projections is computed can be
found in this Technical Report.
Given the fact that length along the principal curves is measured using
a PDF dependent non-Euclidean metric, this kind of projection performs
a nonlinear multidimensional equalization.
Identification of curvilinear coordinates and multidimensional histogram equalization using SPCA. Left
panel represents the data and coordinates at the input domain, and the
right panel represents the data and coordinates at the SPCA transformed
domain. Gray regions represent the underlying data distribution or PDF,
and the lines in bold style represent the projections of the
highlighted sample x onto
the first, second, and third Principal Curves. Using SPCA with
information maximization criterion this implies histogram
uniformization (see section 4 of the Paper for more details and the
Technical Report for full details). This means transforming the
nonuniform curvilinear coordinates into a cartesian grid.
of the PDF corresponding to the natural stimuli leads to expansions and
contractions of the input domain in the response domain. This means
uneven sensitivity of the SPCA sensors in different regions of the input space. Aftereffects come from changes in the operation
regime of a sensor in different environments. The pictures below
illustrate how different spatial texture in the background or different
spectral illumination change the shape of the PDF and (assuming the
equalization goal) change the operation regime.
ADAPTATION OF TEXTURE SENSORS I:Different
directions in the image space represent the presence of certain
elementary patterns of bigger and bigger amplitude (or contrast).
Bigger and bigger stimulation of a linear sensor tuned to certain
pattern implies moving along a certain line. If the sensor is nonlinear
this means moving in nonuniform steps or along a curve. If this
stimulation is done in different environments (i.e. on top of different
backgrounds), it can be seen as equivalent departures from different
points of the space. Different points of the space can also be seen as
different stimulation in sensors tuned to similar or disimilar
patterns. The figure below illustrates this concept: the same
stimulation along the green curve/dimension at different locations
along the red curve (different environments). In this case, the
frequency of the background is similar to the frequency preferred
by the considered sensor (severe masking situation). Different
environments defined as different locations in the linear
representation space is the key concept in the motion and texture
experiments (figures 4 and 7 of the Paper).
ADAPTATION OF TEXTURE SENSORS II: Equalization
of the image manifold implies different deformations in different
points of the space. This implies that responses to the same Euclidean
departures in different environments lead to different lengths along
the principal curves (the responses of SPCA sensors). (a) equivalent
stimulations in different backgrounds in the input domain, (b) SPCA
transform. (c) Associated responses (length along the green principal
curve in each case). This leads to a reduction
in response/visibility in environment 2 with regard to environment
1. Figure 12 in the Paper shows that transform from (a) to (b) is the actual behavior of SPCA on texture samples.
ADAPTATION OF COLOR SENSORS I: Different
spectral illumination implies a general change of orientation of the
data manifold. This change is not always linear [Laparra12]. Similar
changes in the manifold arise when reflectance of the environment is
biased in some way (e.g. mainly yellowish/reddish objects). In the
example below the blue and orange regions represent the color data
manifolds in D65 and A illuminations which basically differ in a
rotation around the origin towards the yellow end of the Blue-Yellow
(BY) direction (shift along the blue curve). What is the perceived hue
a fixed sample (e.g. the sample represented by the open circle) in
these different environments?
ADAPTATION OF COLOR SENSORS II: Equalization
of the color
manifold implies (i) different deformations in different points of the
space, and (ii) shifts of what is considered to be the origin of the
space. This implies (i) nonlinear responses, and (ii) the response to
certain stimulus is interpreted in different way. For instance
responses of an opponent mechanism may change from positive to negative
hence totally changing the judgement of hue. (a) color data in the
domain. (b) and (c) SPCA transformed data equalizing the environment 1
or environment 2 manifolds. Note that the highlighted sample (open
circle) falls on opposite regions of the space. (d) canonical
representation. (e) Associated response in the considered
to samples from natural videos (3D luminance patches) texture samples
(2D luminance patches) and color samples (tristimulus vectors) after
they went through the first linear stage of the models of motion,
texture, and color sensors (see [Simoncelli
98] for motion, [Watson 97] for texture, and [Abrams 07, Fairchild 05]
for color). Natural movies come from undistorted videos of the standard
databases VQEG and LIVE. Natural textures come from the luminance
calibrated McGill image database, and natural colors come from the
color calibrated IPL image database used in previous spatio-chromatic
adaptation papers [Laparra et al. 12, Gutmann et al. 14].
In order to see how motion and texture aftereffects come from signal
statistics we computed the response of
SPCA sensors tuned to specific spatio-temporal frequency patterns in
different environments, i.e. when similar or different sensors have
different degrees of activation because additional adapting patterns
are present in the scene (biased contrast). In the color
case, we computed the response of SPCA sensors with red-green and
yellow-blue sensitivities in environments of different chromatic nature
(balanced versus biased reflectance set).
In all three cases, different operation regimes were found as a
function of the environment (see the different line-styles
in the figure below): therefore the reference axes of the perceptual
repreentatipon shift!. After-effects arise
when stimuli having certain response in the regular regime, have very
different (shifted) response in the adapted regime.
nonlinear responses leading to the MOTION AFTEREFFECT
sensors tuned to ft = 0 (static pattern) in environments where
other moving patterns are present. Top
and 2nd rows): responses with adaptation to vertical
speeds. First row displays the response of SPCA tuned for infomax while
second row shows the equivalent results for SPCA sensors with minimum
representation error. In every row, the vertical speed of the adapting
stimulus increases from left to right. Different line styles correspond
to the responses in different environments: the solid black line
correspond to responses to isolated static stimuli, and the other line
styles correspond to responses of the same sensor in environments where
there is also a moving pattern of progressively bigger contrast: from
the dashed-black line (moving mask with small contrast), to the
dashed-red line (moving mask with bigger contrast). The curves are the
average over equivalent stimulation (phase and speed sign). Error bars
indicate standard deviation. Bottom
panel (3rd and 4th rows): equivalent responses with
to horizontal speeds.
This kind of high attenuation for close frequencies and weak impact of
distant frequencies (that comes from the statistics!) is compatible
with the reported nonlinearities of motion sensors [Morgan06] and
question raised by the motion
nonlinear responses leading to the TEXTURE AFTEREFFECT
(adaptation to different high contrast patterns) lead to different
operation regimes of
the same texture sensor which are compatible with the frequency and
orientation properties of post-adaptation masking.
Figures show the
response of a sensor tuned to a particular frequency
as a function of the contrast of stimuli of that frequency. Each plot
represents the adaptation to a different environment (with similar or
frequencies). Solid black
lines correspond to the responses of the considered sensor when the
other sensors are not stimulated (in zero contrast adaptation
conditions). Progressively lighter lines represent the responses in the
regime induced by the adaptation to progresively higher contrast of
the adapting stimuli.
When adapting to the stimuli described above (high contrast in the
center and zero contrast in the surround), the sensors in the center
work according to the red lines while the ones in the periphery work
according to the black-solid line. Therefore, when looking at the
uniform image the central part of the image seems to disappear (in the
first case when frequencies are similar), because the response, or
visibility, is strongly reduced, which explains the
question in the
nonlinear responses leading to the COLOR AFTEREFFECT
(e.g. adaptation to an environment withrestricted
to shifts in the chromatic responses (from gray to colored curves in
red-green -left- and yellow-blue -center- channels) that explain theshift
crossings in the gray curves) tend to cyanish hue after adaptation (red
arrows). In both
channels the achromatic stimuli lead to negative values, i.e. green and
also that the locus of stimuli corresponding to a balanced range of huesand
in the region of the stimuli that correspond to green-blue, which
question raised by the color
NOTE 1: The corresponding colors result (transformation of Luo's
corresponding colors from CIE A to the artificial world we crated with
a subset of reflectances) is not in the paper but it is an equivalent
evidence of the SPCA explanation.
NOTE 2: These infomax results (which also reproduce the aftereffect)
are not shown in the paper since in the color case it is obvious that
the infomax solutions give rise to way too sharp nonlinearities (see
[Laparra et al. 12]).
the responses and the change of operation
regime come from the way samples
are distributed. In the case of motion and texture, the general
from (1) the increased concentration of samples around zero contrast,
and (2) the faster decay in the density in some directions in
the neighborhood of similar stimuli. In the case of color, the general
behavior comes from (1) peak density at achromatic reflectance, and (2)
deformation of the tristimulus manifold when considering clustered hues
or specific illumination. The specific
organization principle implies different saturation nonlinearity in the
responses. Both criteria lead to shifts and changes of operation regime
compatible with the observed aftereffect in all modalities (motion,
texture and color). In the case of color, where the theoretical
responses were closely compared to the actual responses in [Laparra
12], infomax can be
neglected because of the unrealistic sharpness of the response.
It is important to note that these trends were not imposed by using
some specific functional form in the response, but emerges from the way
data distribution changes in different environments and by optimal
encoding strategies of the data.
can conclude that what
appears to be a dysfunctional behavior
actually comes from clever information processing: the system uses the
operation regime in each environment. Your visual brain is not fooling
you, but trying to do its best in a dynamic environment!.
on illusions and the companion technical report on SPCA:
V. Laparra, and J. Malo,
"Visual Aftereffects and Sensory Nonlinearities from a Single
Statistical Framework" Frontiers in Human Neuroscience.Special
Issue on Perceptual Illusions. 2015
V. Laparra, and J. Malo,
"The full report on Sequential Principal Curves
Analysis" IPL-Technical Report. Universitat
de València. 2015
J. Malo and
J. Gutiérrez V1 non-linear properties emerge
from local-to-global non-linear ICA.Network: Comp. Neural Systems.
17(1): 85-102 (2006)
Laparra, S. Jimenez, G.
and Adaptation of Color Vision from Sequential Principal Curves Analysis,
on related statistical techniques:
V. Laparra, S. Jimenez, D. Tuia,
G. Camps-Valls and J. MaloPrincipal
Polynomial Analysis (PPA). International Journal of Neural
Systems, 24(7) Nov. 2014.
V. Laparra, J. Malo and G.
Reduction via Regression in Hyperspectral Imagery. IEEE Journal on Selected Topics
of Signal Processing. Vol. 9, Num. 9. September 2015.
Hillis, J., and Brainard, D. "The relation between color discrimination
and color constancy: When is optimal adaptation task dependent?" Neural
Computation, 19(10):2610–2637. 2007
"Possible principles underlaying the transformation of sensory
messages". Sensory Communication,
Barlow, H. “A
theory about the functional role and synaptic mechanism
of visual aftereffects,” in Vision: Coding and Efficiency, C.
Blakemore, Ed. Cambridge, UK: Cambridge
"Redundancy reduction revisited". Network:
and Campbell F.W. "On the existence of neurons in the human visual
system selectivity sensitive to the orientation and size of retinal
images". J. Physiol.
and Heeger D., “Normalization as a canonical neural
Neurosci., vol. 13, no. 1, pp. 51–62,
Webster, M., Stanley, G.,
adaptation: Neural, psychological and
computational aspects,” Vision
Wenderoth, P., and
Spehar, B. (2000), A functional angle on some after-effects in cortical
vision, Proc. Roy. Soc.
R., Dayan, P., and Schwartz, O. (2010), Statistical models of linear
and nonlinear contextual interactions in early visual processing, in Adv. Neur. Inf. Proc. Syst. NIPS 09'. vol. 22, 369–377
Color Appearance Models, 2nd Ed. Chichester, UK:
original sources of the natural scene data used to train the algorithm.
Patches of natural movies were necessary as training set to optimize
the response of motion sensors to derive the motion aftereffect .
We used the achromatic channel of undistorted -raw- videos from
standard databases on subjective video quality):
Patches of natural images were used as training set to (1) optimize the
response of texture sensors to derive the texture aftereffect, (2)
design image representations for nonlinear ICA or optimal transform
coding, and (3) check the convergence of SPCA transform in texture data
-technicality not shown in the paper-. We used the luminance channel of
the colorimetrically calibrated McGill database:
Natural Colors Natural
response of color sensors to derive the color aftereffect, and (2)
convergence of SPCA transform in color data -technicality not shown in
the paper-. We
Image Processing Lab (IPL) database:
data subsets from the above sources (when preprocessing is
straighforward), or includes routines to read the raw data and
rearrange it as required in each experiment. You may
download all the required data in a single data file:
This is the
reproduce all the results in the paper "Visual Aftereffects from biased
Statistics". Before running a specific experiment, make
downloaded the required data and the general purpose toolbox (if
required). You can try the all-in-one
version (yellow box).