Modeling Combined Proximity-Similarity Effects in Visual Search Tamar Avraham* Yaffa Yeshurun** Michael Lindenbaum* *Computer science dept. , Technion, Israel **Psychology dept. , Haifa University, Israel
Outline ► The FLNN and COVER models for computer vision Avraham & Lindenbaum, IEEE-PAMI 2006 ► Study 1: Adapting the models for human performance Avraham, Yeshurun & Lindenbaum , Journal of Vision 2008 ► Study 2: Extending the models to account for spatial organization effects.
The FLNN model Farthest Labeled Nearest Neighbor feature extraction feature space T D D 0° 30° 60° orientation T = 0° D = 30°, 60° Avraham & Lindenbaum, IEEE-PAMI 2006
The FLNN model – cont. 2 4 1 3 Alternative parallel explanation: dynamic priority map Avraham & Lindenbaum, IEEE-PAMI 2006
Qualitative model behavior ► Homogeneous distractors – pop-out behavior 1 2 D T ► Clustered distractors – maximum one from each cluster D 2 D 1 T D 3 Avraham & Lindenbaum, IEEE-PAMI 2006
Visual Search Difficulty ► Search difficulty depends on two factors: § T-D similarities § D-D similarities Duncan & Humphreys 89 similarity theory D T easy ► Quantitative T difficult measures of search difficulty § the saliency measure (Rosenholtz 99) § our COVER measure Avraham & Lindenbaum, IEEE-PAMI 2006
The COVER measure ►A minimum-d-cover (Kolmogorov 61): the minimum number of spheres with diameter d covering all points d Avraham & Lindenbaum, IEEE-PAMI 2006
The COVER measure ► If d = d. T (the minimum T-D distance), minimum-d-cover (COVER) the search difficulty D d. T T COVER = 1 D D d. T T D COVER = 3 D T COVER = 2 T-D similarity effects D-D grouping Avraham & Lindenbaum, IEEE-PAMI 2006
COVER and FLNN COVER = an inherent limitation for all models/algorithms FLNN Performance ≥ COVER Avraham & Lindenbaum, IEEE-PAMI 2006
Study 1: Testing the ability of COVER and FLNN to predict human performance Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
► COVER with internal noise: Low noise T Higher noise T d. T ► Other visual search models § Temporal-serial (Bergen&Juletz 83’) § Signal-Detection-Theory (Palmer et. al. 93’, Eckstein 00’) § Target-Saliency model (Rosenholtz 99’) § Best-Normal (Rosenholtz 01’) § RCref (Rosenholtz 01’) Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
Study 1 Manipulated T-D and D-D similarity Accuracy. 2 IFC. Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
COVER: Prediction Comparison correlation between accuracy and measure Saliency COVER A. P. 0. 812 0. 999* Y. B. 0. 538 0. 998* D. A. 0. 572 0. 997* V. S. 0. 570 0. 999* A. P. Z. 0. 510 1* A. D. - 0. 926* A. A. - 0. 903* M. D. - 0. 962* L. F. - 0. 873 D. A. 0. 900 0. 862 S. M. 0. 945 0. 883 E. D. 0. 996* 0. 993* G. S. 0. 880 0. 997* R. A. - 0. 967* O. R. - 0. 956* R. I. - 0. 979* A. O. - 0. 951* Participant (Rosenholtz 99) Exp 1 Exp 2 Exp 3 Exp 4 Good correlation between accuracy and COVER Avraham, Yeshurun & Lindenbaum, Journal of Vision 2008
FLNN - Prediction Comparison FLNN best in 2 test. Lowest 2/df values Avraham, Yeshurun & Lindenbaum, Lindenbaum VSS 2011 Avraham, Yeshurun, Journal of Vision 2008
Study 1 Summary ► FLNN and COVER predict T-D and D-D similarity effects better than other prominent computational models. ► The models quantify grouping-by-similarity involved in visual search, by suggesting that the degree of withingroup heterogeneity depends on the T-D similarity. Avraham, Yeshurun, Lindenbaum VSS 2011
Study 2: Spatial Organization Effects test and model how the effects of grouping by proximity and grouping by similarity are combined in the context of visual search
Manipulating Spatial Organization Experiment 1 T=0° D = 23°, 47°, 70° The same 30 elements in all conditions condition 1 condition 2 condition 3 no clusters 6 clusters 3 clusters Does spatial organization matter? Avraham, Yeshurun, Lindenbaum VSS 2011
Experiment 1: Results ► Previous models do not account for this significance Avraham, Yeshurun, Lindenbaum VSS 2011
Spatial Organization Effects - Modeling ► One possibility: a multi scale approach (e. g. , Itti et al. 1998, Rosenholtz et al. 2007) no clusters 3 clusters COVER=2 COVER=3 How to combine the measure over scales? max? weight and sum? Avraham, Yeshurun, Lindenbaum VSS 2011
Spatial Organization Effects – Modeling ► Our models need only some measure of distance between each two elements ► Combine feature difference and spatial distance Indicates the relative effect of the forces Avraham, Yeshurun, Lindenbaum VSS 2011
Spatial Organization Effects – Modeling ► Advantages: § same treatment for similarity and proximity § understand quantify the relative effect of each ► Questions to answer in this study: § Will this enable our models to account for the spatial organization effect? § What is the value of ? § Is stable or stimuli dependent? Avraham, Yeshurun, Lindenbaum VSS 2011
FLNN predictions prediction with ► predictive ability vs. Maximum value of 2 to pass the 2 test Conclusions: § Combination of feature distance and spatial distance is essential for prediction § Limited possibilities: implies that the model is informative § Relates to previous findings regarding the combined effects of proximity and similarity on perceptual grouping (e. g. , Kobovy & van den Berg 2008) Avraham, Yeshurun, Lindenbaum VSS 2011
Preliminary: Is stable or stimuli dependent? ► Experiment 2: Manipulate the number of distractor types ► Experiment 3: Manipulate the distractors variance Avraham, Yeshurun, Lindenbaum VSS 2011
Exp 2 (preliminary): manipulating the number of distractor types no clusters 4 clusters 2 distractor types (30° , 15°) 4 distractor types (60° , 45° , 30° , 15°) Avraham, Yeshurun, Lindenbaum VSS 2011
Exp 3 (preliminary): manipulating distractors variance D = 15°, 60° D = 15°, 45° D = 15°, 30° clustered not clustered Avraham, Yeshurun, Lindenbaum VSS 2011
Experiment 2 and 3: (preliminary) Results Experiment 2 Results Experiment 3 Results Avraham, Yeshurun, Lindenbaum VSS 2011
Experiment 2 and 3: FLNN Predictions with Experiment 2 Predictions Experiment 3 Predictions Avraham, Yeshurun, Lindenbaum VSS 2011
Summary ►A study of the effects of spatial organization on visual search ► The FLNN model can predict effects of grouping by similarity and grouping be proximity ► As it uses an explicit combination of feature difference and spatial distance, it can help us understand the relative effect of similarity and proximity on visual search Avraham, Yeshurun, Lindenbaum VSS 2011
Thank you Tamar Avraham Michael Lindenbaum Yaffa Yeshurun
Скачать презентацию Modeling Combined Proximity-Similarity Effects in Visual Search Tamar
fe253889e5e6dc1734e168fd6e79e67c.ppt