Università degli Studi di Modena e Reggio Emilia

Università degli Studi di Modena e Reggio Emilia Dipartimento di Ingegneria dell’Informazione Image. Lab: Attività di ricerca e sviluppo in video-sorveglianza Prof. Ing. Rita Cucchiara Image. Lab Dipartimento di Ingegneria dell’Informazione Università di Modena e Reggio Emilia, Italy

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Imagelab Dipartimento di Ingegneria dell’Informazione Università di Modena e Reggio Emilia, Italy Imagelab Laboratory of Image Analysis for Computer Vision and Multimedia Staff Prof. Rita Cucchiara, Ph. D (Faculty staff) Ing. Andrea Prati, Ph. D (Faculty staff) Ing. Costantino Grana, Ph. D (Faculty staff) Ing. Roberto Vezzani, Ph. D Ing. Simone Calderara Ing. Giovanni Gualdi Ing. Daniele Borghesani (post-Doc ) (Ph. D student) (Ph-d Student) (fellow) Ing, Rudi Melli, Ing. Giuliano Pistoni, Ing. Emanuele Perini Vision-E Dr. Amit Tilvankar Prof. Massimo Piccardi http: //imagelab. ing. unimo. it (University of Technology, Sidney) 10 Un-graduated students… Ø (visiting researcher)

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Research Activities at Image. Lab Computer Vision for robotic automation Transcoding for Multimedia Image analysis for Medical Imaging Multimedia: video annotation Video analysis for indoor/outdoor surveillance Video analysis for telemetry People and vehicle surveillance Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia CV for robotic automation Projects in collaboration with companies with local and regional support: some examples Vision-Based Navigation for AGV Vehicle (2005 -2006) l Research activity in vision-based localization, navigation and Obstacle detection for Automated Guided Vehicle. System for automatic creation of ceramic mosaics (2005 -2006) l Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Detecting borders in labels for textile implants (2006 -2007) l Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Analysis of car vibrations (2006 -2007) l analysis of head vibration in Ferrari cars Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Medical Imaging: Dental tomography (1999 -2002) Echocardiography ( Burhnam Institute, CA, USA) (2002 -2004) Dermatological image analysis (1999 -2007) l Segmentation l Feature analysis l Mosaicing l Retrieval Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Multimedia Cache performance analysis (2000 -2004) l MIUR ITALIAN Project (2002 -2004) Video Transcoding (2002 -2008) Video Annotation (2004 -2008) l EU VI FP DELOS Project (2004 -2008) l EU VII FP VIDI-VIDEO Project (2007 -2009) Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Video surveillance- old projects Ø Experiments since 1996 ( University of Ferrara, University of Bololgna) Ø Projects with Bologna Municipality (1997 -1999) Ø Sakbot project (University of Modena) (1998 -2001) Ø “Domotica per Disabili” Project indoor surveillance (2001 -2003) Ø Sakbot for Traficon (2003 -2006)

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Video-surveillance now (1/2) Ø LAICA (Laboratorio di Ambient Intelligence per una Città Amica) funded by Regione Emilia-Romagna, Italy (2005 -2006 -? ) Collab. With WTI Techn. , Regulus, Comune Reggio Emilia… Surveillance in public parks with distributed and ptz cameras. Ø BE-SAFE Bheavior l. Earning foir Surveillance application with feature extraction –NATO Science for Peace program (2007 -2008) With Israel (Hebrew Yniv, Magal) Ø Automatic real-time detection of infiltrated objects for security of airports and train stations (2006 -2008) Australian Research Council with University of tecnlogy, Sidney Ø FREE-SURF: Free Surveillance in a Privacy Respectful way Funded by Italian MUR PRIN Project (2006 -2008) With Univ. of firenze, Palermo, IRST Trento

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Video-surveillance now (2/2) Ø Collaboration with Municipal Policy of Modena forensis analysis (2006 -2007) Ø Mobile remote Surveillance (2006 -2007) funded by ER PRIITT project with Sistemi integrati spa Ø Traffic analysis on highway (2007) with Digitek spa Ø VISOR Project in Vidivideo (2007 -2009) Ø Smoke detection (WTI 2007)

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia People real time remote surveillance 2003 -2004 “Domotica per Disabili” FCR Modena Ø Posture detection for domotic application Ø 2006 PRIITT “Videosorveglianza mobile” Sistemi Integrati remote surveillance with h 264 streaming Ø

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Remote Surveillance Mobile surveillance Ø Int. Symp. On Multimedia 2007 Ø

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Projects at imagelab Ø 2004 -2006 vehicle and traffic surveillance - Traficon (B) S. Diego highway

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Projects at Image. Lab Ø 2005 -2006 LAICA Project Laboratorio di Ambient Intelligence per una Città Amica Piano Telematico ER ØPublic park in Reggio Emilia, ØModena Campus

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Projects at Image. Lab Ø (2006 -2008) Automatic real-time detection of infiltrated objects for security of airports and train stations Australian Research Council

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Project at imagelab Ø Ø Ø NEW 2007 -2009 Free SURF free surveillance in a privacy respectful way MIUR PRIN Project (national coordinator) 2007 -2009 BE SAFE Behavioral l. Earning in Surveilled Areas with Feature Extraction NATO Science for Peace project with Israel

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Multiple-people Multiple-camera surveillance 1) single fixed camera l l l SAKBOT: Moving object segmentation (PAMI 2003) Shadow analysis (PAMI 2003) Tracking with occlusions (PR-submitted) Posture analysis (T SMC 2004) Face detection (PRL 2005) 2) single moving PT camera l l Mosaicing Motion clustering (RTI 2004) 3) multiple (overlapped Fo. Vs) cameras l HECOL Homography and Epipolar based consistent labeling (PAMI Submitted, CVIU) 4) multiple (not overlapped Fo. Vs) cameras l Color analysis and retrieval 5) Trajectory analysis • trajectory clustering for abnormality detection

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Video surveillance Surveillance of Natural environment (fire detection, fog detection…, ) Urban Environment ( crowd monitoring in stations, metro. . ) Ø Environment Abnormal situations detection Ø Traffic Ø People Road traffic control: Statistical control ( vehicles monitoring, classification ) Accident detection ( queue, incidents. . ) Urban control: people at road crossing, privileged path. . Naval, aerial traffic control Security: people detection in specific area people tracking for abnormal behavior detection people interaction analysis Safety: people behavior monitoring in house and offices people behavior analysis in public place people behavior analysis in car

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Human-based video surveillance Ø Most of commercial available video surveillance systems still provide acquisition, storage (and access) only Data Acquistion Data Storage internet/ intranet Data Compression Universal Multimedia Access modelli Data Compression

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Human-based video surveillance System generations Ø 1) analog cameras and CCTV video monitors Ø

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Human-based video surveillance Ø System generations 2) Digital video recorder system 3) Digital PC based or embedded encoding system

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Human-based video surveillance Ø System generations 4) LAN cameras 5) LAN cameras with streaming Wireless cameras

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Human-based video surveillance Ø Effort in new hardware solutions l l Distributed vs centralized New cameras • PTZ, infrared, thermal, omnidirectional panoramic cameras Standard vs. stand alone encoding solutions Ø User friendly (multimedia) human interface Ø Exploiting motion detection Ø l l Improving human attention Storage space saving Knowledge extraction Data Acquisition DBMS modelli Intra-inter net Human/machine based Control center

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia (Automated) computer vision-based video surveillance Ø Exploiting image processing, pattern recognition and computer vision techniques in order to l Extract moving objects l Detect variations of objects’ motion status l Measure object flow l Classify people and other objects l Track people and objects l Consistent Track in distributed system l Recognize Bheavior, posture, interaction

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Multiple-people Multiple-camera surveillance at Image. Lab Sakbot Ad hoc Hecol PPC Projects and prototypes since 1996

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Related Imagelab bibliography l TRAFFIC CONTROL l l R. Cucchiara, A. Prati, R. Vezzani, L. Benini, E. Farella, P. Zappi, "An Integrated Multi-Modal Sensor Network for Video Surveillance" Journal of Ubiquitous Computing and Intelligence (JUCI), 2006 DISTRIBUTED SURVEILLANCE l l R. Cucchiara, C. Grana, A. Prati, R. Vezzani, "Probabilistic Posture Classification for Human Behaviour Analysis" in IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 35, n. 1, pp. 42 -54, 2005 MULTISENSOR l l Prati, I. Mikic, M. M. Trivedi, R. Cucchiara, "Detecting Moving Shadows: Algorithms and Evaluation, " IEEE Transactions on Pattern Analysis and Machine Intelligence, July 2003 POSTURE ANALYSIS l • R. Cucchiara, C. Grana, M. Piccardi, A. Prati “Detecting Moving Objects, Ghosts and Shadows in Video Streams“, IEEE Transactions on Pattern Analysis and Machine Intelligence, july 2003 SHADOW DETECTION l • IEEE Transactions on Intelligent Transportation Systems, vol. 1, n. 2, pp. 119 -130, 2000 SAKBOT l l R. Cucchiara, P. Mello, M. Piccardi, "Image Analysis and Rule-Based Reasoning for a Traffic Monitoring, " S. Calderara, A. Prati, R. Cucchiara, HECOL Homography and epipolar based consistent labeling for outdoor park surveillance in press on Computer vision and Image Understanding 2007 FACE DETECTION l R. Cucchiara, A. Prati, R. Vezzani, "A System for Automatic Face Obscuration for Privacy Purposes" in Pattern Recognition Letters, vol. 27, n. 15, pp. 1809 -1815, 2006

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Moving object segmentation: SAKBOT 1) Statistical and knowledge based background modeling Ø 2) color based background suppression with shadow detection Ø 3) Classification and MVOs segmentation Ø It Background Update Bt Background Suppression DB t Shadow detection Segmentation Object validation Ghost MVO Ghost shadow t MVO shadow Tracking Scene ghost Objects History MVO shadow Ghost shadow

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia SAKBOT RESULTS Ø Ø Ø Experiments against PETS 2001, PETS 2006 VSSN dataset First classified in background suppression competition ACM Multimedia VSSN 06 S. Barbara USA Future implementation in Intel Open. CV Used l in indoor and outdoor applications (people detection) l In traffico control l In sea surveillance

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Multiple people Tracking with occlusions: Ad Hoc Ø Ø Ø Ad hoc : Appearance-based, with occlusion classification tracking Probabilistic model of tracking Appearance based (probabilistic at pixel level) Discriminative: detect the best position and appearance at each frame Handling occlusion l Occlusion classification l Keep object appearance also during occlusion

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Example Ø Example of tracking without occlusion update

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Example

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia HECOL: distributed surveillance park Ø HECOL Homography and epipoles based consistent labeling, in a probabilistic framework. . road 1) 2) 3) 4) Compute overlapped camera Fo. V Construct the Camera Transition Graph Sakbot + Ad Hoc tracking for each camera when one or more new objects are in the overlapped area 1) Prune the search space with CTG 2) Construct the hypotheses space 3) Construct the forward and backward probability to link to other single or group of people 4) Solve with a double MAP the consistency building sec. entrance building Main entranc e c 2 c 1 benches

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Homography and epipolar lines

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia A priori and conditional probability Matching With occlusion With multiple people

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia New and Future works Tracking people with PT cameras Bheavior analyis Face tracking for identification Surveillance video annotation (VIDI Video Project)

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Smoke detection

Rita Cucchiara- Image. LAB-DII Università di Modena e Reggio Emilia Future events Ø ICIAP Ø http: //localhost/iciap 07/ Ø International Conference on l l l Pattern recognition and computer vision Multimedia Human centered application Security and surveillaince Industrial applications Workshops l Computational Color imaging l DELOS Visual and Multimedia digital library Ø The Industrial Forum Ø European project forum Ø For more information… http: //imagelab. ing. unimo. it Rita Cucchiara Tel. 059 2056136 rita. cucchiara@unimo. it