Скачать презентацию Anchorage Alaska 3 May 2010 ICRA Planetary Rover Скачать презентацию Anchorage Alaska 3 May 2010 ICRA Planetary Rover

8a46d96f2a8b3dcc58a22d69b5f3e70a.ppt

  • Количество слайдов: 41

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Rover Navigation and Visual Odometry: Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Rover Navigation and Visual Odometry: A New Framework for Exploration Activities Enrica Zereik, Enrico Simetti, Alessandro Sperindé, Sandro Torelli, Fabio Frassinelli, Davide Ducco and Giuseppe Casalino GRAAL Lab, DIST, University of Genoa

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Why a Framework? Develop a Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Why a Framework? Develop a software architecture to let researchers focus their attention on the control algorithm only, without caring about the underlying physical system • to deal with the underlying specific hardware platform • to solve problems related to real-time constraints of control systems • to provide data-unaware communication mechanisms • to be reused for different control systems in several applications

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Main Objectives • Independency of Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Main Objectives • Independency of each control algorithm from the underlying software platform • Minimization of the number of code lines not strictly related to the control algorithm • Capability of coordination between remote frameworks • Standard communication mechanism between control tasks (minimum impact on the algorithm)

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of each control algorithm from the underlying software platform • Minimization of the number of code lines not strictly related to the control algorithm • Capability of coordination between remote frameworks KAL • Standard communication mechanism between control tasks (minimum impact on the algorithm)

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of each control algorithm from the underlying software platform • Minimization of the number of code lines not strictly related to the control algorithm • Capability of coordination between remote frameworks • Standard communication mechanism between control tasks (minimum impact on the WF algorithm)

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Abstraction Levels • Independency of each control algorithm from the BBS underlying software platform • Minimization of the number of code lines not strictly related to the control algorithm • Capability of coordination between remote frameworks • Standard communication mechanism between control tasks (minimum impact on the algorithm)

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop KAL: Kernel Abstraction Layer • Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop KAL: Kernel Abstraction Layer • Work. Frame Name Server: abstraction of the OS Server resources and services

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop WF: Work. Frame • System Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop WF: Work. Frame • System Manager: resource request handling Manager • Sched: Rel Sched can synchronize frameworks Sched • Logger: communication toward user Logger

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop BBS: Black. Board System • Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop BBS: Black. Board System • Inter-task communication • Resource access • Both local and remote tasks • Shared Black. Board publishing data • Local execution of computation involving BB data

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Framework Hierarchy 1 1 4 Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Framework Hierarchy 1 1 4 2 3 Resources, Scheduling Device I/O 2 Mutually Exclusive Interprocess Data Sharing (also with remote tasks) 3 Network Communication 4 C++ Math Routines

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair • Stereo Matching: correspondence research Matching • Triangulation: Triangulation computation correspondent 3 D point • Tracking in Time: tracking the same features in Time the following image acquisition • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair LOG filtering • Stereo Matching: correspondence research Matching + SURF (robust • Triangulation: correspondent 3 D point Triangulation descriptors ) computation • Tracking in Time: tracking the same features in Time the following image acquisition • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair • Stereo Matching: correspondence research Matching • Triangulation: correspondent Triangulation Epipolar constraint, computationdescriptor-based 3 D point • Tracking in Time: tracking the same features in Time the following image acquisition • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair • Stereo Matching: correspondence research Matching • Triangulation: Triangulation computation correspondent 3 D point • Tracking in Time: tracking the same features in Time Subject to erros, the following image acquisition outliers rejected • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair • Stereo Matching: correspondence research Matching • Triangulation: Triangulation computation No external estimation, correspondent descriptor-based 3 D point • Tracking in Time: tracking the same features in Time the following image acquisition • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Visual Odometry Module • Feature Extraction: in each image of the stereo Extraction pair • Stereo Matching: correspondence research Matching • Triangulation: Triangulation computation correspondent 3 D point Least Square (outlier rejection, • Tracking in Time: tracking the same features in Time initial estimation) + Maximum the following image. Likelihood Estimation acquisition • Motion Estimation: estimation of the motion occured Estimation between the two considered stereo image pairs

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Experimental Setup • Custom mobile Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Experimental Setup • Custom mobile platform @ GRAAL • Tricycle-like structure • Bumblebee 2 stereo camera system

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Preliminary Results Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Preliminary Results

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Robotic Crew Assistant for Exploration Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Robotic Crew Assistant for Exploration Missions: Vision, Force Control and Coordination Strategies Enrica Zereik, Andrea Sorbara, Andrea Merlo, Frederic Didot and Giuseppe Casalino GRAAL Lab, DIST, University of Genoa European Space Agency Thales Alenia Space, Italy

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Eurobot Wet Model Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Eurobot Wet Model

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Eurobot Ground Prototype pan/tilt stereo Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Eurobot Ground Prototype pan/tilt stereo cameras for rover navigation 7 d. o. f. arms, one camera on each JR 3 force/torque sensor exchangeable end-effector arm cameras four-wheeled rover for autonomous navigation pan/tilt stereoscopic head for manipulation

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop EGP - Control Aspects Coordination Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop EGP - Control Aspects Coordination • Coordination Rover and Arms • Dynamic Programming-based strategy Vision • Object recognition and centering • ARTool. Ki. TPlus and Open. CV support Force • Approaching and actual grasping • Contact detection

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop General Control Architecture with Priority Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop General Control Architecture with Priority Tasks Dynamic Programming-based • Coordination of robotic macro-structures • Independent from the specific system configuration • Many different control objectives can be required Velocity control task requirement Associated cost-to-go Moving platform velocity

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop General Control Architecture with Priority Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop General Control Architecture with Priority Tasks Task i-th:

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Backward Phase Use of relationships Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Backward Phase Use of relationships Monitoring MM tendency toward

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Forward Phase Remarks • The Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Forward Phase Remarks • The risk of MM losses still exists (e. g. if the object must be very high lifted) • If a MM loss is detected the last resort solution is modulating • Implicit Priority Change

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Implicit Priority Change Backward phase Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Implicit Priority Change Backward phase at platform level

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Vision-based Recognition of Objects Marker-based Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Vision-based Recognition of Objects Marker-based object tracking • Reliability • Robustness Occurring problems Preliminary Thresholding • Lighting conditions • Complexity of the captured scene • Distance from which the marker is seen Image Cleaning Image Zooming

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Image Processing Chain Auto threshold Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Image Processing Chain Auto threshold To Estimator Image zooming Image cleaning Pose Estimator Image from camera Pose estimation LPF To E-GNC

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Implicit Priority Change angular error Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Implicit Priority Change angular error linear error angular error after zooming angular error after LPF linear error after zooming linear error after LPF

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Force-based Approach towards Objects Direct Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Force-based Approach towards Objects Direct Force Control Strategy • Detect a contact with the object to be grasped • Compensate residual errors Pure Force Only at the Palm Level • felt by the JR 3 sensor • the contact point must belong to the palm surface known and constant

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Force-based Approach towards Objects Velocity Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Force-based Approach towards Objects Velocity Generation • Contact point estimation • Velocity assigned to the estimated contact point with • Compute velocity reference with respect to the robot end-effector Remarks • Noisy sensor and too long distance from palm • Initial error very small thanks to vision

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Simulative Results Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Simulative Results

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Experimental Results Video • 3. Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Experimental Results Video • 3. EGP Failed Equipment Replacement. avi

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Conclusions and Future Work EGP Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Conclusions and Future Work EGP • Effective and autonomous robotic crew assistant • Marker removal • Potentially, flight model Potentially Planetary Rovers • Visual Odometry error less than 1% • 3 D reconstruction of the environment • DEM construction and autonomous navigation

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, I EGP [1] T. Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, I EGP [1] T. Kröger, D. Kubus and F. M. Wahl, “ 6 D Force and Acceleration Sensor Fusion for Compliant Manipulation Control”, IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, October 2006. [2] B. J. Waibel and H. Kazerooni, “Theory and Experiments on the Stability of Robot Compliance Control”, IEEE Transactions on Robotics and Automation, February 1991, vol. 7, no. 1, pp. 95 -104. [3] G. Bradski and A. Kaehler, “Learning Open. CV: Computer Vision with the Open. CV Library”, O'Reilly. [4] C. P. Lu, G. D. Hager and E. Mjolsness “Fast and Globally Convergent Pose Estimation From Video Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, June 2000, vol. 22, no. 6, pp. 610 -622.

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, II [5] B. Kainz Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, II [5] B. Kainz and M. Streit, “How to Write an Application with Studierstube 4. 0”, Technical report, Graz University of Technology, 2006. [6] J. Cai, “Seminar Report: Augmented Reality: the Studierstube Project”, Seminar report. [7] E. Zereik, A. Sorbara, G. Casalino and F. Didot, “Autonomous Dual. Arm Mobile Manipulator Crew Assistant for Surface Operations: Force/Vision-Guided Grasping”, International Conference on Recent Advances in Space Technologies, Istanbul, Turkey, June 2009. [8] E. Zereik, A. Sorbara, G. Casalino and F. Didot, “Force/Vision. Guided Grasping for an Autonomous Dual-Arm Mobile Manipulator Crew Assistant for Space Exploration Missions”, International Conference on Automation Robotics and Control Systems, Orlando, USA, July 2009.

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, III [9] G. Casalino Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, III [9] G. Casalino and A. Turetta, “Coordination and Control of Multiarm Nonholonomic Mobile Manipulators", MISTRAL: Methodologies and Integration of Subsystems and Technologies for Robotic Architectures and Locomotion, B. Siciliano, G. Casalino, A. De Luca, C. Melchiorri, Springer Tracts in Advanced Robotics, Springer-Verlag, April 2004. [10] G. Casalino, A. Turetta and A. Sorbara, “Dynamic Programming based Computationally Distributed Kinematic Inversion Technique”, Advanced Space Technologies for Robotics and Automation, Noordwijk, The Netherlands, November 2006. [11] G. Casalino, A. Turetta and A. Sorbara, “DP-Based Distributed Kinematic Inversion for Complex Robotic Systems”, 7 th Portuguese Conference on Automatic Control, Lisbon, Portugal, September 2006. [12] E. Zereik, “Space Robotics Supporting Exploration Missions: Vision, Force Control and Coordination Strategies”, Ph. D. Thesis, University of Genova, 2010.

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, IV Visual Odometry [1] Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, IV Visual Odometry [1] M. Maurette and E. Baumgartner, “Autonomous Navigation Ability: FIDO Test Results”, 6 th ESA Workshop on Advanced Space Technologies for Robotics and Automation, Noordwijk, The Netherlands, November 2000. [2] M. Maimone, Y. Cheng, and L. H. Matthies, “Two Years of Visual Odometry on the Mars Exploration Rovers”, Journal of Field Robotics, March 2007, vol. 24, no. 3, pp. 169 -186. [3] A. E. Johnson, S. B. Goldberg, Y. Cheng and L. H. Matthies, “Robust and Efficient Stereo Feature Tracking for Visual Odometry”, IEEE International Conference on Robotics and Automation, Pasadena, USA, May 2008. [4] L. Matthies, “Dynamic Stereo Vision”, Ph. D. Thesis, Carnegie Mellon University.

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, V [5] D. Nistér, Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop References, V [5] D. Nistér, O. Naroditsky and J. Bergen, “Visual Odometry”, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, USA, June 2004. [6] Richard Hartley and Andrew Zisserman, “Multiple View Geometry in Computer Vision”, Cambridge University Press, March 2004. [7] E. Trucco and A. Verri, “Introductory Techniques for 3 -D Computer Vision”, Prentice Hall, 1998. [8] I. J. Cox, S. L. Hingorani, S. B. Rao and B. M. Maggs, “A Maximum Likelihood Stereo Algorithm”, Journal of Computer Vision and Image Understanding, 1996, vol. 63, no. 3, pp. 542 -567. [9] M. Fischler and R. Bolles, “Random Sample Consensus: a Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography”, Communications of the Association for Computing Machinery, June 1981, vol. 24, pp. 381 -395.

Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Thank you for your kind Anchorage, Alaska, 3 May 2010 ICRA Planetary Rover Workshop Thank you for your kind attention!!