WP 1 Action Development V Tikhanoff 1 2

WP 1: Action Development V. Tikhanoff 1, 2, F Nori 2, G Metta 2, A. Gijsberts 2, A. Cangelosi 1 D. Marocco 1, S. Nolfi 3 1 Adaptive Behaviour and Cognition, University of Plymouth (UK) 2 Robotics Brain & Cognitive Sciences, Italian Institute of Technology (Italy) 3 Institute for Cognitive Science and Technology, CNR, Rome (Italy) ITALK Year 1 Review Düsseldorf, 30 June 2009

WP 1, goals • To design and develop an enhanced i. Cub robotic platform • To develop an i. Cub simulation application • To carry out robotic experiments on the development of object manipulation • To develop a motor controller “compatible” with language development, i. e. : – Compositional – General – Hierarchical • To investigate how to develop complex compositional and hierarchical actions ITALK Year 1 Review Düsseldorf, 30 June 2009

WP 1, tasks • Task 1. 1: Development of extended i. Cub platform for action manipulation and language learning experiments • Task 1. 2: Development and test of i. Cub simulation software for multi-agent scenarios • Task 1. 3: Development of object manipulation capabilities • Task 1. 4: Experiments on compositional and hierarchical actions • Task 1. 5: Interaction and integration between multiple motor skills ITALK Year 1 Review Düsseldorf, 30 June 2009

Task 1. 1, i. Cub and the construction of multiple copies ITALK Year 1 Review Düsseldorf, 30 June 2009

The i. Cub: quick summary • The i. Cub is the humanoid baby-robot designed as part of the Robot. Cub project • The i. Cub is a full humanoid robot sized as a three and half year-old child. • The total height is 104 cm. • It has 53 degrees of freedom, including articulated hands to be used for manipulation and gesturing. • The robot will be able to crawl and sit and autonomously transition from crawling to sitting and vice-versa. • The robot is GPL/FDL: software, hardware, drawings, documentation, etc. ITALK Year 1 Review Düsseldorf, 30 June 2009

i. Cub “size” • Three large assemblies – Head – Upper body – Lower body • More than 120 sub-assemblies • 230 commercial codes for a total of 2967 parts • 394 custom-made codes for a total of 1110 parts • Electronics, 200 components per card (on average): approximately 5000 additional parts • Each robot is made of more than 9000 parts ITALK Year 1 Review Düsseldorf, 30 June 2009

Degrees of freedom • Head: vergence, common tilt + 3 dof neck • Arms: 7 dof each – Shoulder (3), elbow (1), wrist (3) • Hands: 9 dof each ► 19 joints – 5 fingers ► underactuated • Legs: 6 dof each – Hip (3), knee (1), ankle (2) • Waist: 3 dof Σ ITALK Year 1 Review = 53 dof (not counting the facial expressions) Düsseldorf, 30 June 2009

Sensorization • Absolute position – On most joints, AMS magnetic encoder • Cameras – Pointgrey Dragonfly firewire cameras • Microphones, speaker – Standard condenser electrect miniature microphones – Pinnae • Gyroscopes, linear accelerometers – Xsense: Mtx ITALK Year 1 Review Düsseldorf, 30 June 2009

Custom electronics • ADC card – Special connectors (40 pins < 1 cm length) – 200μm stainless steel wires, coated in Teflon • Motor control – C programmable DSP 40 MIPS – Up to 4 A DC motor 80 x 30 mm 58 x 42 mm ITALK Year 1 Review Motorola DSP 56 F 807 (5680 x family) MAC instructions PWM generation ADC Digital I/O Can bus C programmable Düsseldorf, 30 June 2009

Wiki CVS Part lists Drawings

By spring 2010 we will have 18 working platforms ITALK Year 1 Review Düsseldorf, 30 June 2009

Promoting the i. Cub • Robot. Cub Open Call – 31 participants, 7 winners will receive a copy of the i. Cub free of charge – UPMC Paris, Imperial London, Inserm Lyon, TU Munich, METU Ankara, Pompeu Fabra Barcelona, Urbana-Champaign USA, IST Lisbon, EPFL Lausanne • Further development… – – EU EU project ITALK: 4 i. Cub’s have been built Im. Clever: 3 i. Cub’s will be built Robo. Skin: a skin system compatible with i. Cub CHRIS: safety features for the i. Cub • Collaborations – Univ. of Karlsruhe: new and longer legs • Simulator: – Open Source simulator based on ODE and Newton and as a model in Webots ITALK Year 1 Review Düsseldorf, 30 June 2009

In the pipeline… • Force control: joint level sensors, SEA or stain gauges based sensing • Skin/tactile sensors: almost everywhere on the robot surface • Robot general improvements: e. g. zero-backlash everywhere, better control electronics, higher resolution position sensors ITALK Year 1 Review Düsseldorf, 30 June 2009

The skin Principle Lot of sensing points Structure of the skin

ITALK Year 1 Review Düsseldorf, 30 June 2009

Fingertips • Capacitive pressure sensor with 12 sensitive zones • 14. 5 mm long and 13 mm wide, sized for i. Cub • Embedded electronics: twelve 16 bit measurements of capacitance – either all 12 taxels independently at 50 Hz or an average of the 12 taxels at about 500 Hz ITALK Year 1 Review Düsseldorf, 30 June 2009

ITALK Year 1 Review Düsseldorf, 30 June 2009

6 -axial force/torque sensor • Design: Nikos Tsagarakis and Darwin Caldwell (IIT) • Electronics: Claudio Lorini (IIT) ITALK Year 1 Review • Semiconductor strain gauges • On board signal conditioning, sampling, and calibration • Digital output: CAN bus D üsseldorf, 30 June 2009

The Software Ø Goals: – Foster collaboration in “space” and “time”… – … since we’re a relatively large community and we don’t want to reinvent the wheel too often – Manage the complexity of the hardware… – … since humanoid robots (like the i. Cub) are complicated ITALK Year 1 Review Düsseldorf, 30 June 2009

i. Cub structure s r so en S Relay station (head) DSP HUB tu Ac DSP PC 104 s or at DSP Low-level control Embedded ITALK Year 1 Review Gbit Ethernet Cluster PC 1 PCN i. Cub. API Yarp higher level capabilities control of attention reaching grasping learning imitation … … Düsseldorf, 30 June 2009

• YARP is an open-source middleware for humanoid robotics • History – An MIT / Univ. of Genoa collaboration – Born on Kismet, grew on COG – With a major overhaul, now used by Robot. Cub consortium – Exists as an independent open source project – C++ source code • In short: it’s the plumbing ITALK Year 1 Review Düsseldorf, 30 June 2009

• It factors out: – the data flow: inter-process communication • it is often useful to keep algorithms away from the plumbing – the hardware: device drivers model • it is useful to avoid references to the hardware in the source code • …while being portable: – across OS and development tools – across languages • libs in C++, bindings for many other languages ITALK Year 1 Review Düsseldorf, 30 June 2009

The i. Cub With Peter Ford-Dominey (INSERM, Lyon) With a lot of students @ Robot. Cub summer school 2008 With Auke Ijspeert, Ludovic Righetti, Sarah Degallier (EPFL) With Vis. Lab (IST Lisbon)

T 1. 1 i. Cub extensions

i. Cub platform extensions • Software: • • definition of a yarp-portable audio data format; design of a portaudio library wrapper; design of a ASR (automatic speech recognition) interface; integration of ASR software (Sphinx and Esmeralda). • Hardware: • facial expressions hardware standardization and production (installed in Bielefeld); • microphones integration (installed in Bielefeld); • stereo line-in amplifier integration (installed in Bielefeld). ITALK Year 1 Review Düsseldorf, 30 June 2009

Hardware extensions • Face expressions: • testing; • hw standardization; • production. • Integrated microphones: • electret condenser; • functional pinnae; • custom amplifiers. • Enhanced PCI card: • 4 CAN buses interfaces; • 2 firewire; • stereo line-in amplifiers. ITALK Year 1 Review D üsseldorf, 30 June 2009

Software extensions • yarp-portable audio format: yarp: sig: sound • yarp-wrapper for libportaudio: yarp: dev: Port. Audio. Device. Driver ITALK Year 1 Review Düsseldorf, 30 June 2009

Speech Recognition Engine Integration • CMU SPHINX: • a group of open source systems related to speech recognition; • speech recognition systems developed at Carnegie Mellon University. These includes a series of speech recognizers (Sphinx 2 - 4) and an acoustic model trainer (Sphinx. Train). ITALK Year 1 Review Düsseldorf, 30 June 2009

YARP/ASR interface design Segment yarp: sig: sound Microphone (windows version) Microphone (linux version) Classify According to acoustic model type “Get a sound” Interface (IAudio. Grabber. Sound) Cluster Cepstral Analysis Decode, DAG-search Port. Audio library interface Acoustic Adaptation Decode, DAG -search N-Best FFMPEG Grabber library interface Words ITALK Year 1 Review Reformat & Collate N-Best rescore Düsseldorf, 30 June 2009

Task 1. 2, simulator ITALK Year 1 Review Düsseldorf, 30 June 2009

Cognitive Robotics & Simulators • Why do we need a robot simulator? • Not an alternative to physical robot, but an additional tool; • preliminary testing/feasibility studies; • experimenting alternative configurations; • to ease accessibility to robot and collaboration within and outside the italk consortium. • Existing robot simulation applications: • • Stage / Gazebo Project ( 2 D / 3 D multiple-robot simulator); Simbad Project (3 D multiple-robot simulator); Darwin 2 K Project (Simulation & Automated Synthesis for Robotics); Evo. Robot (Evolutionary Robotics Simulator); ITALK Year 1 Review Düsseldorf, 30 June 2009

Physics & rendering engines • A Physic Engine library provides: • • Simulation of rigid bodies (mass, friction, sensors etc. . ) Collision Detection Algorithms for all the physical interactions Advanced joint types (also customizable) Terrains and/or Meshes for complex object creation • There are many available physic engines with their pro and cons • Different solvers for rigid-body dynamic • Different spaces for collision detection • More joint types already implemented • Open. Source, GPL license • Multi-thread capability • More accurate friction simulation • API for custom joint creation and control • Free license but not Open. Source Robot Simulation – Robot Programming with Open Dynamics Engine Morikita Publishing Co. Ltd Tokyo 2007 ITALK Year 1 Review Düsseldorf, 30 June 2009

Physics engine & rendering • World. Sim library has been created for providing • • Support different physic engines Object-Oriented design for an easy creation of a custom version of simulator Possibility to simulate more than one i. Cub 3 D renderiomg classes based on Open. GL ITALK Year 1 Review Düsseldorf, 30 June 2009

Software Architecture • Objective: control robot with same commands for both real & simulated i. Cub: • use the same software infrastructure and inter-process communications. • Tools: real and simulated robot software architecture is based on YARP: • motor interface: yarp: : dev: : control. Board; • video interface: yarp: : sig: : Image; • sensor interface: yarp: os: Portable. ITALK Year 1 Review Düsseldorf, 30 June 2009

The i. Cub Platforms i. Cub Physical Robot and i. Cub. Simulated robot • Total height 105 cm • Total weight 20. 5 kg • 53 Do. Fs in Total 12 for the legs 3 for the torso 32 for the arms 6 for the head • Emotion interface • Touch sensors ITALK Year 1 Review Düsseldorf, 30 June 2009

Robot Control Ports Description /icub. Sim/left_leg/rpc: i /icub. Sim/left_leg/command: i /icub. Sim/left_leg/state: o /icub/left_leg/state: o The left leg has 6 joints in the standard configuration /icub. Sim/right_leg/rpc: i /icub. Sim/right_leg/command: i /icub. Sim/right_leg/state: o /icub/right_leg/state: o Same as left leg /icub. Sim/torso/rpc: i /icub. Sim/torso/command: i /icub. Sim/torso/state: o /icub/torso/state: o The Torso has 3 joints in the standard configuration /icub. Sim/left_arm/rpc: i /icub. Sim/left_arm/command: i /icub/left_arm/command: i User types /icub. Sim/left_arm/state: o /icub/left_arm/state: o The arm includes the hand for a total of 16 controlled degrees System responds of freedom Help (a list of available yarpdev commands) /icub. Sim/right_arm/rpc: i /icub/right_arm/rpc: i As for the left arm [get] [axes] /icub. Sim/right_arm/command: i [is] [axes] 2 [ok] /icub/right_arm/command: i /icub. Sim/right_arm/state: o /icub/right_arm/state: o [set] [pos] 0 100. 0 [ok] /icub. Sim/head/rpc: i /icub/head/rpc: i The head has 6 joints in the [get] [enc] 0 [is] enc 100 [ok] /icub. Sim/head/command: i /icub/head/command: i standard configuration. . /icub. Sim/head/state: o /icub/head/state: o ITALK Year 1 Review Düsseldorf, 30 June 2009

Transferability – Simulator to Physical Requires just switching the robot name in the configuration files or programs from “icub. Sim” to “icub”. • • The simulated robot shares some important robot features: • kinematic specifications; • motor interface (control. Board interface) ; • vision. • The simulated robot is not a “perfect” match of the physical robot: • rough (body) dynamical model; • rough trajectory generator; • rough actuators dynamical model. ITALK Year 1 Review Düsseldorf, 30 June 2009

i. Cub. Sim demo • Launching the simulator. • Accessing the motor interface: – via command line (yarp rpc); – via the robot. Motor. Gui; – via C++ code (preprogrammed sequences). • Accessing the video interface: – via the yarpview. • Object manipulation scenario. ITALK Year 1 Review Düsseldorf, 30 June 2009

Task 1. 3, motor (and cognitive) capabilities ITALK Year 1 Review Düsseldorf, 30 June 2009

On-line life-long learning • Developmental robotics entails: – Quasi online • at least memory bounded • adapt to changes in the dynamics (e. g. tools) – Fast computation of the solution • useful in control loops – Large data-sets • perhaps collecting data for the entire “life” of the robot – Guarantees • convergence, bounds, etc. ITALK Year 1 Review Düsseldorf, 30 June 2009

Methods • Kernel methods – Online variants – More of a research area • ANN, feedforward, recurrent – Training via backpropagation – Via reinforcement – More established ITALK Year 1 Review Düsseldorf, 30 June 2009

Scenario: dataset • Inputs: Joint positions, velocities and accelerations for 4 joints (3 shoulder, 1 elbow) • Outputs: Forces and torques in x, y, z directions (here we only consider Fx) • 100. 000 samples collected at 50 Hz during random movements • Inputs scaled “roughly” within [1, +1] ITALK Year 1 Review Düsseldorf, 30 June 2009

Some batch results ITALK Year 1 Review Düsseldorf, 30 June 2009

Passive Aggressive regression • Variant of Perceptron algorithm • Margin-based (also updates with margin errors) like SVM • Easily kernelized using kernel expansion • Binary and multiclassification, regression, etc. • Efficient and easy to implement Crammer et al. , Online passive-aggressive algorithms, 2006. ITALK Year 1 Review Düsseldorf, 30 June 2009

On-line, efficient, bounded Time vs. # of samples for polynomial kernel Time vs. # of samples explicit polynomial Time vs. # of samples for random approximation Accuracy for random approximation ITALK Year 1 Review Düsseldorf, 30 June 2009

Learning motor control • Map sensory information into appropriate behaviors: – Gazing – Reaching – Grasping ITALK Year 1 Review Düsseldorf, 30 June 2009

Example: gazing ITALK Year 1 Review Düsseldorf, 30 June 2009

Reaching • Model acquisition for reaching ITALK Year 1 Review Düsseldorf, 30 June 2009

Task-space control In order to implement a controller to achieve a desired pose in task space (position + orientation) an inverse kinematics library has been developed. Major drawbacks of available algorithms for IK (Pseudoinverse, DLS): need to manage singularities, need of boundary functions to embody the notion of joints limits. Our solution uses interior point optimization to solve the constrained nonlinear problem of the form: Singularities automatically handled Joints limits are automatically included and satisfied Easy description of multiple and simultaneous tasks (e. g. secondary tasks)

Example Comparison of random XYZ positions of objects, with the actual resulting position of the robots hand ITALK Year 1 Review Düsseldorf, 30 June 2009

Grasping Trained by the Associative Reward Penalty algorithm (ARP) (Barto and Jordan, 1989) The reward is proportional to the time the object is stable in the hand of the robot ITALK Year 1 Review Düsseldorf, 30 June 2009

Example • Trained on a single object, tested on many others Location of the six touch sensors on the i. Cub’s simulator hand ITALK Year 1 Review Düsseldorf, 30 June 2009

Task 1. 4, composition of actions (and in part T 4. 3) ITALK Year 1 Review Düsseldorf, 30 June 2009

Action Compositionality • Two current studies: 1. Embodiment and self-organisation of action ( motion verbs) 2. Hierarchical learning of actions ITALK Year 1 Review Düsseldorf, 1 July 2009

Embodiment and self-organisation of action concepts Different levels of embodiment Touch / Grasp Roll / Slide / Hit Pick-up / Put-down Move (from a to b) / Put q into p q slides on p / q pick-up p (Siskind, 2001) Vision based system Pre-defined primitives ITALK Year 1 Review Düsseldorf, 30 June 2009

Experimental Setup Categorisation of actions (and acquisition of corresponding linguistic labels) is based upon the integration of various sensory information that allow to characterize differently the situations. E. g. Different objects produce different reactions to the same action. Ball - rolling ITALK Year 1 Review Cube - sliding Düsseldorf, 30 June 2009

Action (T 1. 4) and Verb Learning (T 4. 3) • Only few studies (Wermter & Elshaw, 2003) focus the attention on the acquisition of verbs that underpin a dynamical and force-varied interaction with object, such as hit or slide or verbs (actions) that are connected to property of objects, i. e. rolling for a ball • Our work will follow this direction ITALK Year 1 Review Cangelosi & Riga (2006) Marocco et al. (2003) Düsseldorf, 30 June 2009

Future work ITALK Year 1 Review Düsseldorf, 30 June 2009

Future work • T 1. 1: finishing certain parts of the copies of the i. Cub (force/torque sensors, covers) • T 1. 2: improve the simulator on a perneed basis • T 1. 3: proper grasping behavior including tactile sensors • T 1. 4: composition of actions • T 1. 5 (starting M 18) – Interaction and integration between multiple motor skills ITALK Year 1 Review Düsseldorf, 30 June 2009