WEBVTT 00:00:03.477 --> 00:00:10.517 This is the Convergent Science Network podcast. Leading researchers in the domain 00:00:10.517 --> 00:00:16.797 of neuroscience, brain theory and technology are interviewed by Paul Verscher and Tony Prescott. 00:00:19.717 --> 00:00:26.157 This is Tony Prescott for the Convergent Science Network podcast from the Barcelona 00:00:26.157 --> 00:00:29.957 Summer School on Cognition, Brain and Technology 2011. 00:00:31.257 --> 00:00:38.217 I'm here with Maria Chiara Carroza from PISA, who's the director of the Scuola 00:00:38.217 --> 00:00:43.897 Superiore Santana and the leader of the Neuro-Robotics Research Group. 00:00:44.897 --> 00:00:51.857 So, Chiara, in your talk today, you used the phrase human-robot symbiosis. 00:00:51.857 --> 00:00:55.277 Can you explain what you mean by that? 00:00:55.997 --> 00:00:59.557 Symbiosis for us, we are developing robots 00:00:59.557 --> 00:01:02.637 and means that the robot and the 00:01:02.637 --> 00:01:10.997 user of the robot should share the same objectives and are interdependent in 00:01:10.997 --> 00:01:21.137 going to task and in making actions and in doing some activities of daily living. 00:01:21.137 --> 00:01:26.897 So the robot and the human being are in contact and share the same goals. 00:01:28.277 --> 00:01:33.357 Okay, so how is it that the robot can understand the goals of a person? 00:01:33.357 --> 00:01:36.617 So this is the crucial problem to be solved, 00:01:36.757 --> 00:01:43.397 how to exchange information between the robot and human subject and in order 00:01:43.397 --> 00:01:51.857 to have the subject being in control of the robot with what we call natural control. 00:01:52.877 --> 00:01:56.997 And the title of your group is the Neuro-Robotics Research Group. 00:01:57.097 --> 00:02:02.537 So that implies that you're in some way trying to read the neural activity of 00:02:02.537 --> 00:02:03.577 the person. Is that right? 00:02:05.357 --> 00:02:13.577 The neuro-robotics means that we would like to join the knowledge in the field 00:02:13.577 --> 00:02:18.197 of robotics and in the field of neuroscience in order to develop better robots 00:02:18.197 --> 00:02:22.197 robot able to read the activity of the brain. 00:02:22.357 --> 00:02:26.197 It's not important to read the activity of neurons, but of the brain. 00:02:26.457 --> 00:02:32.437 So we would like to exploit the signals that are available in order to understand 00:02:32.437 --> 00:02:39.397 what the subject really wants to do and try to send this command to the robot 00:02:39.397 --> 00:02:45.457 in order to have the robot execute the action that the human being wants to perform. 00:02:45.837 --> 00:02:51.697 So for the robot to To understand the human, there are various signal channels that you could access. 00:02:51.977 --> 00:02:54.577 And so which are the ones that you think are the most promising? 00:02:55.880 --> 00:03:01.400 It's very difficult to... We are still investigating, so it's difficult to say 00:03:01.400 --> 00:03:04.100 which are more promising. 00:03:04.420 --> 00:03:09.160 We are for the moment using non-invasive signals, which means that we are not 00:03:09.160 --> 00:03:15.840 implanting interfaces in the brain and in the cortical areas of the brain. 00:03:15.840 --> 00:03:21.740 We are trying to exploit the signals that are available without providing a 00:03:21.740 --> 00:03:24.500 surgical implant to the subject. 00:03:24.660 --> 00:03:32.100 That means that we are using electromyographic signals from the surface electrodes on the skin. 00:03:32.480 --> 00:03:38.240 We are also using the movements of the limbs in order to detect the intention. 00:03:38.240 --> 00:03:44.060 We are using the movement of the eyes and all physiological signals that are 00:03:44.060 --> 00:03:48.120 available in order to understand what the subject wants to do, 00:03:48.260 --> 00:03:53.980 and also the conditions of the subject, which are important to provide a good 00:03:53.980 --> 00:03:56.480 interaction between the robot and the subject. 00:03:56.980 --> 00:04:03.620 So it sounds like this is some kind of wearable technology that people would have. Yes, yes. 00:04:03.760 --> 00:04:11.720 We are investigating wearable robots and in particular we are investigating wearable interfaces. 00:04:12.000 --> 00:04:16.300 So interfaces that are in contact with the human skin and the human body, 00:04:16.420 --> 00:04:23.040 able to detect the movements and also able to stimulate the human skin in order 00:04:23.040 --> 00:04:26.640 to provide some sensory feedback to the user. 00:04:26.640 --> 00:04:31.180 Because percepting the environment is important for the user. 00:04:31.340 --> 00:04:35.540 And so we have to provide some sensory feedback in order to achieve, 00:04:35.700 --> 00:04:42.360 to reach the perception in the human brain, which is a mix of sensory feedback, 00:04:42.720 --> 00:04:45.400 experience and knowledge and motivation. 00:04:45.580 --> 00:04:50.940 So we need several agents in order to provide perception. 00:04:51.200 --> 00:04:55.320 And perception is fundamental in order to provide natural control. 00:04:55.900 --> 00:05:01.020 So the types of robots that you're mainly interested in are ones that are interacting 00:05:01.020 --> 00:05:02.480 very closely with people. 00:05:02.640 --> 00:05:06.740 And one type of robot, I guess, would actually be a prosthesis, 00:05:06.900 --> 00:05:11.740 something like a replacement limb, which would be attached to a person and read 00:05:11.740 --> 00:05:14.260 these kinds of signals from their skin, for instance. 00:05:14.260 --> 00:05:21.100 Yes, we are investigating prosthetic hands and in particular how to control 00:05:21.100 --> 00:05:29.040 the hand with the brain, which means to achieve this body ownership with an artificial hand. 00:05:29.040 --> 00:05:33.940 That means that the artificial hand is in contact with the human skin because 00:05:33.940 --> 00:05:40.080 it's implanted on the stump of the subject, and we want to develop. 00:05:41.100 --> 00:05:46.200 A bridge between the hand and the brain in order to provide this exchange of 00:05:46.200 --> 00:05:50.540 signals between the hand and the brain. So this is the main point. 00:05:51.271 --> 00:05:56.371 So a lot of people are building robot hands, but what is special or different 00:05:56.371 --> 00:06:01.171 about a robot hand that could be worn by a person as a prosthesis? 00:06:01.331 --> 00:06:08.351 First of all, you have to develop a hand which is cosmetic, which means that 00:06:08.351 --> 00:06:11.591 the external appearance must be good. 00:06:11.591 --> 00:06:22.311 You cannot think that the user were sort of non-static hand with iron or similar, 00:06:22.591 --> 00:06:27.151 reminding us to Terminator or to fiction robots. 00:06:27.151 --> 00:06:32.571 So the user wants a hand which is similar to the natural hands and wants to 00:06:32.571 --> 00:06:41.931 have social interaction and satisfaction in contacting people and in being in the external world. 00:06:42.491 --> 00:06:47.731 So first of all, the hand should be with a cosmetic glove, then must be lightweight 00:06:47.731 --> 00:06:53.891 because it is implanted on the stump, so the mechanical interaction between 00:06:53.891 --> 00:06:56.451 the hand and the stump must be good. 00:06:56.711 --> 00:07:01.311 And also there is the problem of power consumption. If the hand is active, 00:07:01.631 --> 00:07:08.511 the power internal to the hand is coming from the battery, and And in order 00:07:08.511 --> 00:07:12.731 to provide eight hours or eight operating hours, 00:07:13.071 --> 00:07:18.411 the motors and the activities of the hand must be provided by a single battery. 00:07:18.691 --> 00:07:22.691 So you cannot think to plug the hand during normal activities. 00:07:23.151 --> 00:07:28.771 For example, during working activities or entertainment activities or similar, 00:07:29.011 --> 00:07:32.931 you must provide eight hours or normal operation. 00:07:32.931 --> 00:07:40.691 That means that the system must monitor the energy consumption and try to distribute 00:07:40.691 --> 00:07:46.151 the energy in an appropriate way without losing energy. 00:07:46.151 --> 00:07:48.651 And also that there is another problem. 00:07:48.711 --> 00:07:50.791 It's related to sensory feedback. 00:07:50.911 --> 00:07:54.251 So the hand is in contact with the environment. 00:07:54.391 --> 00:07:59.491 So you should provide the sensors for monitoring the interaction between the 00:07:59.491 --> 00:08:00.971 hand and the environment and 00:08:00.971 --> 00:08:07.711 also sensors for providing proprioception and body ownership to the user. 00:08:08.251 --> 00:08:12.511 So you mentioned body ownership and you described a really interesting experiment 00:08:12.511 --> 00:08:18.591 in your talk where people, subjects in the experiment somehow get a sense of 00:08:18.591 --> 00:08:21.591 body ownership for a hand that's made out of rubber. 00:08:22.151 --> 00:08:24.191 Now, how can that come about? 00:08:24.787 --> 00:08:29.407 Yes, this is an interesting example, well known in neurophysiology, 00:08:29.767 --> 00:08:38.647 and we are trying to export this typical experiment of neurophysiology into robotics. 00:08:38.967 --> 00:08:45.487 That is an experiment where a subject is watching a cosmetic hand, 00:08:45.747 --> 00:08:48.727 a rubber hand, which is close to the natural hand. 00:08:48.947 --> 00:08:56.847 And the subject, by doing some interaction, by brushing on the hand, 00:08:56.907 --> 00:09:02.587 on the rubber hand, on the natural hand, the subject is looking only at the artificial hand. 00:09:02.727 --> 00:09:07.887 If the hand is really similar to the natural hand, the subject starts to develop 00:09:07.887 --> 00:09:13.847 a sort of body ownership for the rubber hand, which starts to belong to the subject. 00:09:13.887 --> 00:09:20.887 So when you stab Stab the rubber hand the subject feels that you are stabbing the real hand. 00:09:21.267 --> 00:09:26.407 So it's something about the correlation of the sensory signals that I get from 00:09:26.407 --> 00:09:32.327 touch on my real hand and seeing somebody stroking the rubber hand that gives 00:09:32.327 --> 00:09:35.407 me this illusion that the rubber hand is part of my body. 00:09:36.207 --> 00:09:41.567 Yes. And so it's a mixture of the visual signal and the tactile signal. 00:09:41.567 --> 00:09:47.127 Okay, the task for neurophysiologists, neuroscientists is to understand if it 00:09:47.127 --> 00:09:56.747 is only related to vision and involving only the areas that are related to vision in the brain. 00:09:57.027 --> 00:10:02.647 And there are some papers demonstrating that also the sensory motor areas of 00:10:02.647 --> 00:10:04.487 the brain are involved in the task. 00:10:04.607 --> 00:10:08.567 That means that there is this involvement of perception for us. 00:10:08.567 --> 00:10:13.687 We are roboticists, we are much more interested in the implications for that. 00:10:13.727 --> 00:10:22.767 That means that we are in principle able to provide a sort of rubber hand, which really. 00:10:24.507 --> 00:10:30.967 Can be introduced in the loop with the brain and also the sensory motor areas 00:10:30.967 --> 00:10:36.067 of the brain can be involved in perceiving the hand as a natural hand. 00:10:36.067 --> 00:10:41.667 So, what we would like to develop is a sort of cosmetic active hand, 00:10:41.867 --> 00:10:44.847 which is replacing the hand which is missing. 00:10:45.067 --> 00:10:53.347 So it's a step forward because in that case the subject is without hand and 00:10:53.347 --> 00:10:57.867 must own the new hand, which is artificial. 00:10:57.867 --> 00:11:04.767 So, existing prostheses are presumably made out of plastic or wood or whatever. 00:11:05.307 --> 00:11:11.127 But if I was wearing such a hand, presumably I still have some feel of ownership 00:11:11.127 --> 00:11:15.267 over the hand because when I touch something with a prosthesis, 00:11:15.749 --> 00:11:19.409 I would still get feedback through my arm from that contact, 00:11:19.669 --> 00:11:22.749 even though it's mediated by a large piece of plastic. 00:11:23.189 --> 00:11:30.209 But you plan to go beyond that and give some direct feedback using sensors on the hand? Yes, yes. 00:11:31.689 --> 00:11:38.489 The stamp is providing some direct feedback about the force and the movement 00:11:38.489 --> 00:11:44.249 of the hand because the proprioception on the stump of the subject is still there. 00:11:47.069 --> 00:11:50.089 And so the subject can feel something in the stump. 00:11:50.289 --> 00:11:55.909 But we would like to do more because what we would like to do is that in the 00:11:55.909 --> 00:12:03.549 socket, which is the interface between the hand and the stump, The socket is the place, 00:12:03.769 --> 00:12:09.449 the mechanical interface between the subject body and the hand. 00:12:09.569 --> 00:12:16.849 In the socket we would like to put some new motors and these motors are intended 00:12:16.849 --> 00:12:19.689 to provide some stimulation on the skin. 00:12:19.689 --> 00:12:29.109 So we made a lot of experiments in understanding how far we can go in delivering 00:12:29.109 --> 00:12:35.689 different stimuli by varying the frequency and also the amplitude of the stimuli. 00:12:35.689 --> 00:12:41.969 And in order to assess the discriminational ability of the subject, 00:12:42.249 --> 00:12:48.429 what we would like to do is to make a sort of mapping between the activity of 00:12:48.429 --> 00:12:53.849 the hand and the stimuli that we provide internal to the socket, to the skin. 00:12:54.069 --> 00:13:02.729 So the subject can learn about and can learn to recognize the activity of the 00:13:02.729 --> 00:13:06.089 hand thanks to this mapping. 00:13:06.389 --> 00:13:11.349 For example, we touch the finger and we provide some stimuli and the subject 00:13:11.349 --> 00:13:18.309 feels on the stamp a stimuli according to the intensity of the contact force or similar. 00:13:19.129 --> 00:13:24.829 Or the force, the intensity of the force can be modulated by providing some 00:13:24.829 --> 00:13:30.569 frequency signals to the stamp of the subject. 00:13:30.569 --> 00:13:35.169 So this activity is in fundamental in order to provide this sensory feedback 00:13:35.169 --> 00:13:38.929 and to develop this body ownership for the subject. 00:13:39.549 --> 00:13:44.309 So you say that they feel this on the skin of the arm, but the sense is on the 00:13:44.309 --> 00:13:48.069 tip of the robot finger. Now, is it possible? 00:13:48.814 --> 00:13:53.594 That, after wearing this device for some time, even though the stimulus is on 00:13:53.594 --> 00:13:56.994 the arm, the stimulus is actually coming from the finger. 00:13:57.114 --> 00:14:01.514 So is it possible that the experience of the subject migrates so that they, 00:14:01.634 --> 00:14:06.574 like in the rubber hand illusion, they have the experience that the stimulus 00:14:06.574 --> 00:14:07.794 is on the tip of the finger? 00:14:08.094 --> 00:14:13.114 Yes, this is exactly the kind of experiment that we are doing now. 00:14:13.314 --> 00:14:16.874 We did some experiments with the subject with 00:14:16.874 --> 00:14:19.814 an implanted interface in the 00:14:19.814 --> 00:14:26.134 peripheral nervous system by stimulating the fibers of the nervous system and 00:14:26.134 --> 00:14:32.694 providing a sort of stimulation we can elicit some sense of the fingers that 00:14:32.694 --> 00:14:39.514 the subject had before having the amputation. 00:14:39.674 --> 00:14:46.214 So in principle, we think that it's possible to provide this kind of mapping 00:14:46.214 --> 00:14:51.334 and try to investigate on the reaction of the subject. 00:14:51.534 --> 00:14:57.314 But in order to do that, we must provide a sort of flexible interface with different 00:14:57.314 --> 00:15:02.494 vibrators and stimulators internal to the socket. 00:15:02.494 --> 00:15:06.114 And we would like to make experiments with amputees 00:15:06.114 --> 00:15:09.034 in order to understand how far we can go 00:15:09.034 --> 00:15:12.794 because in principle we have some models from neurophysiology 00:15:12.794 --> 00:15:15.914 we know from different experiments but 00:15:15.914 --> 00:15:20.014 the data with human beings are very few and 00:15:20.014 --> 00:15:29.214 thanks to the experiment which is not non-invasive it's not requiring surgical 00:15:29.214 --> 00:15:37.334 intervention we think that we would find some amputees, volunteers. 00:15:37.614 --> 00:15:39.834 In order to make experiments on that. 00:15:40.414 --> 00:15:46.654 And one of the things that we're taking advantage of here is that the brain 00:15:46.654 --> 00:15:49.534 maps for the body are very adaptive. 00:15:49.734 --> 00:15:55.174 And I guess they can change and learn to incorporate something which is actually 00:15:55.174 --> 00:15:59.674 an artificial limb as a replacement for a real limb. 00:16:00.986 --> 00:16:08.766 Yes, we rely on that. Neuroscientists are telling us that the ability of the 00:16:08.766 --> 00:16:11.606 brain to adapt is enormous. 00:16:12.026 --> 00:16:19.006 We know that the motivation for the amputee, for the subject to be in the loop 00:16:19.006 --> 00:16:23.786 and to achieve this kind of body ownership is strong. 00:16:23.786 --> 00:16:31.786 We know that subjects using prosthetic hands are waiting for a new development, 00:16:31.906 --> 00:16:40.686 and feeling the external environment is one of the major points when they raise 00:16:40.686 --> 00:16:44.266 issues about future development of the hands. 00:16:44.406 --> 00:16:47.726 They want to feel the external environment. environment so 00:16:47.726 --> 00:16:50.546 we think that because we know that 00:16:50.546 --> 00:16:53.906 the motivation from from the subject is fundamental in 00:16:53.906 --> 00:16:56.706 this kind of research so we we know 00:16:56.706 --> 00:16:59.686 that we can we can do something for them 00:16:59.686 --> 00:17:03.426 but we need some experiments so sometimes 00:17:03.426 --> 00:17:06.766 people that have lost a limb report a 00:17:06.766 --> 00:17:09.606 phantom limb experience where they feel they still 00:17:09.606 --> 00:17:12.826 have the limb and that it's twisted or in some painful position 00:17:12.826 --> 00:17:16.086 so is it possible that a prosthetic would 00:17:16.086 --> 00:17:19.306 be able to help them get over that phantom limb experience 00:17:19.306 --> 00:17:22.266 yes we know that we already developed a 00:17:22.266 --> 00:17:28.246 sort of virtual reality system based on our hand in order to investigate on 00:17:28.246 --> 00:17:38.046 people suffering from phantom limb pain and we hopefully we will be able When 00:17:38.046 --> 00:17:42.106 we will know more about this artificial body ownership. 00:17:42.486 --> 00:17:50.806 We will be able also to face this challenge, which is still a challenge for some individuals. 00:17:50.826 --> 00:17:56.346 They are suffering this kind of pain, which cannot be treated with normal therapy. 00:17:56.626 --> 00:18:02.826 So we know that this can be also a side effect, a positive side effect of our research. 00:18:03.443 --> 00:18:08.983 So another kind of robot technology that you're developing is not so much a 00:18:08.983 --> 00:18:12.843 prosthesis, but an add-on to the body that you call an exoskeleton. 00:18:13.103 --> 00:18:18.703 Could you tell me what that is? The exoskeleton is a sort of external structure 00:18:18.703 --> 00:18:22.163 which is coupled to the joint. 00:18:22.323 --> 00:18:28.943 So in the case of upper limb is a sort of external suit with articulated system 00:18:28.943 --> 00:18:33.303 which is providing assistance to the limb. 00:18:33.803 --> 00:18:41.423 So it's providing, for example, additional torque in order to accomplish some task in the upper limb. 00:18:41.423 --> 00:18:47.103 For example, in reaching an object, when the subject is weak after suffering 00:18:47.103 --> 00:18:52.883 a stroke or similar pathology and has the limb which is weak, 00:18:53.163 --> 00:18:58.623 this kind of external structure is able to provide some force to perform the task. 00:18:58.903 --> 00:19:04.823 So that can be done for assistance, which means a sort of tool to provide some 00:19:04.823 --> 00:19:11.063 additional force for the subject, or can be seen as a therapy. 00:19:11.423 --> 00:19:16.923 What is called the neurorehabilitation therapy, which means that this, 00:19:17.103 --> 00:19:23.163 by doing some exercises in combination with the exoskeleton. 00:19:23.843 --> 00:19:33.163 Like exercise in reaching an object in the space, so flexo-extension of the elbow or the wrist, 00:19:33.443 --> 00:19:43.643 the subject is recovering the synergies and recovering the ability to move again the limb. 00:19:43.923 --> 00:19:51.763 And these are additional to the normal, ordinary physical therapy and must be 00:19:51.763 --> 00:19:56.403 delivered by a therapist in combination with normal therapy. 00:19:56.543 --> 00:20:04.363 And we know that there are some data on behavioral experiments and also clinical 00:20:04.363 --> 00:20:10.203 data telling us that this kind of exercise size in combination with a robot 00:20:10.203 --> 00:20:12.183 are good for restoring movements. 00:20:13.112 --> 00:20:17.872 So I'm imagining some device that's strapped to my forearm, and then there's 00:20:17.872 --> 00:20:22.492 a hinge at the elbow, and then there's a piece on the upper arm as well. 00:20:22.732 --> 00:20:27.152 Yes. And what kind of patients would benefit from using this? 00:20:27.412 --> 00:20:36.472 This kind of therapy in general is aimed at restoring motion in subjects with post-stroke. 00:20:36.472 --> 00:20:45.592 Stroke so with weak limb after stroke and they in general they have problems in moving again their. 00:20:46.072 --> 00:20:53.112 Limb and they are doing some therapy for learning again to control the movement 00:20:53.112 --> 00:20:59.452 for the first step is for for reaching an object in the space so the first step 00:20:59.452 --> 00:21:04.112 is to reach the object the second is to open the hand and use the hand. 00:21:04.312 --> 00:21:10.092 So these kind of exercises are repetitive, which means that they have to do 00:21:10.092 --> 00:21:14.832 the exercise several times, and it's an exercise following a trajectory. 00:21:14.832 --> 00:21:18.492 For example, by holding a sort of handle in the space, 00:21:18.832 --> 00:21:25.892 they have to hold the handle of the joystick and move the joystick in the space 00:21:25.892 --> 00:21:29.872 according to a trajectory by following a prescribed trajectory, 00:21:29.872 --> 00:21:35.972 trajectory and doing this kind of reaching and moving in the space several times 00:21:35.972 --> 00:21:40.152 is useful in order to obtain motor recovery. 00:21:40.452 --> 00:21:46.692 So doing that with a therapist is useful but doing that with a robot assisting 00:21:46.692 --> 00:21:52.792 the subject which is able to provide the same force, the same pattern and in a very. 00:21:54.363 --> 00:21:57.403 Control the environment can provide 00:21:57.403 --> 00:22:00.203 a better recovery so these are people 00:22:00.203 --> 00:22:03.743 that have got completely normal limb function and 00:22:03.743 --> 00:22:07.423 the problem that they have is in the brain yeah where the stroke has damaged 00:22:07.423 --> 00:22:11.423 the part of the brain which would normally control the arm yes so we're counting 00:22:11.423 --> 00:22:17.723 on brain plasticity to find another way of controlling the arm so if we can 00:22:17.723 --> 00:22:20.823 in a repeatable way, move the arm using the robot, 00:22:21.623 --> 00:22:24.623 eventually the brain will take over. Is that how it works? 00:22:24.803 --> 00:22:31.983 Yes, it's like that. It's more complex because also the arm is weak because it's not used anymore. 00:22:32.943 --> 00:22:41.043 And the final objective is to recover also the correct synergies in the muscular 00:22:41.043 --> 00:22:43.343 structure of the subject. 00:22:43.423 --> 00:22:47.663 So it's not only a problem of brain, but it's also a problem of muscles and 00:22:47.663 --> 00:22:55.323 structures and to the correct movements in the space by moving the muscles, but it is exactly that. 00:22:55.583 --> 00:23:02.183 And from our point of view, in robotics, we have to develop such kind of systems 00:23:02.183 --> 00:23:06.203 which are acting in parallel to the natural limb. 00:23:06.323 --> 00:23:13.643 So it's completely different from prosthetics, where you replace a limb which 00:23:13.643 --> 00:23:16.003 is missing, so it's not a parallel. 00:23:16.203 --> 00:23:20.403 A system is substituting the hand. 00:23:20.643 --> 00:23:27.203 In that case of exoskeleton, it's much more complicated because you are introducing 00:23:27.203 --> 00:23:29.783 a system which is acting in parallel. 00:23:30.163 --> 00:23:35.303 So you have two manipulators in parallel. One is the natural limb and one is 00:23:35.303 --> 00:23:42.843 the artificial limb and they are moving together and this is the real symbiosis, which means that the. 00:23:43.729 --> 00:23:49.829 Natural and the artificial manipulators must share the same goal and do the same actions. 00:23:49.949 --> 00:23:54.909 If this symbiosis is not working, the system is failing. 00:23:55.329 --> 00:24:00.449 So this is a really big research problem, that if I have an artificial hand 00:24:00.449 --> 00:24:05.509 and I'm trying to control it using my brain and the hand isn't understanding 00:24:05.509 --> 00:24:09.469 my intentions, it could do all sorts of things that maybe I don't want it to do. 00:24:09.609 --> 00:24:13.309 So how are we going to avoid that? It's very difficult. 00:24:13.949 --> 00:24:20.249 This is our problem in developing shared control, which means that you have 00:24:20.249 --> 00:24:26.269 the subject, which is who is supervising the hand, and then there is an autonomous 00:24:26.269 --> 00:24:28.529 control system internal to the hand. 00:24:28.529 --> 00:24:34.869 And so for robotics, the most important thing is to have the system working 00:24:34.869 --> 00:24:36.869 together with appropriate harmony. 00:24:37.289 --> 00:24:41.249 And so when the system for us is failing, 00:24:41.369 --> 00:24:47.349 when it is not executing exactly what the subject is willing to do, 00:24:47.429 --> 00:24:54.009 and in that case, the subject wants to open the hand and the hand is not opening. 00:24:54.009 --> 00:24:57.089 And that is frustrating for the subject and 00:24:57.089 --> 00:25:00.729 in that case the subject refuses to 00:25:00.729 --> 00:25:04.829 use the hand because the hand is not good 00:25:04.829 --> 00:25:11.569 in performing the task and also for social interaction for the subject is not 00:25:11.569 --> 00:25:17.409 good if the sub if the hand is opening or closing or doing things when the the 00:25:17.409 --> 00:25:21.789 subject doesn't want in causing some embarrassing. 00:25:22.889 --> 00:25:24.889 Situation for the subject. 00:25:25.169 --> 00:25:31.049 So we know and we in our experiments we measure the, 00:25:31.801 --> 00:25:37.381 success percentage of tasks. So which means that if the system is good, 00:25:37.501 --> 00:25:41.801 you can go to 90% of success rate. 00:25:42.101 --> 00:25:48.221 If you go to 60%, the system is not good because that means that you have 40% 00:25:48.221 --> 00:25:55.901 of cases when the hand is not executing the intended task and the system is not good. 00:25:59.881 --> 00:26:06.201 And Because one of the goal is that they use the burden for the user to learn 00:26:06.201 --> 00:26:09.801 and then to use the system must be low. 00:26:10.441 --> 00:26:16.841 So in one of the Star Wars films, I'm sure you know, Luke Skywalker loses a 00:26:16.841 --> 00:26:19.001 hand in a fight with Darth Vader. 00:26:19.121 --> 00:26:23.841 And he goes on a hospital ship and they replace it with an artificial hand. 00:26:24.001 --> 00:26:29.001 Is that science fiction or is it something that we might achieve one day? 00:26:29.001 --> 00:26:30.921 I think we will achieve that. 00:26:31.181 --> 00:26:34.381 I'm sure. I'm pretty sure we will achieve. I don't know when. 00:26:35.301 --> 00:26:40.821 I don't know if I will be alive, but I think we will achieve that. 00:26:40.941 --> 00:26:42.881 And this is exactly our objective. 00:26:43.501 --> 00:26:47.681 So you have a wider approach. 00:26:47.881 --> 00:26:52.141 So you described the exoskeleton as a way of doing some kind of therapy, 00:26:52.281 --> 00:26:56.781 but you're interested more generally in how we could use robots in therapy. 00:26:57.041 --> 00:27:01.341 Are there some other ideas related to that that you're developing? 00:27:02.341 --> 00:27:07.141 We are also developing artificial skin and we are interested in developing the 00:27:07.141 --> 00:27:11.541 skin because the skin is fundamental in order to provide a safe interaction. 00:27:11.541 --> 00:27:18.141 It is not the skin of the robot, it is also the skin between the robot and the subject, 00:27:18.281 --> 00:27:24.601 which is much more important, because the interface between the robot and the 00:27:24.601 --> 00:27:33.541 human body must be monitored in order to avoid situations where you can provide 00:27:33.541 --> 00:27:38.001 too much force or too much stress to the skin of the subject. 00:27:38.001 --> 00:27:43.821 So we are interested in developing the skin of the robot for what concerns the 00:27:43.821 --> 00:27:49.801 interaction with the environment, which means the properties of the object that 00:27:49.801 --> 00:27:51.421 is manipulated or similar. 00:27:51.641 --> 00:27:57.761 And also we are interested in developing the skin between the robot and the human body. 00:27:57.961 --> 00:28:04.041 And in this kind of skin, we are also including the stimulation system that 00:28:04.041 --> 00:28:05.701 we were illustrating before. 00:28:06.461 --> 00:28:10.221 The skin between the robot and the human body, so this is something that goes 00:28:10.221 --> 00:28:12.941 inside the sleeve that fits over the arm? 00:28:13.081 --> 00:28:15.781 Yes, inside the sleeve, yes. Right, okay. 00:28:16.081 --> 00:28:21.841 And that will be sort of a tight fit but not invasive? Tight fit but not invasive. Yeah. 00:28:22.520 --> 00:28:31.880 And this fit is providing you the distribution of forces and stress on the human skin. 00:28:32.080 --> 00:28:38.340 So it's fundamental because one of the reasons why the subjects are refusing 00:28:38.340 --> 00:28:46.080 system is that they are providing arm to the skin and pain and even too much 00:28:46.080 --> 00:28:50.360 force in some particular positions. 00:28:50.360 --> 00:28:57.480 And this kind of force is dangerous to provide pain for the subject. 00:28:57.760 --> 00:29:07.720 So we know that the good fitting is fundamental for wearing the prosthesis and also the exoskeleton. 00:29:08.020 --> 00:29:12.660 So the big research challenge is right now, because it's still in the laboratory, 00:29:12.900 --> 00:29:14.000 this technology, isn't it? 00:29:14.120 --> 00:29:18.360 You're not yet at the stage of being able to... We are developing experiments 00:29:18.360 --> 00:29:24.220 in the laboratory and also we are doing experiments in clinical facilities with 00:29:24.220 --> 00:29:29.320 amputees in order to make trials for our interfaces. 00:29:29.480 --> 00:29:31.400 So it's not commercial at all. 00:29:31.680 --> 00:29:33.740 What do the amputees report about? 00:29:34.220 --> 00:29:40.420 It depends on the experiments. So their reports is useful because it's a feedback for our design. 00:29:40.420 --> 00:29:45.740 So our design is cyclic, which means that we make the prototype, 00:29:46.040 --> 00:29:49.320 we make the experiments, and then we have the feedback on the design, 00:29:49.540 --> 00:29:50.440 and that is fundamental. 00:29:50.440 --> 00:29:59.180 Some ideas are already transferred to the commercial areas, but it's not directly 00:29:59.180 --> 00:30:01.120 that we develop a new hand. 00:30:01.200 --> 00:30:08.200 We develop some ideas, some details, also some research lines and guidelines 00:30:08.200 --> 00:30:11.280 for designs and also mechanism and patterns. 00:30:11.560 --> 00:30:17.760 And these are pieces that are useful for commercial hands and for new products. 00:30:17.760 --> 00:30:20.780 It's interesting that you say it's an iterative process 00:30:20.780 --> 00:30:24.180 so you're relying on user feedback and that's 00:30:24.180 --> 00:30:27.040 one of the things that guides you and where you go with the design process 00:30:27.040 --> 00:30:30.340 yes all the development in medical devices 00:30:30.340 --> 00:30:35.220 is done like that by making 00:30:35.220 --> 00:30:38.500 experiments with subjects and users and going 00:30:38.500 --> 00:30:41.880 back in changing the 00:30:41.880 --> 00:30:47.620 design according to the user assessment okay well this is all sounds really 00:30:47.620 --> 00:30:53.260 exciting I wonder if you have any general predictions for how this technology 00:30:53.260 --> 00:30:58.340 is going to change the way our society operates in the 10-15 years from now 00:30:58.340 --> 00:31:01.560 I think that the Star Wars prediction is enough for the moment. 00:31:02.540 --> 00:31:08.700 That's a pretty ambitious one The hand plug and play Thank you very much and 00:31:08.700 --> 00:31:10.400 thank you for coming to BCBT. 00:31:13.978 --> 00:31:20.098 The CSN podcast was produced by the Convergent Science Network of Biometrics 00:31:20.098 --> 00:31:26.878 and Biohybrid Systems, a project funded by the European 7th Research Framework Programme. 00:31:28.018 --> 00:31:33.378 For more interviews, recorded lectures or upcoming conferences in the field 00:31:33.378 --> 00:31:39.618 of biometrics and biohybrid systems, go to csnnetwork.eu. 00:31:40.400 --> 00:31:47.760 Music.