This is the convergent science network podcast,
leading researchers in the domain of neuroscience brain theory and technology
are interviewed by paul verscher and tony prescott,
so this is tony prescott for the convergent science network podcasts from the
barcelona Summer School on Cognition, Brain and Technology in 2011.
And I'm here with Housang Hu from the University of Essex Department of Computer
Science and Electronic Engineering, who is one of the speakers that we have
this week on the general topic of biomimetic robots.
So, Housang, can you tell me a little bit about your background and how you became a roboticist?
I came to this country, the UK, 25 years ago.
Before that, I was working at a Chinese university.
My major is automation. When I joined Oxford University, I worked with Professor
based at McBrady, I started doing robotics.
So that will be back to 1987.
So I started working with industry, try to produce the new generation of AGV,
for manufacturing of the industry.
So started from there, and then I developed my interest in my research in the
biomimetics or biologically inspired robotics.
So you have some background in industrial robotics, but your work in recent
times has been more directed towards service robotics and field robotics.
So tell me about your interest in field robotics. How did you get involved in underwater robots?
Yeah, underwater robots is one kind of field robotics. And I'm also working
on the wheeled and tracked robots and the flying robots as well.
The reason I got into the underwater robots is because we have an industry approaching
to us and trying to create robotic fish for the Linden Aquarium.
Because by law they have many fish species cannot be displayed in the aquarium environment.
So they want to create a robotic equivalent species to demonstrate how.
These fish species can swim in the sea, in the aquarium environment.
So you started off really building robot fish for exhibits. Yes.
But you've started now moving them out into the ocean?
Yeah, just the last two years we have support from the EU and FP7 framework,
and we try to develop robotic fish to patrol ports.
And monitoring the ship oil leaking problem and also other kinds of pollution
may cause damage to the sea life.
So this is a three-year project. We work with a number of partners in the EU,
like BMT and Irish partners and also tennis and also Spanish partner as well.
So what would be the advantage in open sea of a robot fish compared to a conventional
submarine-style robot?
The fish, as we know, first thing is swimming peacefully without disturbing the environment,
because the submarine or ship-like, they use the thrust to generate a lot of
turbulence in the the water, which is no good for sea life,
and also it may be disturbing pollution area as well.
So fish is more peaceful and also more flexible in many of the narrow pathways, for example.
And what about energy efficiency? Is it similar or better than a submarine?
Yeah, as we know, the submarine type of the man-made ships' efficiency is up to 60%.
But the fish, theoretically, is higher than the man-made vehicle because after
millions of years evolution,
fish is an excellent swimmer in the water.
They have, of course, the body shape and make the fish swimming much efficient.
And also, So the muscle movements also make the fish is very fearful as well.
So you're copying the streamlined shape of the fish and you're also copying
the type of movement it's making in order to propel itself.
Is the robot biomimetic in any other way, for instance, in terms of the components
that you're using to build it?
Within the robot, are you using conventional motors?
Yes. Ideally, we hope many researchers actually start using some kind of artificial
muscle to try to mimic the real muscle movement of the fish.
But at the moment, artificial muscle is not mature enough for us to use it.
So what we did, we used normally traditional conventional motors to drive the fish movements.
And what about the pattern generation systems that are generating the swimming movement?
Yes, that's what we call the central pattern generator.
And that's generated by software.
And after we actually have used the camera to capture the real fish movements,
We analyze the movements and then to decide the phase difference because the real fish use muscles.
So muscles can be considered as continuous actuators.
But we use discrete motors. Motors have different sizes.
Even if we choose a small size, we only can accommodate four or five or six
motors in one fish. The fish we built is about 60 cm in total.
So half the body with the four or five motors, that's what we call discrete
joints, to actually generate the S-movement of the real fish.
Okay. So is it a segmented body? Yeah, segmented body.
And each segment is actually one motor. Right.
And what about the fins? How many of the fins are you replicating,
and how important are they?
Yeah, the fins actually have the main dorsal fin, for example,
which generates the proponent force for our fish.
And of course, fish also have anal fins, have the bacterial fins,
and try to make sure the balance of the fish. So you'd be using those for steering and balance?
Yeah, that's only for the stationary, static balance.
But once the fish is in motion, we really use the motor to drive it.
What about sensors? What kind
of sensors do you use? We have a number of sensors inside our fish body.
One is a gyroscope to actually measure the a posture of the fish in the water
to maintain the balance.
We also had a force sensor to measure the depth in the water to decide.
3D swimming, and also we had an accelerometer to measure the speed of the fish
as a kind of odometer, odometry.
We also have obstacle detection sensor infrared to detect anything in front
of the fish in order to avoid it.
Of course, we had a voltage monitoring sensor to see whether the energy is enough or not.
If the energy is not enough, we're going to float on the surface.
Right. Just in case.
Okay. So how do you control your height within the water?
To control the height in the water, as we know, real fish have bladder.
They can be stationed in one level without motion.
So we mimic the bladder function,
use the water tank inside of the fish by pumping into the water or pumping out
of the water to actually change the weight of the fish, change the buoyancy.
So this is only statically, but dynamically, you want to change the level of
the fish, go up or go down.
We actually use one motor to drive the central gravity towards the head or towards
the tail. in order for the fish to swim up or swim down.
Very, very agile and very, very speedy operation.
So I'm thinking about pollution monitoring. So I can imagine that you might
just have a boat and then you could trail something out of the side of the boat on the end of a line.
How would having a robot that's submerged be better than other ways of doing that?
Yeah, because it depends on the depth of the water you are going to monitor.
For example, we have the port in Spain.
It's called Gijón, which has a depth of 30 meters.
So the pollution, if we use the ships on the surface, surface may not be able
to detect anything beyond 10 meters or seabed, for example.
So to use the fish, we can automatically change the depth going to the.
Seabed to actually detect anything, pollution is there or not,
especially we want to search the source of the pollution.
So once you put your fish in the ocean, how does it know where to go?
We actually use the structured environment. We put a buoy surrounding the port
to send out the ultrasonic signal,
sonar signal, and the fish has the receiver on board to receive the signal.
This is kind of like underwater GPS?
Yes, exactly. Right, okay. So it knows where it is, but you're never worried
about that it won't come back?
Oh yes, we actually have to prevent in case of underwater GPS malfunction, fish may get lost.
So in that case, fish going to floating on the surface, we use GPS.
Right, so you can use the real GPS, satellite.
Yes, because the antenna will be floating over the surface of the water,
then GPS signal can be received, and then we can navigate back to the home position.
So how close do you think we are
to having a technology that might be commercialized for this sort of use.
Yeah, this project we are running now is third year.
So we hope we're going to test this in the port next year, next May.
And after that, I think we need a couple more years to refine it and improve the performance.
I imagine maybe three or four years
time From now on, we could have some commercial products on the market.
And what other application areas might there be besides pollution monitoring?
And also security. For example, some of the coastlines, they want to secure for different reasons.
So we can use this kind of fish technology actually to detect any illegal ships.
Or any illegal animation. Right, yeah.
So it could be part of the Coast Guard, or it could be part of the Customs and
Excise. Exactly. Those kinds of things. Yes.
Okay, so changing subjects a little bit, I know that you have an interest in
building assistive robots for people with disabilities.
Can you give me an example of the kind of projects that you're doing there?
Yes. This is to try to help disabled and aging population because when people
are getting old or people are disabled,
they have difficulty to move around or independent life has been out of question.
So what we try to help is mobility, to help these people move around,
integrate with society,
and they can see the doctor if they want, they can see their relatives or friends go out if they want.
So that's the main purpose. As we know currently,
commercial wheelchairs are only driven by joysticks, and many users have difficulties
to navigate with joysticks, even go through the doorway.
For example, users suffer from Parkinson's disease, their hand is shaking many
times, sometimes they may not be able to use joysticks.
And some of disabled people, they may be suffering no hand, for example.
It's also difficult to use.
So in that case, we think about the other means.
For example, use the voice, use the gesture, use the muscle signal, even for.
Disabled users. We use like Hawking, Professor Hawking, and cannot move any limbs.
And then, if their brain functions well, we use EEG signal to control wheelchair movements.
So you want to record EEG, sort of an electroencephalogram.
So this is electrical activity from nerves, which you record by placing sensors on the skin. Yes.
And whereabouts would you put the sensors?
We actually put the sensors in the skull, on the motor cortex area.
And the users actually can just imagine.
His hand movements, his or her neck movements, then generated control commands
for the wheelchair to be controlled.
So this is the part of the brain, the motor cortex, which is anyway involved in motor control.
Yes. And so you're hoping that by recording through the skull some of the electrical
activity, you may be able to distinguish when the user wants to,
say, turn left or wants to turn right.
That's right. So, I mean, does this work?
I mean, is it possible really to read these things from that kind of brain activity?
Right now, we actually have some results.
Although it's not real-time, and sometimes the success rate is maybe up to 70%.
So, we think the key problem is the sensor. Yeah.
Right now, we use non-invasive electrodes on the skull, and the signal is very weak.
So we have to improve the sensor technology.
We hope by next few years, if the EEG sensor can be further improved,
then we can use such technology.
This is one thing. And also, another thing is we think maybe multiple modality,
not just EEG, and also facial,
also other biosignal can be held.
Combined with EEG if possible.
What other kind of signal are you thinking of? Like facial, emotion,
facial, eye movements, and mouth movements, simply that clue also can reflect people's needs.
Okay, so if I'm a wheelchair user and I'm wired up with these electrodes on
my head, what do I have to do to make the wheelchair move?
Is it enough that I think I want to go left and then it will turn?
Or do I have to learn the relationship between my thought patterns and what
the wheelchair does? Yes, we do need training.
We need every user to train to generate imaginary signals.
For example, a user can think about using the right hand, or using the left hand, or using the legs.
So we we can generate the electric pattern on the transducers.
Then we need to train our, we use the neural network and we need to train all the,
all the parameters so you train so
you record uh these eeg patterns yeah
for people thinking about a particular kind of movement yeah and you train a
neural network with those patterns to distinguish left thoughts about moving
left from thoughts about moving left right yeah and um so roughly speaking um
how much training data would you need i mean from a person would you need to several
hours of EEG or could you do it quite quickly?
We normally doing several hours. Right. We normally doing several hours.
And with that person having trained on their data, would it be stationary or
would you need to train again if you came back two weeks later,
would it be different or would it still work?
Yeah, that's a good question. Actually, people, they have diverges because they are mental.
Condition. For example, if a user sleeps very well last night,
so today they may function very well.
If they don't sleep very well last night, they may not get a useful signal today.
So in that sense, our,
algorithm have to be more adaptive. So we call it online learning.
So what we want to do, not just offline, and also we want to online continuously
change the parameters of our neural network,
for example, to adapt to new stages of the mental stage of the users.
Okay, so where have you got to in terms of testing these?
Have you got people driving around with with wheelchairs and controlling them
just with their thoughts? Have you got some prototype results now?
Yes, we do, we did. And we have, right now it's health subject.
It's all, in fact, our PhD students.
So next stage we hope to get some disabled people or trial in the real users.
So how good are the students then? And can they, for instance,
drive the wheelchair through a narrow doorway?
Yeah, and they can navigate in the indoor environment in general, no problem.
Without bumping into things? Yes, because the control system has prevented bumping into anything.
Use the sensors. Oh, so there are sensors on the wheelchair as well to prevent collisions. Yes.
So that you have a backup in case the brain reading system is not working very well. Exactly.
So the wheelchair is independently sensing the environment and making some decisions
about when to stop or when to go.
And then really what you're doing is reading the person's brain state in order
to bias what the wheelchair is going to do.
Yeah, either EEG signal or muscle EMG signal. and also maybe other signals like
voice, like hand gesture or head gesture, use the vision-based. This is all...
We call the hands-free control. It's on top of the navigation system.
So all this human intention, biosignal I mean, to detect by the sensors,
use to control the wheelchair motion.
But at the bottom of the wheelchair control system, we have safety control system
there with the laser scanner, with the ultrasound sensors,
they can monitor surrounding objects of the wheelchair.
If any object is close to the moving direction of the wheelchair,
the human intention commands will be not executed because of a safety issue.
Okay, so you're fairly confident that the wheelchair can move around safely
even if the person falls asleep.
Sure. Yeah, exactly. No, I'll give the wrong commands because sometimes that happens.
So I guess when you want to use this in earnest, gluing electrodes to people's
skulls isn't going to be very popular with wheelchair users.
Have you got a plan for how to make this interface more easy to use?
Yes, what we're trying to do is we try to use wearable sensors.
To put the sensor on the head, put the sensor on maybe a glass,
some sensors, and also put the sensors on the body or in the clothes.
Make the sensors wearable, portable, and compact, so people wouldn't...
Wouldn't, how to say, people wouldn't bother by this kind of technology.
Right. Does the technology already exist for, for instance, voice control of wheelchairs?
The voice control wheelchair has been developed for several years now,
but not much use in the real situation because the background noise may cause problems,
and also people with different accents also need training as well.
So I can imagine in the future, we have to combine different modalities with
the voice, for example, voice with visual signal.
For example, if we use a microphone to catch up the human voice,
at the same time we use the camera to catch up the lip motion,
then we can confirm the user's comments by tracking the lip motion and also
the voice to actually make it more robust.
Even background noise is very loud. So you're confident that these kind of controlled
wheelchairs are going to be something that people will be using in maybe a few years' time?
That's exactly it. Recently, we have a charity that would like to invest in this area.
So we have a plan to commercialize in three or four years' time.
So moving to broader questions about robotics, what do you see as the really
big challenges in the field in the next few years?
The things that are maybe the bottlenecks that are stopping us from building
more intelligent robots, more useful robots?
Right. I think the key challenge is the sensing of the environment,
sensing of the human intention.
As we know, the robots is the integration of the sensor technology and also
AI and the learning ability, of course, actuation side as well.
I think the 21st century is a robotic century, means the robots can bring the
benefit to our society to improve our quality of life,
exactly like a computer technology and change our lifestyle,
change our working style in last century.
So, but the key problem here now is how we created an interface between technology.
Human and robots. And right now, there's still a bigger gap.
All the robots have to be used by the people with training.
And so if we want the robots going to individual home,
the robots have to be easy to use without training on the programming side.
Otherwise, still 80% of users wouldn't be able to use robots.
To finish off then, what would be a prediction you might make about the future
of our society and how we might be using robots in say 10 or 20 years from now?
What can you think of an example of one big change that robotics is going to
bring about in our society?
I think, as I said earlier,
the 21st century is a robotic century, and I can see a lot of different forms
of robots, of different kind of robots going into our homes,
and hospitals and everywhere,
especially for the dangerous situation like such a rescue work or like a nuclear
disaster, this kind of things.
And also many of the housework as well.
I can imagine in 20 years' time, many of the robots are going to be portable
and also going to be wearable.
For example, in the future, our soldiers in the battlefield may have some kind
of GPS system located themselves, and have been tracked by the army for safety purposes.
Purpose, and also many of the facilities at home with the robotic technology.
For example, even like a fridge, they can tell users, the milk is going to be
finished, you have to bring milk from the supermarket.
Or maybe get the robot to bring milk from the supermarket. Yeah,
and also, remember, Remember, we can see some kind of iPhone is kind of robots
as well, if that's the meaning.
Right. Well, thank you very much for talking to us. So this is Tony Prescott.
I've been talking to Hu-Seng Hu.
Thank you. Thank you. Thank you very much.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Programme.
For more interviews, recorded lectures or upcoming conferences in the field
of biometrics and biohybrid systems, go to csnnetwork.eu.
Music.
And thank you for listening.