WEBVTT

00:00:03.597 --> 00:00:10.517
This is the Convergent Science Network podcast. Leading researchers in the domain

00:00:10.517 --> 00:00:16.797
of neuroscience, brain theory and technology are interviewed by Paul Vershoor and Tony Prescott.

00:00:18.957 --> 00:00:24.877
Okay, so this is Paul Vershoor with the BCBT Summer School and the CSN podcast

00:00:24.877 --> 00:00:27.257
here with Joe Ayers, one of our speakers.

00:00:27.937 --> 00:00:35.577
And Joe, you started out telling us about a recent report of DARPA that was

00:00:35.577 --> 00:00:39.577
telling us that algorithmic control actually doesn't really work very well.

00:00:40.257 --> 00:00:45.257
So that sounds a bit surprising because most of the world is running on different

00:00:45.257 --> 00:00:49.697
algorithms that are implemented in various ways on digital hardware. So what's the problem?

00:00:49.957 --> 00:00:54.517
Well, the fundamental issue is that if you try to control a robot algorithmically,

00:00:54.617 --> 00:00:59.297
you have to basically anticipate every possible situation it's going to be in

00:00:59.297 --> 00:01:03.637
and program an explicit escape strategy for every situation.

00:01:04.077 --> 00:01:07.537
If you can imagine what a lobster does in the bottom of the ocean,

00:01:07.637 --> 00:01:10.417
I'd be impressed, or an octopus, etc.

00:01:10.837 --> 00:01:14.397
And as a result, that's kind of a futile exercise.

00:01:14.777 --> 00:01:17.577
And the robots end up

00:01:17.577 --> 00:01:20.897
getting tested on very constrained environments and if

00:01:20.897 --> 00:01:24.397
you remember the darpa challenge in the first year the

00:01:24.397 --> 00:01:28.097
robots got a couple of hundred yards and that was it and then the second year

00:01:28.097 --> 00:01:33.637
they made it to the finish well they changed the rules and so it was possible

00:01:33.637 --> 00:01:39.997
to do that you know right and uh so but animals are not in a position to anticipate

00:01:39.997 --> 00:01:43.217
what the world is going to be like in general, unless they're very territorial.

00:01:43.697 --> 00:01:49.797
So any animal that migrates or goes into new places is going to have to have

00:01:49.797 --> 00:01:52.437
an adaptive strategy that enables them to do that.

00:01:52.877 --> 00:01:57.377
Now, if you watch most animals, when they get stuck, what they do is they wiggle and squirm.

00:01:57.817 --> 00:02:02.317
And what they're doing is they're clearly exploring their full parameter space.

00:02:02.917 --> 00:02:06.337
And we think that what they're doing is they're increasing the level of chaos

00:02:06.337 --> 00:02:10.517
in the networks that would generate things like navigation and locomotion.

00:02:10.517 --> 00:02:16.537
And the chaotic variations on the locomotion and navigation are the wiggling and squirming.

00:02:17.278 --> 00:02:21.258
Okay, but now we've made quite some steps forward. This is really a fast forward, right?

00:02:21.618 --> 00:02:25.558
Yeah, yeah. So now we have to sort of fast rewind and then revisit some of these

00:02:25.558 --> 00:02:29.098
issues because in some sense you're saying, well, these algorithmic approaches

00:02:29.098 --> 00:02:32.638
might be nice in a controllable world, but as soon as you start to think about

00:02:32.638 --> 00:02:35.298
real world behavior as an animal behavior,

00:02:35.698 --> 00:02:37.838
we face problems. Unpredictable environments.

00:02:38.318 --> 00:02:43.318
Right. So this is the sticking point there, right? So then if we talk about

00:02:43.318 --> 00:02:48.538
unpredictability in the world and the adaptation that sort of evolution has

00:02:48.538 --> 00:02:50.658
generated to this kind of unpredictability,

00:02:50.858 --> 00:02:53.698
what are the key tricks there?

00:02:53.878 --> 00:02:57.458
So you say chaos, but what do you really mean? Well, I think another approach

00:02:57.458 --> 00:03:01.978
is critter camps, where you put a camera on the back of the animal and see what

00:03:01.978 --> 00:03:03.118
the world looks like for them.

00:03:03.318 --> 00:03:05.978
Okay, what do you see? And that's going to give you a little more insight.

00:03:06.338 --> 00:03:11.238
We do that on lobsters all the time. And what we're trying to establish is what

00:03:11.238 --> 00:03:16.058
sense organs they're contacting the environment with and the patterns of contact

00:03:16.058 --> 00:03:19.498
that exist so we can basically tune up our sensors,

00:03:19.658 --> 00:03:24.078
which are things like antennae and bump sensors on claws and things like that.

00:03:24.258 --> 00:03:28.018
So what kind of sensors do we have on a lobster? Because this is one of your

00:03:28.018 --> 00:03:28.858
favorite preparations.

00:03:29.318 --> 00:03:33.938
Yeah, yeah. Yeah. Well, most people have focused on joint receptors and joint

00:03:33.938 --> 00:03:39.138
receptors tell the animal something about the angle of the joints and the action of the muscles.

00:03:39.478 --> 00:03:44.078
But when you really see what happens, for example, on a claw of a lobster,

00:03:44.298 --> 00:03:47.938
when it bumps into something, it moves all the joints at once.

00:03:48.438 --> 00:03:54.618
So it's getting some signal from all the things, which I think is just a bump It's a lump response.

00:03:55.878 --> 00:04:00.598
But if it comes from the side as opposed from the front, it's going to have

00:04:00.598 --> 00:04:04.238
a slight variation, which is going to give the animal some subtle information

00:04:04.238 --> 00:04:10.118
about where this insult is coming from. But for that to work, we need some…,

00:04:10.896 --> 00:04:15.956
corollary discharge for an efference copy, because the animal must be knowing what it wants to do.

00:04:16.056 --> 00:04:23.016
Then it detects some perturbation on its effectors that make these joints sort

00:04:23.016 --> 00:04:24.916
of bent in different ways than it expects.

00:04:25.296 --> 00:04:27.376
Yeah, yeah, exactly. And that is in the collision detection.

00:04:27.676 --> 00:04:28.276
Would you agree with that?

00:04:28.756 --> 00:04:37.916
Well, as to whether they're using a comparator with efference copy with a sensory input, I don't know.

00:04:37.976 --> 00:04:41.476
I don't know of any real examples. But if they're walking, imagine here we have

00:04:41.476 --> 00:04:43.796
a lobster, right? You're walking, you're a lobster. Yeah, yeah, yeah.

00:04:44.136 --> 00:04:49.116
Okay, so I'm moving my legs, and now one of my legs hits a rock while I'm walking.

00:04:49.176 --> 00:04:54.236
So part of the changes in my joint angles are due to my own walking movements I'm initiating.

00:04:54.376 --> 00:04:56.836
Yeah, yeah. But now on a few of my

00:04:56.836 --> 00:04:59.996
joints, this joint angle will come out differently because of an obstacle.

00:05:00.316 --> 00:05:03.356
Mm-hmm. So how do we know this is an obstacle and not me walking?

00:05:05.056 --> 00:05:09.896
Well, we don't deal with that. Yeah. I mean, I went through originally a real

00:05:09.896 --> 00:05:14.036
plan to have joint receptors and having them feedback.

00:05:14.596 --> 00:05:17.196
And I had a whole set of reflex pathways.

00:05:17.956 --> 00:05:23.516
But before we put those on the robot, we actually tried the robot without them

00:05:23.516 --> 00:05:29.416
over some very irregular substrates, cobble fields and stuff like that. And they do just fine.

00:05:29.776 --> 00:05:31.796
And one of the things about a

00:05:31.796 --> 00:05:37.496
lobster weighs about one-eighth its weight underwater as it would in air.

00:05:37.836 --> 00:05:42.276
So a seven-pound lobster weighs just a few ounces underwater.

00:05:42.796 --> 00:05:48.436
And the actual forces against gravity are very, very small, especially when

00:05:48.436 --> 00:05:51.216
compared with a lateral hydrodynamic resistance to flow.

00:05:51.676 --> 00:05:57.196
So when they get in surge, they're getting a lot more action from the surge than they are from.

00:05:58.491 --> 00:06:01.831
Perturbations of their orientation relative to gravity and the

00:06:01.831 --> 00:06:05.571
muscles tend to be pretty compliant so you

00:06:05.571 --> 00:06:09.231
know right now on the newest generation

00:06:09.231 --> 00:06:13.931
of lobster robot that we're building i have no plans to put limb proprioceptors

00:06:13.931 --> 00:06:18.671
at all okay all right so this is interesting right so so what you've done why

00:06:18.671 --> 00:06:23.131
don't you be studying a lobster in great detail yeah and then in order to let's

00:06:23.131 --> 00:06:26.051
say understand that loves you've been building robot lobsters Yeah.

00:06:26.071 --> 00:06:30.311
So how many generations of robot lobsters have you built? We're now on the fourth generation.

00:06:31.291 --> 00:06:37.131
So when did you build the first one? How long ago was that? The first one was started in 1998.

00:06:37.791 --> 00:06:42.831
Okay. So how did this system now progress? What's the difference between version

00:06:42.831 --> 00:06:44.491
one versus version four?

00:06:44.731 --> 00:06:48.671
Okay. Version one was the infinite power, infinite bandwidth variety.

00:06:48.671 --> 00:06:52.311
Okay. And it was basically a set of legs on a hull.

00:06:52.751 --> 00:06:58.131
It had a claw and tail on it. and then it was controlled by an external computer

00:06:58.131 --> 00:06:59.351
through a serial interface.

00:06:59.731 --> 00:07:06.091
And what we would do is send byte commands to some control boards and the byte

00:07:06.091 --> 00:07:13.451
camp commands would say, turn on this muscle at a particular frequency or turn it off.

00:07:13.951 --> 00:07:17.471
And we would be sending these byte commands over a serial line.

00:07:18.531 --> 00:07:24.271
And that robot performed really quite well and that we got the basic patterns

00:07:24.271 --> 00:07:27.371
of coordination working and stuff like that.

00:07:27.511 --> 00:07:33.151
The second robot had an onboard computer, but it also had the ability to be

00:07:33.151 --> 00:07:35.891
controlled through a serial line, so we could go either way.

00:07:36.091 --> 00:07:38.171
And then it had onboard power.

00:07:38.491 --> 00:07:40.711
So it was carrying the full mass.

00:07:41.091 --> 00:07:46.611
And what we do is just compensate for mass with buoyancy.

00:07:46.831 --> 00:07:51.651
So we would put syntactic foam on it so it had about the same mass as a normal

00:07:51.651 --> 00:07:53.151
seven-pound lobster underwater.

00:07:53.911 --> 00:07:59.771
And now version four? So version four is totally different.

00:08:00.431 --> 00:08:05.971
It has basically the same mechanical system, except the basilar joint,

00:08:06.051 --> 00:08:10.531
which used to be vertical, is now candid at a 45-degree angle.

00:08:10.671 --> 00:08:17.391
And that gives the limb tip a rolling action like a wheel as opposed to a more rectilinear motion.

00:08:17.711 --> 00:08:21.011
And that's also consistent with the morphology of the real lobster?

00:08:21.211 --> 00:08:24.671
It's exactly like a real lobster. That's exactly the angle that a real lobster's

00:08:24.671 --> 00:08:29.151
leg is at. And what's the advantage of that? The advantage of that is it lets the animal rear back.

00:08:29.491 --> 00:08:35.131
So if it's trying to walk up a slope or down a slope, it has a more natural

00:08:35.131 --> 00:08:36.671
angle of attack of the leg.

00:08:37.091 --> 00:08:41.331
Okay. So now you emphasize very much just the legs.

00:08:42.458 --> 00:08:45.798
Um what kind of sensors are you considering well we

00:08:45.798 --> 00:08:49.018
have again from one to four yeah yeah right uh well

00:08:49.018 --> 00:08:52.438
the basic sensor suite we have on is a compass we

00:08:52.438 --> 00:08:55.118
have a pitch and roll inclinometer that is

00:08:55.118 --> 00:08:57.998
also an accelerometer and that gives

00:08:57.998 --> 00:09:01.078
us basically three axes of pitch roll

00:09:01.078 --> 00:09:05.458
and yaw as well well as um the rate

00:09:05.458 --> 00:09:08.318
of change of pitch roll in the off and then

00:09:08.318 --> 00:09:11.698
we have bump sensors on the

00:09:11.698 --> 00:09:18.938
claws and they basically have an accelerometer they full wave rectify the signal

00:09:18.938 --> 00:09:24.938
low pass filter it and then create a square wave that indicates a bump and that

00:09:24.938 --> 00:09:29.958
has a duration associated with it so it has a little bit of memory of having had bumped.

00:09:30.918 --> 00:09:37.698
And then we have antennae, and the antennae have bin sensors in them.

00:09:38.018 --> 00:09:43.078
And the antennae can be deployed at different angles, so they can be held straight

00:09:43.078 --> 00:09:46.318
out in front of the animal or off to the sides laterally.

00:09:46.578 --> 00:09:52.858
And their bin gives us a very quantifiable measure of the flow rate of the ocean around it.

00:09:53.258 --> 00:09:57.698
And we're also putting optical flow sensors on the new robot.

00:09:58.545 --> 00:10:02.845
So it can use optical flow information and compare that with a hydrodynamic

00:10:02.845 --> 00:10:08.925
flow information and get a clearer picture of things that might be ambiguous with one sensor.

00:10:09.305 --> 00:10:15.185
Okay, so if we talk about sensor processing now, are you saying that also lobsters use optic flow?

00:10:15.825 --> 00:10:20.005
Oh, no question they do. Really? Yeah, I published a paper on that in Science in 1972.

00:10:21.765 --> 00:10:23.145
I was barely born.

00:10:26.005 --> 00:10:29.205
So they they often live in

00:10:29.205 --> 00:10:32.105
rather murky and dark environments yeah so how much

00:10:32.105 --> 00:10:36.745
help but their light their eyes are very are very sensitive in low light okay

00:10:36.745 --> 00:10:42.525
so they can and viresma showed years ago that they have unidirectional optical

00:10:42.525 --> 00:10:49.245
flow sensors and we work with jeff barrows from sentai on the RoboBee program,

00:10:49.525 --> 00:10:54.005
and we're going to adapt the sensors from that program on the robot lobster.

00:10:54.245 --> 00:11:00.005
Right. And on the RoboLamprey, for that matter. Right. But then we're skipping ahead, right? Yeah.

00:11:00.885 --> 00:11:07.905
So now we have our lobster. We have the sensors. We have some of the sensors. Now we have the walking.

00:11:09.385 --> 00:11:12.945
Now how about the control? Okay, there's where the big difference is.

00:11:12.945 --> 00:11:19.985
So the new lobster, rather than being controlled by what was effectively a state

00:11:19.985 --> 00:11:22.865
machine that mimicked the operation of central pattern generators,

00:11:23.405 --> 00:11:27.485
it's now controlled by true central pattern generators that are formed from

00:11:27.485 --> 00:11:30.065
what are called discrete-time map-based neurons.

00:11:30.485 --> 00:11:35.085
And discrete-time map-based neurons were developed by Nikolai Rukov,

00:11:35.165 --> 00:11:39.005
my colleague from the Institute for Nonlinear Science at UC San Diego.

00:11:39.005 --> 00:11:44.525
And these are neurons that are a one-dimensional map that are a,

00:11:44.525 --> 00:11:50.045
oh, what's the word for it? I'm blanking on this.

00:11:52.305 --> 00:11:55.405
A phonological model of neurons. Right.

00:11:55.645 --> 00:12:00.185
Okay, so they try to capture the dynamics of neurons, and they have two control

00:12:00.185 --> 00:12:06.485
parameters that let us change the neurons from being either truly spiking neurons

00:12:06.485 --> 00:12:11.645
that fire in tonic spiking patterns or bursting patterns,

00:12:11.885 --> 00:12:16.305
or we can configure the two control parameters so they become chaotic.

00:12:16.825 --> 00:12:20.945
Okay, but now can you control the spike frequency quite easily,

00:12:21.065 --> 00:12:25.325
and the burst, and the bursting? Oh yeah, really. Yeah, there's another parameter,

00:12:25.425 --> 00:12:27.685
which is the synaptic current.

00:12:27.985 --> 00:12:30.865
So we can modify the synaptic current.

00:12:31.105 --> 00:12:36.865
And we can modify that parametrically as would occur during neuromodulation,

00:12:36.965 --> 00:12:40.845
or we can apply regular synaptic pulses from other neurons.

00:12:41.065 --> 00:12:43.925
Okay, so how do you build a central pattern generator with that?

00:12:44.285 --> 00:12:47.765
Well, the neurons are basically a...

00:12:50.075 --> 00:12:56.535
Two equations, and the equations have some fuzzy logic over different ranges of membrane voltage.

00:12:57.055 --> 00:13:08.675
And those equations represent the voltage in cycle n plus one as a function of voltage in cycle n.

00:13:08.755 --> 00:13:15.095
And you loop through cycle by cycle and keep calculating the voltage in cycle

00:13:15.095 --> 00:13:18.535
n plus one, and that turns out to be the voltage of the neuron,

00:13:18.635 --> 00:13:19.575
the transmembrane voltage.

00:13:20.075 --> 00:13:23.815
So by changing the rate, you can speed them up. So we need to speed them up,

00:13:23.875 --> 00:13:28.175
for example, in RoboBee, but we can run them quite slowly in RoboLobster.

00:13:28.375 --> 00:13:32.255
So these elements are intrinsically oscillating. Yeah. Okay,

00:13:32.355 --> 00:13:33.515
so that's what you're exploiting then.

00:13:33.695 --> 00:13:36.955
Well, by varying these two control parameters, alpha and sigma,

00:13:37.135 --> 00:13:39.335
we can put them into a bursting regime.

00:13:39.495 --> 00:13:43.675
Or for postural behavioral practice, we can put them in a spiking regime.

00:13:43.675 --> 00:13:47.675
But now to, for instance, get coordinated movement of the six legs,

00:13:47.955 --> 00:13:51.435
you have to also coordinate among these oscillators.

00:13:51.455 --> 00:13:53.895
Yeah, and we have coordinating neurons that do that.

00:13:55.035 --> 00:13:58.875
So the coordinating neurons pass information from a governing oscillator to

00:13:58.875 --> 00:14:03.555
a governed oscillator so that the governed oscillator maintains the proper phase

00:14:03.555 --> 00:14:06.095
with the governing oscillator to maintain a gate,

00:14:06.295 --> 00:14:11.135
which is an attempt to create a pattern of footfall support to keep the thing

00:14:11.135 --> 00:14:12.975
stable in the pitch and roll plane. Okay.

00:14:13.675 --> 00:14:20.495
And how well is then such a control model validated in biological terms?

00:14:20.595 --> 00:14:25.555
Do you have, let's say, analogs in the lobster nervous system somewhere that

00:14:25.555 --> 00:14:29.755
would match, for instance, to this master oscillator that controls these sub-oscillators?

00:14:29.855 --> 00:14:36.695
Do you have examples of that? Yeah, so the model is based initially off dynamical

00:14:36.695 --> 00:14:41.435
analysis that I did from electromyograms in behaving lobsters in the 70s.

00:14:41.435 --> 00:14:51.115
And then one of Al's, I mean, a Franco-Clarac student named Abraham Shrashri

00:14:51.115 --> 00:14:55.615
did paired recording of many neurons in the thoracic ganglia,

00:14:55.775 --> 00:14:57.755
worked out the synaptic network,

00:14:58.075 --> 00:15:03.715
and we tried to capture that synaptic network in the network model that we built with the neurons.

00:15:03.715 --> 00:15:10.875
So the neurons are connected by synapses, and the synapses are chemical synapses

00:15:10.875 --> 00:15:13.775
that take into account the presynaptic voltage,

00:15:13.935 --> 00:15:18.155
the postsynaptic voltage, and inject current appropriate to the difference between

00:15:18.155 --> 00:15:20.075
pre- and postsynaptic voltages. Okay.

00:15:20.515 --> 00:15:24.455
But then in some sense, I could argue, look, that's nice and well,

00:15:24.575 --> 00:15:29.475
but it might just be a very phenomenological model that is sort of at the functional

00:15:29.475 --> 00:15:32.695
end gives you something that is lobster-like walking.

00:15:33.695 --> 00:15:38.595
But how do I kind of know more specifically that this is actually informing us about the lobster?

00:15:38.675 --> 00:15:42.155
How does it help us understand what lobsters really do and how their brains work?

00:15:43.233 --> 00:15:47.553
Well, for all intents and purposes, the neurons operate in the same patterns

00:15:47.553 --> 00:15:49.713
that you would see from electromyograms.

00:15:49.753 --> 00:15:54.993
So the timing, the patterns of output that we see are quite indistinguishable

00:15:54.993 --> 00:15:59.013
from the patterns that you would see in behaving animals under the same circumstances.

00:15:59.673 --> 00:16:04.593
They don't capture the details of the conductance mechanisms, okay?

00:16:04.793 --> 00:16:08.033
But if we tried to use parallel conductance theory,

00:16:08.353 --> 00:16:11.673
we'd have to use differential equations given the

00:16:11.673 --> 00:16:15.393
fact that some of these neurons probably have five or six currents there

00:16:15.393 --> 00:16:18.233
would probably be 10 or 12 differential equations for each

00:16:18.233 --> 00:16:22.533
neuron and we wouldn't be able to model very many on a real-time processor right

00:16:22.533 --> 00:16:27.773
we're using a model that's based on difference equations and as a result we

00:16:27.773 --> 00:16:32.273
can do hundreds of neurons and synapses on a single digital signal processing

00:16:32.273 --> 00:16:36.413
chip right but we don't program them as.

00:16:38.433 --> 00:16:41.673
Whereas algorithmically, we program them by wiring up networks,

00:16:41.853 --> 00:16:47.033
and we establish the dynamics of the neurons by putting these two control parameters,

00:16:47.313 --> 00:16:48.953
alpha and sigma, in the appropriate range.

00:16:49.173 --> 00:16:52.973
So now, have you found a correlate in the lobster nervous system of these two

00:16:52.973 --> 00:16:55.113
control parameters? What would they be?

00:16:56.153 --> 00:17:00.533
Again, these are phenomenological models, and there's no one-for-one mapping

00:17:00.533 --> 00:17:06.033
of these phenomenological control parameters on any ionic conductance parameter.

00:17:06.873 --> 00:17:12.813
And that's something that at UCSD they spent several years, and as Henry or

00:17:12.813 --> 00:17:16.993
Bob and I would put it, there were a lot of bodies out in the hall trying to solve this problem.

00:17:17.473 --> 00:17:19.593
And the bodies were decomposing.

00:17:20.873 --> 00:17:21.313
Exactly.

00:17:23.073 --> 00:17:28.473
Let this be a warning. So now we are at level thoracic ganglia, right?

00:17:28.633 --> 00:17:32.413
Yeah. So we're controlling these legs, they're moving, we have a neural model

00:17:32.413 --> 00:17:37.413
to do that. But actually, most of the bulk of the nervous system of this animal

00:17:37.413 --> 00:17:38.873
sits above these thoracic ganglia.

00:17:38.933 --> 00:17:42.393
Yeah, yeah. So what are they doing in your robot?

00:17:42.713 --> 00:17:49.973
Okay, so we have a lot of exteroceptive reflexes that we've layered in what

00:17:49.973 --> 00:17:51.633
would be the brain of this robot.

00:17:52.553 --> 00:17:56.413
And, you know, there's been a lot of talk in this meeting about Breitenberg machines.

00:17:57.293 --> 00:18:02.533
Well, I don't know of any alternative to a Breitenberg machine, you know.

00:18:02.973 --> 00:18:06.533
Neurons receives input on one side or the other.

00:18:06.973 --> 00:18:11.293
They project to one side or the other, and they can either project to the same

00:18:11.293 --> 00:18:13.553
side or they can decussate to the opposite side.

00:18:14.193 --> 00:18:17.993
There aren't a whole lot of alternatives in a bilaterally symmetrical nervous system.

00:18:18.413 --> 00:18:23.933
So we have layered reflexes for optical flow.

00:18:24.033 --> 00:18:30.933
We have layered reflexes for hydrodynamic flow, for bump, for deviations in

00:18:30.933 --> 00:18:34.233
the pitch and roll plane and we have...

00:18:35.491 --> 00:18:39.491
As we layer on more and more sensors, and very soon we'll have some very good

00:18:39.491 --> 00:18:44.311
chemical sensors, we'll be able to begin to layer those sorts of reflexes on

00:18:44.311 --> 00:18:49.531
top of these more fundamental reflexes. But now, which structures of that brain are you modeling?

00:18:49.671 --> 00:18:54.391
Do you, for instance, approach the optical lobes of the lobster brain?

00:18:54.731 --> 00:18:59.891
Are these also modeled at the neural level? We're not modeling the… So,

00:18:59.951 --> 00:19:01.171
this is very interesting.

00:19:02.251 --> 00:19:06.211
We operate at the level of what we call releasing mechanisms.

00:19:07.731 --> 00:19:12.031
So our sensors are typical analog electrical sensors.

00:19:12.371 --> 00:19:16.511
And then what we do with them is we create a spiking discharge,

00:19:16.951 --> 00:19:22.111
typically range fractionated, where we take an input variable that might be

00:19:22.111 --> 00:19:25.391
the amount of bending of the right antennae.

00:19:25.391 --> 00:19:31.471
And then we create from that a set of interneurons that are recruited in order

00:19:31.471 --> 00:19:35.911
of size that represent the magnitude of that input variable.

00:19:36.971 --> 00:19:42.931
So if it were, say, if the antennae were held out to directly at right angles

00:19:42.931 --> 00:19:48.531
to the long body axis, as flow came from the front, at low flow rates,

00:19:48.651 --> 00:19:51.031
the antennae would be bent a little bit.

00:19:51.131 --> 00:19:54.051
As the flow rates increased, they'd be bent more and more.

00:19:54.271 --> 00:20:00.911
Sure. And we usually quantize these with about three levels of range fractionation,

00:20:00.971 --> 00:20:05.131
which is fairly typical for the receptors we see in these animals.

00:20:05.391 --> 00:20:10.091
Okay, but then we're really at the periphery of the animal, right? Yeah, yeah, yeah.

00:20:10.311 --> 00:20:15.931
So what's the difference between the structure of, let's say,

00:20:15.971 --> 00:20:18.011
a lobster brain and a drosophila brain?

00:20:19.551 --> 00:20:22.691
If you look at the key structures, are these roughly similar?

00:20:22.691 --> 00:20:25.331
Similar no no there are no mushroom bodies in the

00:20:25.331 --> 00:20:29.071
lobster okay okay so so lobsters

00:20:29.071 --> 00:20:32.371
have a series of four um um

00:20:32.371 --> 00:20:39.271
there's a lamina ganglionaris a medulla externa medulla interna and a medulla

00:20:39.271 --> 00:20:44.471
terminalis which are the four integrative layers going from the omentidia into

00:20:44.471 --> 00:20:48.631
the optic nerve projecting in the central nervous system these are where virsma

00:20:48.631 --> 00:20:51.531
used to be doing pin recordings in crayfish fish,

00:20:51.591 --> 00:20:57.871
and crabs to establish all the six different types of interneurons that come in from the eye.

00:20:58.091 --> 00:21:04.051
The thing that really distinguishes crustaceans from, say, humans is that we

00:21:04.051 --> 00:21:07.231
just have one kind of, or two kinds of ganglion cells.

00:21:07.431 --> 00:21:11.051
We have the on-center, off-surround, and the off-surround, on-center.

00:21:11.051 --> 00:21:18.291
And these animals have, for example, fibers that Wiersma used to call,

00:21:18.491 --> 00:21:24.371
oh, I'm blanking on these names now. So what did he call them?

00:21:25.490 --> 00:21:28.770
Sustaining fibers which would respond to an

00:21:28.770 --> 00:21:31.550
increase in illumination in one area there were

00:21:31.550 --> 00:21:34.750
dimming fibers that would respond to a decrease in in

00:21:34.750 --> 00:21:40.310
illumination in an area there were what he called unidirectional motion fibers

00:21:40.310 --> 00:21:45.190
they're what he called space constant fibers and then he had a kind of fiber

00:21:45.190 --> 00:21:50.270
he called seeing fiber in which these things would respond to things like movement

00:21:50.410 --> 00:21:51.650
of the contralateral legs,

00:21:51.910 --> 00:21:54.190
images in the contralateral eye.

00:21:54.450 --> 00:21:58.150
He described their behavior as complex as the behavior of the whole animal.

00:21:58.550 --> 00:22:02.750
And these would be like whole field. Yeah. And he also had another class of

00:22:02.750 --> 00:22:07.690
fiber called jittery movement fibers, which we would call in a frog bug detectors.

00:22:08.010 --> 00:22:13.510
Right. Okay. And so the thing about lobsters is that they have 30 or 40% of

00:22:13.510 --> 00:22:18.770
their central neurons in their brain are out in their eye stalks doing optical processing.

00:22:18.970 --> 00:22:23.290
Okay. And the number of central cell bodies within the brain itself are confined

00:22:23.290 --> 00:22:29.690
largely to the motor neurons that go to the head appendages and then some releasing

00:22:29.690 --> 00:22:33.630
mechanisms that respond to input from the statuses,

00:22:33.710 --> 00:22:35.870
from the antennules, and from the antennae.

00:22:36.470 --> 00:22:42.130
So that would mean that relatively little hardware is dedicated to olfaction?

00:22:43.590 --> 00:22:48.690
In the lobster, you know, that's Barry Aki's world.

00:22:48.690 --> 00:22:54.930
And chuck derby's world they focus on that and um you know we really haven't

00:22:54.930 --> 00:23:00.270
gotten into that yet and as we get more of a capability of building sensors

00:23:00.270 --> 00:23:03.710
using um synthetic biological approaches,

00:23:04.330 --> 00:23:09.210
then we're going to start working in that area but that's that's right now something

00:23:09.210 --> 00:23:14.110
i've just been funded to do and we really haven't started yet okay but now so

00:23:14.110 --> 00:23:20.410
um so you started this work Or actually, you're a physiologist, right?

00:23:20.570 --> 00:23:23.990
Yeah, I'm a systems neurophysiologist. Right. So at some point,

00:23:24.010 --> 00:23:25.930
you decided to put this robot lobster together.

00:23:28.210 --> 00:23:31.570
And I guess it was with the ambition to actually understand the real lobster.

00:23:31.850 --> 00:23:36.030
Well, no. The way this all started is very interesting.

00:23:36.370 --> 00:23:40.710
So I had spent a lot of time working on sea lamprey.

00:23:42.050 --> 00:23:46.150
And we were interested in how they recover from spinal cord injuries.

00:23:46.150 --> 00:23:54.610
And um we were successful at identifying how they recover the ability to turn on swimming.

00:23:55.310 --> 00:24:01.690
And then the next grant review i put out um i got a review that said uh this

00:24:01.690 --> 00:24:07.630
is very fundable work if you do it in a in a mammal and i don't do that you

00:24:07.630 --> 00:24:11.330
know i'm you're not going to see me working on mice and rats, I guarantee you.

00:24:11.670 --> 00:24:16.350
Uh, and so I decided to go back to the lobster and,

00:24:17.001 --> 00:24:23.321
And this was at a time when people were really beginning to establish a really

00:24:23.321 --> 00:24:28.781
incredible library of neuromodulatory substances in the stomatogastric ganglion.

00:24:28.961 --> 00:24:36.881
And I think at that point, there were about 35 known substances that would alter

00:24:36.881 --> 00:24:40.201
the motor output patterns generated by the stomatogastric ganglion. them.

00:24:40.681 --> 00:24:46.881
So, um, working with Al Silverston and George Heinzel, um,

00:24:47.021 --> 00:24:53.481
we decided to try to get a handle on which of those were really operating in

00:24:53.481 --> 00:25:00.621
lobsters and which were artifactual because for them to be 35 different modulatory

00:25:00.621 --> 00:25:03.901
substances was a little bit over the top.

00:25:03.901 --> 00:25:09.101
So I got together with a company called Massa Products,

00:25:09.361 --> 00:25:14.721
and we started developing a sonar biotelemetry system, which was a physiological

00:25:14.721 --> 00:25:19.761
telemetry system, where we would record from the muscles that are controlled

00:25:19.761 --> 00:25:21.441
by the stomatogastric ganglion.

00:25:22.121 --> 00:25:24.661
And many of these muscles have only one motor neuron.

00:25:25.641 --> 00:25:30.501
Or I think max, there's four motor neurons in a muscle.

00:25:31.281 --> 00:25:34.361
And if you are studying feeding in

00:25:34.361 --> 00:25:37.361
a lobster the lobster feeds when you feed it

00:25:37.361 --> 00:25:40.221
what you feed it it doesn't have a chance to

00:25:40.221 --> 00:25:43.201
select the chinese food or the italian food and it

00:25:43.201 --> 00:25:46.781
doesn't have a chance to pick when it's going to eat so if you really want to

00:25:46.781 --> 00:25:50.901
see what the normal patterns of operation of the stomatogastric ganglion are

00:25:50.901 --> 00:25:55.761
you have to do it in freely behaving animals so the goal of this project was

00:25:55.761 --> 00:26:00.641
to record for the muscles controlled by this ganglion in animals that were free

00:26:00.641 --> 00:26:02.041
to run around in the world.

00:26:02.181 --> 00:26:07.181
And the idea is that we took electromyographic recordings from these muscles,

00:26:07.881 --> 00:26:13.641
did some signal processing, and then would transmit using sonar the on times

00:26:13.641 --> 00:26:15.861
and off times of these different muscles.

00:26:16.341 --> 00:26:23.501
And I created a webpage about this and got some funding from the Office of Naval Research.

00:26:24.861 --> 00:26:28.761
And I got a phone call from a program officer at DARPA,

00:26:28.821 --> 00:26:36.201
and he had seen this website and wanted to know if we could use this to take

00:26:36.201 --> 00:26:40.901
control of a giant lobster to use the lobster for remote sensing purposes.

00:26:43.121 --> 00:26:48.281
And he and I conversed for quite a while, and I told him that it was really

00:26:48.281 --> 00:26:53.461
my belief that if you try to control an animal, it's going to do what it damn

00:26:53.461 --> 00:26:57.101
well pleases. You're not going to have much luck doing this with a lobster.

00:26:57.921 --> 00:27:01.161
And I personally thought it was easier to build a robotic lobster.

00:27:01.821 --> 00:27:07.161
And I had been listening to Randy Beer talk about how they were building robotic insects.

00:27:07.481 --> 00:27:10.501
And this looked like a pretty interesting endeavor to me.

00:27:10.921 --> 00:27:15.581
Well, he took me very seriously and asked me how I would do this.

00:27:15.601 --> 00:27:17.821
And I developed some ideas.

00:27:18.041 --> 00:27:24.401
And then he asked me to write a proposal. And he gave me a considerable sum

00:27:24.401 --> 00:27:26.021
of money to build a robotic lobster.

00:27:26.581 --> 00:27:29.741
So I went out and hired a bunch of engineers.

00:27:31.161 --> 00:27:38.901
And I had a very interesting experience at this point because I found that the

00:27:38.901 --> 00:27:42.581
engineers that I was working with could be divided into two categories.

00:27:43.381 --> 00:27:47.621
One were guys that knew how to make stuff work. And the other were what I called

00:27:47.621 --> 00:27:49.021
experts on what's impossible.

00:27:49.021 --> 00:27:55.941
And they turned out to be kind of lethal because no matter what you wanted to

00:27:55.941 --> 00:27:59.121
build, they would find some reason why in principle it couldn't work.

00:27:59.601 --> 00:28:05.161
And I found myself in situations where we would have something working and some

00:28:05.161 --> 00:28:08.481
of the participating engineers would tell me that that couldn't possibly be happening.

00:28:08.761 --> 00:28:15.121
And so I found it quite necessary to weed out this crew in order to be successful.

00:28:15.281 --> 00:28:18.101
And that was the genesis of the lobster robot. Okay.

00:28:18.501 --> 00:28:22.761
So here we have your neurophysiology, your system's neurophysiology.

00:28:23.481 --> 00:28:26.361
Now we have sort of the robot lobster.

00:28:26.921 --> 00:28:30.221
But now in retrospect, because you're doing this now for quite some time,

00:28:30.701 --> 00:28:33.781
has it helped you in any way to understand the biological system?

00:28:34.561 --> 00:28:38.881
Oh, very much so. Or if that was just a game, would it have been better to stick

00:28:38.881 --> 00:28:40.941
to the neurophysiology? Yeah, yeah, yeah, yeah.

00:28:41.121 --> 00:28:46.401
No, I think what really happens when you try to build a complete system is you

00:28:46.401 --> 00:28:50.561
very quickly identify the lacunae in your knowledge.

00:28:51.021 --> 00:28:55.621
And that really gives you some ideas on what you should really be looking for.

00:28:55.621 --> 00:28:58.561
So for example i have right now a student

00:28:58.561 --> 00:29:02.001
in my laboratory that's doing recordings from the

00:29:02.001 --> 00:29:04.701
brain connectives going from the brain down to the lower

00:29:04.701 --> 00:29:10.541
ganglia to look at the patterns of discharge of some of the the systems that

00:29:10.541 --> 00:29:15.981
we can use to inform the way we drive the systems in our electronic nervous

00:29:15.981 --> 00:29:22.201
system okay so that's really a clear finding from doing that but,

00:29:23.241 --> 00:29:30.201
The other thing that building the robot lobster informed me on was this idea of bump sensing.

00:29:31.241 --> 00:29:35.701
You know, if you look at the literature on crustaceans, everybody gets so worried

00:29:35.701 --> 00:29:39.161
about what inner neurons come from what joint.

00:29:39.261 --> 00:29:44.441
And I think a lot of these inner neurons are just responding to gross disturbances

00:29:44.441 --> 00:29:48.021
of the sort that occur when the claw hits a rock, for example.

00:29:48.961 --> 00:29:54.621
But how would that be achieved? I mean, is that a direct linking to some mechanoreceptor,

00:29:54.761 --> 00:29:57.321
or is it more complicated?

00:29:57.641 --> 00:30:01.581
We're not in a position to create cortitonal organs or bipolar neurons.

00:30:02.701 --> 00:30:11.441
So we do this by using cheap analog sensors, and then we take either a small

00:30:11.441 --> 00:30:16.481
PIC microcontroller or something like that, and create what we call a releasing mechanism.

00:30:16.641 --> 00:30:21.481
And the output of that releasing mechanism are patterns of neuronal activity,

00:30:21.681 --> 00:30:23.301
which form the input of our sensors.

00:30:23.661 --> 00:30:28.921
Right. Okay. So then in your field tests, these robots have shown to be pretty

00:30:28.921 --> 00:30:32.561
robust, which is actually remarkable because you could say, look,

00:30:32.781 --> 00:30:36.741
this is in some of the minimal amount of control you could give them.

00:30:37.001 --> 00:30:41.941
And then you give them this sort of chaos-based way to escape from local minima

00:30:41.941 --> 00:30:45.721
when they're stuck in whatever, between obstacles or whatever.

00:30:46.041 --> 00:30:49.441
And it seems to be sufficient, at least that's what it looks like,

00:30:49.541 --> 00:30:51.501
to give you fairly robust behavior.

00:30:52.541 --> 00:30:56.661
But where does it actually, what are the weaknesses? Are there weaknesses in

00:30:56.661 --> 00:30:58.261
this approach? Does it really get stuck somewhere?

00:30:58.561 --> 00:31:03.761
Are there problems it cannot solve? Yeah, yeah. Well, deciding how you're stuck

00:31:03.761 --> 00:31:06.421
is one of the problems that we're working on right now.

00:31:06.761 --> 00:31:13.381
And we do this with accelerometry. a tree. So what we do is we take a behaving animal,

00:31:14.221 --> 00:31:20.401
And we look at the output of an accelerometer in real time, and we ask ourself,

00:31:20.401 --> 00:31:26.781
what is the pattern of acceleration you would see when a lobster normally starts walking forwards?

00:31:27.021 --> 00:31:34.101
Now, if you tell your robot lobster to walk forwards, and you see a different

00:31:34.101 --> 00:31:37.221
pattern of acceleration, that's a pretty good indication you're stuck.

00:31:37.661 --> 00:31:41.801
If you back up and you see the normal pattern of acceleration,

00:31:42.281 --> 00:31:45.561
that's a pretty good indication that you aren't impeded in this direction.

00:31:46.121 --> 00:31:51.641
So by comparing the patterns of movement in response to a command-initiated

00:31:51.641 --> 00:31:56.301
behavior with the actual movement, we can determine whether we're stuck or not.

00:31:56.441 --> 00:32:00.601
So that's in fact one of the areas of research that we're really going to get

00:32:00.601 --> 00:32:03.281
into big time with the fourth generation robot.

00:32:03.281 --> 00:32:06.361
Okay but but then then we are back at something like

00:32:06.361 --> 00:32:09.121
a corollary discharge to make that happen there there's yeah

00:32:09.121 --> 00:32:12.521
yeah yeah and in fact i think the the

00:32:12.521 --> 00:32:19.401
you hit the nail right on the head that the in fact what we would see in response

00:32:19.401 --> 00:32:25.741
to normal acceleration would be the corollary discharge exactly yeah yeah all

00:32:25.741 --> 00:32:29.521
right but i think that would more reflect what the command neuron would be doing

00:32:29.521 --> 00:32:31.901
rather than the output of the CPG.

00:32:32.341 --> 00:32:37.581
But still, it would need some internal model of how its own body is acting.

00:32:37.661 --> 00:32:42.121
I think, well, the way we actually plan to do this, and this is something that

00:32:42.121 --> 00:32:47.401
we are fooling around with, is to have a leaky integrator that receives input

00:32:47.401 --> 00:32:48.521
from the command neuron.

00:32:48.821 --> 00:32:54.561
And by adjusting the rise and fall time of the leaky integrator so it mimics

00:32:54.561 --> 00:32:56.961
what the normal pattern of acceleration would be.

00:32:56.961 --> 00:33:02.041
And then we can compare that with the actual pattern of acceleration but this

00:33:02.041 --> 00:33:06.901
leaky integrator you now propose you sort of.

00:33:08.000 --> 00:33:12.040
Do not care whether at this stage you know whether the real lobster has that

00:33:12.040 --> 00:33:14.200
same leaky integrator or not.

00:33:14.440 --> 00:33:20.880
Well, again here, the robot's going to inform what to look for in the real animal. Exactly.

00:33:20.960 --> 00:33:24.600
And then that's the kind of experiment we will continue to do in the real animals.

00:33:24.800 --> 00:33:29.100
Right. So I'm always going to have with every model I work with somebody doing

00:33:29.100 --> 00:33:30.780
biology and somebody doing robotics.

00:33:31.020 --> 00:33:33.500
And they're going to inform each other. Yeah, well, it's interesting because

00:33:33.500 --> 00:33:38.080
in analyzing all the history of the project, it's more that the robot is based

00:33:38.080 --> 00:33:41.400
on, let's say, some informed imagination that you test in the biology.

00:33:41.600 --> 00:33:45.440
Yeah. But not that the biology has made concrete suggestions of what to do on

00:33:45.440 --> 00:33:47.660
the robot, except to give it six legs and a claw.

00:33:48.640 --> 00:33:51.960
Yeah, well, I mean, I don't think we, before we started building robots,

00:33:52.160 --> 00:33:56.500
that we really knew what to look for as well as we do after having built robots. Okay.

00:33:56.740 --> 00:34:01.320
Okay, so it really helps you, right? Yeah, yeah. But now, in some sense,

00:34:01.440 --> 00:34:05.820
this approach has been extremely successful because you have been now expanding

00:34:05.820 --> 00:34:07.680
into many other projects, right?

00:34:07.800 --> 00:34:11.600
And one of the issues, there's a great interest in these kinds of machines and

00:34:11.600 --> 00:34:16.500
robots also because practical applications of, let's say, autonomous technology

00:34:16.500 --> 00:34:18.960
are actually still fairly limited, right?

00:34:19.040 --> 00:34:24.600
So you pointed out the reality of what it means to keep one of these drones up in the air, right?

00:34:24.720 --> 00:34:28.900
So what's the problem there exactly? Exactly.

00:34:28.920 --> 00:34:33.120
What's the problem with keeping drones going, and will we actually ever make them autonomous?

00:34:33.560 --> 00:34:38.080
Well, most of the military robots right now are teleoperated.

00:34:38.160 --> 00:34:43.860
And they're teleoperated primarily from the perspective that...

00:34:47.120 --> 00:34:51.200
What's the best way to say this? This is complicated. First of all,

00:34:51.660 --> 00:34:55.760
most of them are very large, expensive pieces of equipment. So,

00:34:56.060 --> 00:35:00.180
a Predator, for example, is a big piece of equipment that you don't want falling

00:35:00.180 --> 00:35:01.220
in somebody's backyard.

00:35:01.840 --> 00:35:06.460
Nope. As a result, nobody attempts even to try to fly these autonomously.

00:35:07.500 --> 00:35:10.700
And I don't really know what the details of this are.

00:35:10.860 --> 00:35:17.300
I imagine we're hearing on the radio now that the pilot of a typical passenger

00:35:17.300 --> 00:35:19.860
airplane is really only flying at about three minutes.

00:35:20.240 --> 00:35:26.600
And the rest of the time it's on autopilot. So I guess the autopilot is autonomous

00:35:26.600 --> 00:35:30.740
behavior that's very closely watched by a human operator.

00:35:32.480 --> 00:35:35.800
But the risk management issues,

00:35:36.060 --> 00:35:44.020
given the cost of these vehicles and the potential danger of them having a mishap,

00:35:44.140 --> 00:35:48.700
has led the military to use teleoperation for almost every vehicle.

00:35:49.140 --> 00:35:54.440
Now, once they get used to teleoperation, you know, they're not going to be.

00:35:56.000 --> 00:36:02.540
So willing to play with autonomous behavior because they might lose a very expensive piece of equipment.

00:36:02.900 --> 00:36:08.860
Now, the way we build these robots, probably the most expensive part outside

00:36:08.860 --> 00:36:12.080
of the intellectual capital is the batteries.

00:36:12.700 --> 00:36:16.700
So we can build vehicles where if you lose a few, it's all right.

00:36:16.940 --> 00:36:20.960
And if we base the fabrication and our fourth generation robot,

00:36:21.660 --> 00:36:24.920
is really designed for manufacture so it'll

00:36:24.920 --> 00:36:28.700
be very cheap to produce these in quantity and and

00:36:28.700 --> 00:36:34.440
we're already building four of them right now now when we used to our original

00:36:34.440 --> 00:36:39.060
robots we built them one at a time right so we're just starting off with a mess

00:36:39.060 --> 00:36:43.280
of them from the beginning so we won't feel so bad if one of them rides off

00:36:43.280 --> 00:36:47.300
over the horizon yeah sure yeah so it's So the point is that,

00:36:47.300 --> 00:36:50.860
so there's this great interest to have more,

00:36:50.940 --> 00:36:55.360
let's say, a biomimetic approach to building these kinds of machines we can

00:36:55.360 --> 00:37:02.820
use to identify or deal with agents of harm like landmines or missing nukes or what have you.

00:37:03.760 --> 00:37:08.840
And a number of programs are underway in the US now to also realize these systems,

00:37:08.900 --> 00:37:11.460
right? Like the MESS program from DARPA and so on.

00:37:12.840 --> 00:37:20.640
So in that context you have now been expanding from let's say the lobster also into the insect right,

00:37:20.680 --> 00:37:26.800
that was an artificial bee project so to what extent will the artificial bee

00:37:26.800 --> 00:37:30.460
be a flying version of your lobster what's different to it.

00:37:31.614 --> 00:37:37.614
I mean, I'm a comparative physiologist, and I don't think that we're ever going

00:37:37.614 --> 00:37:39.834
to completely understand any organism.

00:37:40.674 --> 00:37:45.594
But I think we're going to find general principles that are shared by broad

00:37:45.594 --> 00:37:51.854
numbers of organisms that by using comparative physiology to figure out how

00:37:51.854 --> 00:37:55.014
these things work in the most technically accessible species,

00:37:55.454 --> 00:38:01.994
we can assemble a pretty complete library of control principles that that apply to all arthropods.

00:38:02.514 --> 00:38:07.434
So, I mean, that's the sort of approach I take. You know, I don't think we're

00:38:07.434 --> 00:38:11.814
gonna be able to record from very many neurons in behaving flies ever,

00:38:12.634 --> 00:38:14.954
or behaving bumblebees, or behaving bees.

00:38:15.754 --> 00:38:20.734
Their nervous system is quite small, the neurons are small, the electrodes we

00:38:20.734 --> 00:38:22.794
use are still pretty big, you know.

00:38:23.034 --> 00:38:28.214
There may be some advances using optogenetics where we are able to look at some

00:38:28.214 --> 00:38:33.074
of these neurons using optical recording in the future that will give us better access.

00:38:33.494 --> 00:38:39.314
But I think we're going to be forced to stick with this idea of find the general

00:38:39.314 --> 00:38:43.774
principles by looking at animals where you get the best technical access and

00:38:43.774 --> 00:38:46.434
try to build a more general model.

00:38:47.354 --> 00:38:51.054
So in that regard, I think a lot of the control principles that apply to the

00:38:51.054 --> 00:38:53.714
lobsters, certainly optical flow control.

00:38:54.734 --> 00:38:57.574
Pitch and roll control, accelerometry, tree that sort of

00:38:57.574 --> 00:39:01.414
thing will apply equally well to the bee okay so

00:39:01.414 --> 00:39:07.234
so what are the principles that that stand out for you now for let's say the

00:39:07.234 --> 00:39:15.394
control of the behavior of of these animals well certainly use of decussation

00:39:15.394 --> 00:39:18.594
and and ipsilaterally projecting uh,

00:39:19.866 --> 00:39:24.186
things give us both positive and negative feedback control that's proportional.

00:39:24.966 --> 00:39:29.266
So, for example, we can use differences in the magnitude of an input stimulus

00:39:29.266 --> 00:39:34.306
on the two sides to give us proportional control to maintain course,

00:39:34.366 --> 00:39:36.346
say, in the yaw plane, for example.

00:39:36.866 --> 00:39:40.306
So I think that's a principle that applies to all of our robots.

00:39:41.606 --> 00:39:48.146
I think using range fractionation to fractionate sensory inputs into different

00:39:48.146 --> 00:39:54.966
levels that we can use to apply different logical control projections over different

00:39:54.966 --> 00:39:58.346
ranges of stimulus magnitude is another principle.

00:39:58.806 --> 00:40:05.046
So say, for example, if you've got optical flow information going from front

00:40:05.046 --> 00:40:10.246
to rear at low velocities, you might want that to project to the opposite side.

00:40:10.446 --> 00:40:14.906
But as you approach a wall or an object, you might want it to project to the

00:40:14.906 --> 00:40:18.166
same side to cause it to steer away, for example.

00:40:18.926 --> 00:40:23.086
Right. But then you could argue if we go from lobster to the bee,

00:40:23.246 --> 00:40:27.106
in some sense, the medium is changing. We go from water to air.

00:40:27.346 --> 00:40:31.266
The scale is changing. We go from a pretty big animal to a pretty small animal.

00:40:31.586 --> 00:40:37.126
So this has quite an impact on the dynamics of the behavior. Yeah.

00:40:37.306 --> 00:40:41.766
And you could argue maybe these principles of the lobster will just fall down

00:40:41.766 --> 00:40:48.046
or crumble in the face of the scale reduction and also this change in the relationship to the medium.

00:40:48.946 --> 00:40:53.586
So the bee is following different principles, right? So what makes you hopeful

00:40:53.586 --> 00:40:55.126
that this generalization will work?

00:40:56.433 --> 00:40:58.853
Well, we're testing it right now. Okay.

00:40:58.893 --> 00:41:04.773
So right now we're building helicopter-based robots to mimic the bee,

00:41:04.913 --> 00:41:10.993
trying to work out these principles where we can use larger sensors and find

00:41:10.993 --> 00:41:15.553
ways to get the coding right and then find ways to miniaturize that.

00:41:15.833 --> 00:41:21.893
Right. And at that point, we'll sort of test the idea of whether scaling works or not.

00:41:21.953 --> 00:41:25.193
And I don't think we can really do those tests until we have the hardware.

00:41:25.193 --> 00:41:27.993
Work but then do that does it also mean that

00:41:27.993 --> 00:41:30.933
you see the b brain as a miniaturized version of the lobster brain

00:41:30.933 --> 00:41:33.913
i mean if to some extent yeah yeah this

00:41:33.913 --> 00:41:37.433
should be true right yeah yeah yeah i think for certainly if we were going to

00:41:37.433 --> 00:41:43.713
put chemoreception on the b and flow sensing etc i think those all are pretty

00:41:43.713 --> 00:41:48.693
generally organized among the animal among the arthropods okay but but bees

00:41:48.693 --> 00:41:50.453
have mushroom bodies and lobsters

00:41:50.453 --> 00:41:53.493
do not yeah yeah yeah so where does that difference and come from Well,

00:41:53.753 --> 00:41:57.833
I mean, we got to ask ourselves what the mushroom bodies are doing, you know.

00:41:57.893 --> 00:42:00.273
Good question. Yeah, yeah. That's a very good question.

00:42:00.473 --> 00:42:07.133
And I think my approach is to see what we're missing with the implementation

00:42:07.133 --> 00:42:10.433
of the reflex pathways that we know about.

00:42:11.653 --> 00:42:15.533
And that will certainly give us some indication of what we might want to look

00:42:15.533 --> 00:42:20.553
to in the mushroom bodies. Okay, well, mushroom bodies in the insect literature

00:42:20.553 --> 00:42:25.093
would be seen as a classification, a memory system, right? That's certainly

00:42:25.093 --> 00:42:26.313
my impression. Of multimodal input.

00:42:27.093 --> 00:42:31.273
So if you look at the chemical world, and I also would assume that for a lobster,

00:42:31.493 --> 00:42:34.253
most of its interactions with the world are chemical.

00:42:34.913 --> 00:42:37.673
Certainly when we talk about distal interactions with the world.

00:42:37.873 --> 00:42:42.813
So is the complexity of the kind of compounds it's exposed to lower than what

00:42:42.813 --> 00:42:43.793
you would expect from a bee?

00:42:43.993 --> 00:42:46.833
Might that be a difference that explains the absence of a mushroom body?

00:42:47.633 --> 00:42:52.853
Yeah. I mean, the lobsters really have two sets of, they have a smell receptor

00:42:52.853 --> 00:42:56.373
in the antennules and a taste receptor on the walking legs.

00:42:57.313 --> 00:43:01.733
And that's just one type of receptor. So there's no variable set of receptors.

00:43:02.173 --> 00:43:06.933
Well, Chuck Gerby's done a lot of work on this and Beriaki, and they know exactly

00:43:06.933 --> 00:43:12.093
what amino acids are responding to and in what proportion of the hair cells respond to that.

00:43:13.833 --> 00:43:18.793
Again, in that we're just beginning to deal with that, I haven't paid the level

00:43:18.793 --> 00:43:21.553
of attention that I should be paying and will be paying.

00:43:22.433 --> 00:43:26.153
You'll be back. Yeah, exactly. No, that's very good. No, but this is interesting,

00:43:26.353 --> 00:43:29.193
right? Because when you say, and this is completely plausible, right?

00:43:29.233 --> 00:43:33.533
That these, let's say functional principles to generalize, then in some sense

00:43:33.533 --> 00:43:36.533
we're committing ourselves to the implication which is, well,

00:43:36.533 --> 00:43:38.533
these brains should also generalize, right?

00:43:38.533 --> 00:43:40.973
We should find similarities between the structure because that,

00:43:41.033 --> 00:43:44.493
in the end, gives you that function, one way or the other, right?

00:43:44.613 --> 00:43:48.113
Yeah, yeah. But now one other principle that you...

00:43:49.880 --> 00:43:53.300
Mentioned is the one of reflex chaining to to

00:43:53.300 --> 00:43:56.100
let's say deal with with a bit

00:43:56.100 --> 00:43:59.320
more complex behavior so what do you mean with reflex chaining

00:43:59.320 --> 00:44:04.040
exactly well i mean i think the best example of reflex chaining is feeding in

00:44:04.040 --> 00:44:07.660
the leech the work that was originally done by mike dickinson and charlotte

00:44:07.660 --> 00:44:15.340
chuck lamb in which they found that at each stage uh of feeding that the animals

00:44:15.340 --> 00:44:17.260
would encounter reflex feedback,

00:44:17.600 --> 00:44:20.140
which would trigger the next phase of the behavior.

00:44:20.460 --> 00:44:25.560
And that went from initially perceiving the water waves to swimming,

00:44:25.680 --> 00:44:29.440
to contact with the leech, where they would then start crawling,

00:44:29.660 --> 00:44:33.520
to contact with a warm spot, where they would then start biting,

00:44:33.820 --> 00:44:41.540
to the flow of blood, which would then start them to do this suction paralysis, I mean, peristalsis.

00:44:41.740 --> 00:44:45.940
So that was one of the best examples of reflex training. What we're looking

00:44:45.940 --> 00:44:51.920
at with the RoboBee is when it leaves the hive, it's going to be given a search

00:44:51.920 --> 00:44:55.400
vector, which is a compass heading and some odometry information.

00:44:56.722 --> 00:45:01.362
We're going to have it fly out for a distance specified by the odometer,

00:45:01.582 --> 00:45:08.102
at which point it will switch to looking ahead with UV ommatidia in a 3x3 array.

00:45:08.422 --> 00:45:13.362
And in the horizontal plane, those ommatidia are going to cause yaw changes,

00:45:13.482 --> 00:45:16.782
and in the pitch plane, they're going to cause pitch changes,

00:45:16.982 --> 00:45:22.722
so that the robot will home in on a UV source, which would be a flower. Okay.

00:45:22.882 --> 00:45:29.002
Once it, it will then use optical flow information to slow itself down as it

00:45:29.002 --> 00:45:33.122
approaches the flower and then sort of bump around and get some pollen.

00:45:33.622 --> 00:45:41.402
And at that point it will reverse heading and fly back to the hive using the

00:45:41.402 --> 00:45:44.362
inverse of the odometry information that used to fly out.

00:45:44.362 --> 00:45:52.142
And then we'll have a UV LED on the hive, which you can use to home in to a

00:45:52.142 --> 00:45:53.742
docking station to recharge. Okay.

00:45:53.902 --> 00:45:56.582
So that could work in a vacuum, right?

00:45:56.742 --> 00:45:59.762
But if we now do it in an airflow, it will be drift.

00:46:00.122 --> 00:46:03.342
So how is your autometer going to be corrected for drift?

00:46:03.342 --> 00:46:08.902
Well, we're going to use optical flow information to respond to deviations of

00:46:08.902 --> 00:46:15.602
rotation that are superimposed on the normal translation to get it to come back on heading.

00:46:16.042 --> 00:46:22.942
Okay. And we'll probably periodically, if it exhibits a big deviation of that,

00:46:23.142 --> 00:46:27.962
we'll probably get it to listen to its compass a bit and get back on heading.

00:46:28.122 --> 00:46:29.742
Have you considered using a solar compass?

00:46:30.682 --> 00:46:35.742
We're toying with that idea. So we have some people on the team that are involved

00:46:35.742 --> 00:46:41.222
in the whole visual sense of the bee and the idea of a sun compass that uses

00:46:41.222 --> 00:46:43.462
polarized light is something we're exploring.

00:46:43.602 --> 00:46:47.282
All right. That would be a powerful solution. So now here we have the robot bee.

00:46:47.662 --> 00:46:50.162
It's like a scaled down version of the lobster. Sure.

00:46:51.431 --> 00:46:54.891
Okay, this is my claim. This is not your claim. Okay, but I'm sort of summarizing this.

00:46:55.571 --> 00:46:59.191
So we have these sort of more uniform principles that we try to identify.

00:46:59.591 --> 00:47:01.511
But the other thing, you also really want to deploy this.

00:47:02.351 --> 00:47:06.571
So that means there's a lot of engineering involved to make such a system actually really work.

00:47:06.811 --> 00:47:12.711
I mean, you can have these ideas based on neurothology, how we can control it.

00:47:12.851 --> 00:47:15.931
But to get it done is actually not the story. We have to think about power,

00:47:15.991 --> 00:47:17.571
sensing, integration, computation.

00:47:18.331 --> 00:47:21.811
So how are you going to get that done? Well, there's a big team.

00:47:21.871 --> 00:47:27.891
We have eight other investigators that are charged with addressing all those parts.

00:47:28.151 --> 00:47:33.331
And my part is really to focus on the innate control of flight behavior.

00:47:34.031 --> 00:47:38.151
There's another crew that's doing colony interactions. There are people that

00:47:38.151 --> 00:47:39.691
are building the airframes.

00:47:39.691 --> 00:47:44.451
There are people that are doing the wings, the airfoils. There are people doing the power supply.

00:47:44.451 --> 00:47:48.031
There are people doing the and there are people doing the power electronics

00:47:48.031 --> 00:47:50.951
to provide power to the actuators.

00:47:50.951 --> 00:47:54.411
So this is not a one-man operation by any stretch.

00:47:54.411 --> 00:48:00.591
Right. And the PI, Rob Wood, has really put together an extraordinarily elegant

00:48:00.591 --> 00:48:02.311
plan for coordinating this.

00:48:02.311 --> 00:48:05.571
And we're in our second year right now and and boy it's

00:48:05.571 --> 00:48:09.511
looking good yeah okay yeah i mean i'm uh we just

00:48:09.511 --> 00:48:12.751
did our our second year progress report to nsf we

00:48:12.751 --> 00:48:15.511
had a site visit and and they were quite

00:48:15.511 --> 00:48:18.911
happy with where we're at at this point uh-huh right oh that's

00:48:18.911 --> 00:48:22.071
impressive so when are you going to see the first one fly well they

00:48:22.071 --> 00:48:24.971
there are flying now but they're flying in sort

00:48:24.971 --> 00:48:30.791
of open loop and they do crash and burn and we're we're we're flying the helicopters

00:48:30.791 --> 00:48:35.351
that's where we're really try to do the ground truth experiments on all these

00:48:35.351 --> 00:48:40.891
sensor systems so when will we see the first one fly autonomously for longer

00:48:40.891 --> 00:48:44.411
than 10 minutes the b yeah.

00:48:46.871 --> 00:48:53.211
The current expectation with the available power source rehab right now is on

00:48:53.211 --> 00:48:56.911
the order of three minutes right okay um,

00:48:58.871 --> 00:49:06.251
the whole issue of power for these things is an area of intense exploration.

00:49:07.131 --> 00:49:10.751
We're looking at varieties of chemical batteries.

00:49:11.111 --> 00:49:16.091
We're looking at solid oxide fuel cells, and we're also looking at supercapacitors.

00:49:16.591 --> 00:49:20.271
So there's a broad variety of power supplies we might employ,

00:49:20.531 --> 00:49:24.871
and this is why this is not a one-year project.

00:49:25.111 --> 00:49:27.931
Right, exactly. you know but it's sort of humbling right

00:49:27.931 --> 00:49:31.831
because we're sitting here having all these concerns about the brain but we

00:49:31.831 --> 00:49:35.091
can give these things a super brain and still they won't fly because we just

00:49:35.091 --> 00:49:38.951
have don't have the right power sources yeah so there are some other problems

00:49:38.951 --> 00:49:43.871
and and how does actuation look how reliable is actuation really the wing the

00:49:43.871 --> 00:49:46.831
wing control extraordinary and um.

00:49:48.341 --> 00:49:52.481
Where my colleagues are in the process of writing up the fabrication process,

00:49:52.841 --> 00:49:54.301
which is truly impressive.

00:49:54.501 --> 00:50:02.021
And out of deference to their intellectual property rights, I think it's best

00:50:02.021 --> 00:50:05.041
to not talk about that. Right. But we're just going to say it's fantastic.

00:50:05.401 --> 00:50:08.841
Yeah, it's fantastic. I am a true believer right now.

00:50:08.961 --> 00:50:14.721
There was a period of time where I was skeptical, but I really think this is going to happen.

00:50:14.721 --> 00:50:20.161
And I think you hit the nail right on the head that getting the power to get

00:50:20.161 --> 00:50:25.461
the appropriate duration of flight is one of the bigger challenges we face. Right, absolutely.

00:50:26.141 --> 00:50:29.001
So here we have to be.

00:50:29.561 --> 00:50:34.061
But now another challenge that you highlighted was this whole issue of chemical sensing, right?

00:50:34.441 --> 00:50:37.641
So chemical sensing is a really interesting problem. I think it's also very

00:50:37.641 --> 00:50:38.881
much an underestimated problem.

00:50:40.221 --> 00:50:45.041
And you could also argue that this might be the oldest sense that we have from

00:50:45.041 --> 00:50:46.121
an evolutionary perspective.

00:50:46.341 --> 00:50:50.401
This is how cells will start to deal with the world, it's through chemical sensing.

00:50:51.701 --> 00:50:55.821
So what you were sketching is that in terms of our technology for chemical sense,

00:50:55.921 --> 00:50:59.961
if you want to understand this biological phenomenon and again build the technology,

00:51:00.121 --> 00:51:04.241
right now this technology is not good enough in terms of sensitivity and robustness and so on.

00:51:04.241 --> 00:51:10.721
And you are seeking to overcome this, taking a sort of a synthetic biology approach, right?

00:51:10.841 --> 00:51:13.981
So what does it actually really mean with respect to chemical sensing?

00:51:14.281 --> 00:51:19.381
Okay. With chemical sensing, there's basically three processes that we have to deal with.

00:51:19.621 --> 00:51:25.001
One is the actual sensor itself, the receptor that binds an odorant molecule

00:51:25.001 --> 00:51:33.061
that is going to cause some change in the cell, which might be an inward current.

00:51:33.061 --> 00:51:39.361
It might be an enzymatic cascade that leads to a great amplification of the

00:51:39.361 --> 00:51:44.781
response so that you might get a big cellular response to a single input molecule.

00:51:45.241 --> 00:51:49.781
And then some sort of reporter that's going to report to us that,

00:51:49.821 --> 00:51:53.701
in fact, this cell has contacted the odorant.

00:51:53.801 --> 00:51:59.681
And then finally, some transduction mechanism by which we take the reporter

00:51:59.681 --> 00:52:03.441
and activate of eight is sensory neuron and electronic nervous system.

00:52:03.521 --> 00:52:05.941
So those are the three levels that we're working at.

00:52:06.141 --> 00:52:12.121
Now, clearly for the actual receptor, we're going to use some sort of G protein coupled receptor.

00:52:13.089 --> 00:52:17.209
And in some of these, they can be coupled with a calcium channel through a G

00:52:17.209 --> 00:52:22.049
protein, such that when it binds an odorant, there'll be an inward current of calcium.

00:52:22.049 --> 00:52:27.609
We can also have a G-protein coupled receptor that's coupled to nitric oxide

00:52:27.609 --> 00:52:34.209
synthase so that the cell will generate nitric oxide, and then we can use a

00:52:34.209 --> 00:52:35.789
nitric oxide electrode,

00:52:35.969 --> 00:52:42.809
which is basically a Nafon membrane over a pair of silver and carbon electrodes

00:52:42.809 --> 00:52:46.669
that would report with nitric oxide.

00:52:46.929 --> 00:52:50.429
Okay. Okay, so then those electrodes,

00:52:50.669 --> 00:52:54.689
which might be a photodiode that responds... Well, wait a minute.

00:52:54.769 --> 00:53:00.109
When the calcium in the first case enters the cell, we would put an aqueorin

00:53:00.109 --> 00:53:03.029
and silenzorime, which give off light.

00:53:03.209 --> 00:53:06.849
Or we might put a luciferase, which gives off light.

00:53:07.069 --> 00:53:11.309
And then our actual transduction mechanism would be a photodiode.

00:53:11.449 --> 00:53:20.529
Right. Mm-hmm. But so, what will be the spectrum of compounds such a sensor can be sensitive to?

00:53:20.949 --> 00:53:23.809
Well, there are a broad variety of things.

00:53:24.529 --> 00:53:28.229
Pretty much anything you can smell has a G-protein coupled receptor.

00:53:28.489 --> 00:53:35.569
And if you go to the library of biological parts at MIT, you can find a library of receptors.

00:53:35.949 --> 00:53:38.509
And you can use those genes.

00:53:39.569 --> 00:53:44.149
There are some two-component receptors, for example, that respond to RDX,

00:53:44.329 --> 00:53:46.529
which is the explosive in C4.

00:53:47.089 --> 00:53:52.869
And Jude Mulford has done some work with this where they, if you've seen these

00:53:52.869 --> 00:53:58.989
experiments where they put a explosive receptor into grasses and the grasses

00:53:58.989 --> 00:54:01.329
change color if they're planted over a mine.

00:54:01.929 --> 00:54:05.289
So there's some really interesting tricks that can be put there.

00:54:05.289 --> 00:54:07.969
There but then if you want to have multiple compounds you'll have

00:54:07.969 --> 00:54:10.909
multiple diodes yeah right so you

00:54:10.909 --> 00:54:13.769
might have a problem at your readout level then yeah yeah

00:54:13.769 --> 00:54:16.709
yeah so you can have this reporting with light of different

00:54:16.709 --> 00:54:23.169
wavelengths or with nitric oxide okay and and we can there's a uh a charge coupled

00:54:23.169 --> 00:54:29.709
kind of photodiode uh that we can put filters over and then we can get different

00:54:29.709 --> 00:54:34.649
filters to respond to different but then you would have a very so then in some of the very local cool.

00:54:35.487 --> 00:54:43.547
But if you look at bees, it's true for your lobster, they have many chemoreceptors

00:54:43.547 --> 00:54:47.847
distributed over their antenna, over parts of the body.

00:54:48.007 --> 00:54:53.687
And this information arguably is also used for detection and localization.

00:54:54.207 --> 00:54:59.707
So how do you scale up from let's say this very localized detection and readout

00:54:59.707 --> 00:55:06.307
to this more global, spatially organized way of detection and reporting to some

00:55:06.307 --> 00:55:08.707
controller what's going on out there. Yeah, yeah, yeah.

00:55:08.787 --> 00:55:14.527
Well, I think we've got to create sense organs, and the sense organs might have a variety of sensors.

00:55:14.907 --> 00:55:21.207
And one of our ways of increasing the scale of the kinds of features that we

00:55:21.207 --> 00:55:26.187
can use is to use a new technology called EJET printing.

00:55:26.187 --> 00:55:33.687
And EJET printing is a new technology that was developed by John Rogers and

00:55:33.687 --> 00:55:37.747
Andrew Elwine at University of Illinois at Urbana-Champaign,

00:55:37.927 --> 00:55:44.567
in which they use a sputter-coated microelectrode that's sputter-coated with gold.

00:55:44.567 --> 00:55:47.727
And they put an ink in that and then

00:55:47.727 --> 00:55:50.607
put that over a nanometer stage that can

00:55:50.607 --> 00:55:53.887
move in x and y planes in nanometer increments

00:55:53.887 --> 00:56:00.087
and then use very high voltages to put droplets to print very very small droplets

00:56:00.087 --> 00:56:04.027
and they've been able to print protein features that are five microns in diameter

00:56:04.027 --> 00:56:11.987
so then we can use this to create very very fine detail on an electrode system

00:56:12.187 --> 00:56:17.627
that could be combined several photoreceptors on the same sense organ.

00:56:17.767 --> 00:56:24.087
And the sense organ might be constructed on a glass cover slip, for example,

00:56:24.307 --> 00:56:30.507
where we glue the bacterial cells that have the receptors we want on one side,

00:56:30.567 --> 00:56:34.947
and then have the photodetectors on the opposite side of a piece of glass,

00:56:35.087 --> 00:56:38.627
and we could be sniffing for several different things at once there.

00:56:38.727 --> 00:56:41.787
Right. So that's one way we could approach that, yeah. Okay,

00:56:42.007 --> 00:56:43.567
that's pretty impressive.

00:56:44.207 --> 00:56:48.947
So in chemical sensing, another issue is that, for instance,

00:56:49.027 --> 00:56:54.907
if you look at the moth, the male moth, the sensitivity at the periphery is

00:56:54.907 --> 00:56:59.047
an order of magnitude or so lower as that at the neural end.

00:56:59.287 --> 00:57:02.407
That's when you can look at changes in heart rate due to a pheromone,

00:57:02.407 --> 00:57:06.427
and you can see that you can already detect changes in heart rate at concentration

00:57:06.427 --> 00:57:09.147
levels that single receptors cannot detect reliably.

00:57:09.687 --> 00:57:15.967
So there's a boost somewhere occurring in the neural processing of these signals.

00:57:17.087 --> 00:57:20.727
So are you already looking into that issue or that's the future?

00:57:21.127 --> 00:57:23.407
That's the future, definitely. Okay.

00:57:23.887 --> 00:57:28.347
So now, so okay, we have the lobster, we have the bee project,

00:57:28.567 --> 00:57:31.247
and this one was in the robotics domain, right?

00:57:31.367 --> 00:57:34.127
So what's cooking in parallel to this in the neurophysiology?

00:57:38.197 --> 00:57:43.857
At this point, we have proposals under review that I don't think I should be talking about.

00:57:44.977 --> 00:57:48.557
But they're fantastic. They are fantastic. Exactly.

00:57:49.357 --> 00:57:51.817
And in that they're under review

00:57:51.817 --> 00:57:54.537
at this point, I just don't think it's appropriate to bring that up.

00:57:54.637 --> 00:57:56.957
No, don't worry about it. We have plenty of ideas, believe me.

00:57:57.397 --> 00:57:59.077
Oh, yeah, no, I'm sure about that.

00:57:59.537 --> 00:58:04.677
So in a broader sense, right, outside of the specifics of your proposals,

00:58:04.897 --> 00:58:06.957
where do you see this field go?

00:58:06.957 --> 00:58:09.657
Go do you see that the field of biomedics as you

00:58:09.657 --> 00:58:13.277
envision it is actually having an impact is it changing the way we do science

00:58:13.277 --> 00:58:18.697
certainly so yeah yeah and i think really what's going on here is that we've

00:58:18.697 --> 00:58:23.617
switched from analytical neuroscience to synthetic neuroscience and it's time

00:58:23.617 --> 00:58:28.917
we start building things using principles that we established through analytical neuroscience.

00:58:29.857 --> 00:58:34.297
Yeah but i could also argue that this might be true for you and very few of

00:58:34.297 --> 00:58:38.457
your colleagues but that the majority of neuroscience has ended up in more sort

00:58:38.457 --> 00:58:40.097
of a massive data-gathering exercise.

00:58:40.557 --> 00:58:46.637
So will this have an impact to the majority of neuroscientists and how they

00:58:46.637 --> 00:58:47.997
think about their discipline?

00:58:48.477 --> 00:58:54.057
Well, I think it will depend on how many of them read our papers and take advantage

00:58:54.057 --> 00:58:56.417
of the knowledge we've been able to create.

00:58:57.277 --> 00:59:01.037
But I understand you're still optimistic. I'm very optimistic.

00:59:01.197 --> 00:59:03.657
I think this is really where it's got to go.

00:59:04.597 --> 00:59:07.997
I mean, this is the wide open frontier of neuroscience.

00:59:08.597 --> 00:59:14.457
There is so much opportunity here. And I think that my colleagues that lag in

00:59:14.457 --> 00:59:17.137
getting involved in this are going to miss out.

00:59:18.357 --> 00:59:22.417
But so then the other issue is how do you see the scale up? Because I could

00:59:22.417 --> 00:59:25.177
argue like, well, that's nice and well for the bee.

00:59:25.337 --> 00:59:29.657
But in some sense, who cares about the bee? That's not really advanced organism.

00:59:29.657 --> 00:59:32.637
Organism why don't you show me something around the

00:59:32.637 --> 00:59:35.717
macaque or why don't you go up to humans or

00:59:35.717 --> 00:59:38.877
even a rodent you know that would be more impressive so

00:59:38.877 --> 00:59:43.617
how do you see the scaling up challenge i'm quite satisfied eating my experiments

00:59:43.617 --> 00:59:49.797
and the lobsters are an extraordinary model for that and if i just uh create

00:59:49.797 --> 00:59:56.257
a truly biomimetic lobster that does all the things that lobsters do I think

00:59:56.257 --> 00:59:57.517
that will be quite an accomplishment,

00:59:57.737 --> 01:00:00.177
you know, I, I.

01:00:02.323 --> 01:00:09.623
I'm not particularly interested in working on mammals. I do enjoy some of the

01:00:09.623 --> 01:00:14.763
technical advantages of working on lower, simple lower vertebrates like sea lamprey.

01:00:15.243 --> 01:00:21.023
But in terms of the warm-blooded animals and consciousness and all that,

01:00:21.083 --> 01:00:22.843
I'll leave that to others. Okay.

01:00:23.643 --> 01:00:28.963
But you see this more as a personal idiosyncrasy or that's really also a very

01:00:28.963 --> 01:00:30.183
clear scientific strategy?

01:00:30.183 --> 01:00:34.643
In other words, you say, look, if I can crack this nut of the lobster or of

01:00:34.643 --> 01:00:38.763
the bee, I can capture these general design principles, then I'm really close

01:00:38.763 --> 01:00:41.823
to understanding any other brain that evolution has generated.

01:00:42.063 --> 01:00:47.683
Yeah, but I think we need to figure out the lobster first, and that may be my

01:00:47.683 --> 01:00:49.283
own personal idiosyncrasy.

01:00:49.723 --> 01:00:54.763
Okay, right. So then to finish up, two questions.

01:00:55.023 --> 01:00:58.183
So you came a long way in some sense.

01:00:58.223 --> 01:01:02.843
I'm quite sure when you were postdoc, you weren't really thinking about ending

01:01:02.843 --> 01:01:03.943
up talking about robots.

01:01:04.323 --> 01:01:10.623
I'm not sure, but this is what I imagine. In fact, the whole robot business came up 10 years ago.

01:01:10.803 --> 01:01:15.403
And if you had asked me 11 years ago if I was going to be working on robots,

01:01:15.743 --> 01:01:18.223
I would have given you an odd look.

01:01:18.863 --> 01:01:22.643
Exactly. But you tend to do that anyway, I realized.

01:01:25.623 --> 01:01:29.583
But so on the basis of all this experience in terms of trying to understand

01:01:29.583 --> 01:01:34.863
the brain, what's this one law of John Ayer you would like to give us?

01:01:35.543 --> 01:01:39.143
The John Ayer law. The Ayer's principle. Ayer's principle, right?

01:01:39.143 --> 01:01:42.263
The Ayer's principle is choose a model organism you can eat.

01:01:45.723 --> 01:01:50.463
You can eat or you're allowed to eat. There's no problem eating lobsters. Okay.

01:01:51.743 --> 01:01:56.763
All right. And I do have a recipe in my cookbook for honey smoked lobsters.

01:01:57.083 --> 01:02:00.103
We're beginning to integrate all these lines of work.

01:02:02.383 --> 01:02:08.303
So the last question is, five years from now, I'm going to trace you down and

01:02:08.303 --> 01:02:10.623
I'm going to confront you with the prediction you're going to give me today.

01:02:10.623 --> 01:02:16.503
So what's this one strong prediction you would commit yourself to today in this

01:02:16.503 --> 01:02:21.023
domain of building lobsters, understanding lobsters, building bees, understanding bees?

01:02:21.223 --> 01:02:23.703
What's the one prediction that you care about the most? Well,

01:02:23.703 --> 01:02:28.363
I believe that we're going to be able to produce vehicles that will...

01:02:30.338 --> 01:02:37.478
To realize what i call reactive autonomy uh at the scale certainly of a of a lobster,

01:02:38.038 --> 01:02:40.878
uh within five years i have no doubt of that

01:02:40.878 --> 01:02:47.438
and that that looks very very clear at this point all right and so where i think

01:02:47.438 --> 01:02:53.058
i should define by what i mean by supervised autonomy well we actually we call

01:02:53.058 --> 01:02:58.398
it supervised reactive autonomy right and the idea is that say if you take your dog for a walk.

01:02:58.678 --> 01:03:01.978
Your dog is autonomously following you around.

01:03:02.818 --> 01:03:09.258
If you chose to throw a stick out in the water and the dog swims out and retrieves

01:03:09.258 --> 01:03:14.278
the stick, it's reactively autonomous under your supervision.

01:03:14.878 --> 01:03:18.138
And it's doing that portion of the mission on its own.

01:03:18.838 --> 01:03:25.318
And certainly my sense is that the people that would like to be operating robots

01:03:25.318 --> 01:03:29.178
in the field would like to have that level of control over them.

01:03:29.658 --> 01:03:33.778
I don't, I think the idea of these things just going off and dreaming up what

01:03:33.778 --> 01:03:36.698
they want to do on their own is not the best idea.

01:03:36.738 --> 01:03:40.058
And we wouldn't even want our dogs doing that, you know?

01:03:40.098 --> 01:03:44.798
Right. So I think maintaining some modicum of supervision, uh,

01:03:44.978 --> 01:03:51.018
is, is the best idea, but I think we can truly expect these robots to be able to perform,

01:03:52.118 --> 01:03:55.298
small aspects of missions completely autonomously.

01:03:56.458 --> 01:03:58.878
But okay, now that you went to qualify autonomous.

01:04:01.438 --> 01:04:04.578
There's this issue that controls often also illusion, right?

01:04:04.618 --> 01:04:08.478
We often believe we control our technology, and that we can have controlled

01:04:08.478 --> 01:04:13.238
autonomy, and then in practice turns out actually our control was an illusion.

01:04:13.578 --> 01:04:16.358
There was no real, also partially your dog might be conditioning you,

01:04:16.618 --> 01:04:20.638
and just thinks, well, look, he likes to throw sticks, so I bring him another one.

01:04:21.018 --> 01:04:25.378
So, to what extent can you actually

01:04:25.378 --> 01:04:29.938
really quantitatively constrain and define this notion of control?

01:04:31.977 --> 01:04:36.897
That's an interesting issue. Well, we're certainly going to have control over

01:04:36.897 --> 01:04:39.477
the amount of power and the mission duration they're going to have.

01:04:39.737 --> 01:04:47.457
So I think that's going to give us ultimately a degree of control that we can be confident in.

01:04:47.457 --> 01:04:56.217
I think the general impression of the state of the art of robotics is that the

01:04:56.217 --> 01:05:04.077
state of the art of autonomy is for a teleoperated vehicle to be able to recover

01:05:04.077 --> 01:05:07.137
its teleoperative link if it loses it.

01:05:07.137 --> 01:05:11.737
And I think there's this really interesting example that occurred last summer

01:05:11.737 --> 01:05:17.417
where the Navy was doing an exercise in Chesapeake Bay with 13 Remuses,

01:05:17.417 --> 01:05:20.637
and they lost four of them.

01:05:20.817 --> 01:05:24.017
What are Remuses? Remuses are a torpedo-like robot.

01:05:24.137 --> 01:05:29.757
It's the only autonomous vehicle in the Navy inventory right now.

01:05:29.757 --> 01:05:32.917
And they were operating three of

01:05:32.917 --> 01:05:36.317
them as a group and four of

01:05:36.317 --> 01:05:39.377
them lost their teleoperative link and got lost

01:05:39.377 --> 01:05:42.357
and they ended up recovering one of

01:05:42.357 --> 01:05:46.557
the four that were lost was a marine mammal which is what they were intended

01:05:46.557 --> 01:05:50.777
to replace in the in the beginning right so i think that will give you this

01:05:50.777 --> 01:05:54.857
the good quantitative measure of the state of the art right exactly it's it's

01:05:54.857 --> 01:05:58.897
sobering so joe ayers thank you very much for this conversation it's a pleasure

01:05:58.897 --> 01:06:01.037
Pleasure. That's great.

01:06:03.477 --> 01:06:09.297
The CSN podcast was produced by the Convergent Science Network of Biometrics

01:06:09.297 --> 01:06:15.697
and Biohybrid Systems, a project funded by the European 7th Research Framework Program.

01:06:17.317 --> 01:06:22.577
For more interviews, recorded lectures, or upcoming conferences in the field

01:06:22.577 --> 01:06:28.817
of biometrics and biohybrid systems, go to csnnetwork.eu.

01:06:29.177 --> 01:06:30.997
And thank you for listening.

01:06:29.360 --> 01:06:36.720
Music.