This is the Convergent Science Network podcast. Leading researchers in the domain
of neuroscience, brain theory and technology are interviewed by Paul Verschure and Tony Prescott.
Paul Verschure here with the Convergent Science Network podcast,
together with my colleague Tony Prescott.
And we're speaking now with Christine Icarly. Yes. Welcome to the podcast, Christine. Thank you.
Christine, you spoke this morning about, if you want, responsible research and
responsible innovation.
So how would you define responsible research and innovation?
There are many definitions out there and they don't all align.
If I stick to the one that we have been trying to follow in the human brain project,
we are using the one proposed by the Engineering and Physical Science Research
Council in the United Kingdom,
which is more process-oriented in that it proposes to anticipate,
reflect,
engage, act, kind of area, framework, where you are trying,
when you're looking at an innovation process and new technology,
to anticipate on the potential implications for society that this particular
technology could lead to,
that you analyze it and reflect on it, and reflect on it with the scientists
themselves by bringing in as many perspectives as you can to bear on the topic,
trying really to widen your inquiry to...
Try to understand how this kind of narrow technological scientific development
could pan out in society based on political, economic, social drivers out there.
The engagement part is part with the scientists, part with like seeking all
this kind of complementary expert opinion,
on key sticky points you are going to identify while you are doing this kind of horizon scanning.
And then the act part, which is, like I said this morning, the hard bit,
closing the loop between anticipation and acting,
Which is getting back to the scientists with not necessarily normative frameworks
or ethical frameworks belonging to any kind of schools,
but with a bucket full of, for the school of social come from,
more empirical findings that are going to be things to bear in mind that matter.
To think of when you are...
Going on about your research and acting accordingly, which for me is sort of
dissociating between when you come into hard problems, hard ethical problems.
You are hardly ever going to resolve them on your own. Otherwise, they are not that big.
But you can try to disentangle how much belongs to a policy layer.
And that's a certain kinds of of action with certain kinds of actors that can
target the policy level.
How much of it is about eventually doing some, I would say, bad word,
lobbying, but really interacting with constituencies, civil society groups,
different kinds of stakeholders, and make them aware of certain things you have found.
And then the level of really, for me, one of the most interesting things for
me as a researcher is going back to the level of the researcher themselves.
So really the level of the professional activity and the self-regulation of the professions.
Because usually this is what the researchers are at the cutting edge.
They are at the forefront.
And they are also at the forefront of a governance that is always lagging behind.
So for me, this is a very important layer to try and address.
So I think the area framework is kind of targeting researchers themselves to
think about what they're doing, why they're doing it, and so on.
If we take larger-scale projects like the Human Brain Project.
At what point do you think this sort of push for individual action for researchers
to take more responsibility for what they're doing, at what point do you need
to complement that with,
at the level of the project,
some integrated approach to research governance that ensures delivery of the framework?
It's actually in the act part because this is for instance what we saw when we,
because the very first thing we tackled as the ethics and society program of
the Human Brain Project was the question of data protection and privacy because
it was a kind of burning issue in view of the plans,
the research plans of the medical informatics platforms and then became apparent
that it was not just a medical informatics platform and there was a lot of potential
sticky point with animal data, with human data, and there was a need to sort
of coordinate an entire governance strategy across the projects.
And that's at this point that we sort of stepped up beyond the level of the
individual researchers to also to advocate, for instance, for the creation of
a job of data protection officer,
which was basically not something that the ethics and society subproject could
decide on their own, but was something that needed the approval of the,
well, first of the science and infrastructure board of the project,
of the directorate, and eventually of the European Commission to accept that
such a job was going to be created for the project and take up some of the funds of the coordination.
So I would say that's some of the actions you can take. But one of the things
I said this morning as well is that one of the main issues with this kind of
project is that when you find those kind of...
You identify those...
The things that would need action at a kind of higher policy level,
so policymakers or other constituency,
because responsible research and innovation or whatever you want to call it,
is implemented at project level, it's sort of, I would say, to be really.
For me to get this and become really effective,
it should be at the level of entire research programs and actually funders themselves
should be submitted to responsible research and innovation practices when they
define research programs, funding strands, etc.
Because once the European Commission or the Research Council UK have decided
that they are going to fund a number of particular strands of research and they
are going to push for or a certain way of approaching,
for instance, science with and for society, etc., etc.
Part of the frame is already set. And we are not asked as people trying to implement
responsible research and innovation practices and to foster them,
to question the way into these research programs have been framed, into these projects,
where these projects are fitting.
And there is definitely, for me, a
need to eventually go one step up and for research councils and funders themselves
to accept that they should themselves
include this kind of reflective approaches in what they are doing.
It sounds to me that you're advocating something a bit more radical than maybe
what has been implemented so far.
Because in terms of data management, for instance, in a way that's a continuity
with what has been traditionally seen as reach ethics around sort of human participants, use of animals.
Data management is a consequence of all this data we're now collecting and concerns about privacy.
But the response to an innovation idea is, as you say, it's anticipate.
So there is something more proactive about RRI than is perhaps captured by these
steps towards extending traditional ethics, if you like.
Yes. And since it's meant to be proactive and to anticipate,
if you are entering something where a certain frame has already been defined
and is already guiding research in many ways,
and that this frame is not to be questioned, and
somehow this can get in the way of the way you anticipate and the way you can
deploy actions is already constrained by this pre-existing frame of usually
the research followers or the policymakers.
But if I understand you correctly, the model you're advocating is very much
one of collective reflection and consensus. consensus, if I get it right.
So, the most important aspect of responsible research, in other words,
ethical research, is that all the participants, all the stakeholders commonly
reflect on the process of research.
Is that correct? Would that be a very exciting?
That's right. Whether there is a consensus coming out of it is… Secondary.
Is secondary. Would be the goal to sort of at least exchange views. Would be the goal.
And sometimes I don't think you can always reach consensus.
The idea that's one of the things that is very...
In the way responsible research and innovation is deployed is that by this kind
of participatory engagement, you reach consensus.
You don't always reach consensus. And sometimes people just get antagonistic.
And then you don't have mechanisms to resolve the disputes.
But now, if reflection then is the core value, right, of ethical research,
at best, it's a necessary condition, right?
Because we can reflect collectively a lot, and we can all agree to build the
world's most devastating porcelain gas, because that's what we believe needs to be done,
which from the outside might look like a rather unethical decision, right?
So how then is reflection actually of operationally?
Because in science, when we deal with questions of truth, truth,
in some sense, now we have the orthogonal perspective of right and wrong.
And also scientists, of course, in the end, have to make those decisions.
Like, can I perform this animal, this experiment on this animal?
Is the payoff sufficiently justified to induce this kind of suffering?
Or should I take an alternative approach? These are the very concrete decisions
that the scientist faces in which they need guidance.
Right so how does the reflection if that's our core value with its own limitations
as i just sketched and help us as scientists to make better and more ethical
decisions in how we pursue science,
um i would say um.
This kind of for me this the diverse diversity of using in in in being included
in participation mechanisms,
might not necessarily aim for consensus.
But I think that as many views as possible are always more valuable than thinking on your own.
And in the case of, for instance, animal experimentations, and on deciding,
I think you are doing this research with one particular aim in mind.
But then it might be interesting to bring into the process if an outcome is
towards eventually doing some new medications or some new treatments or I don't know what,
to bring in the views of the patient groups might be implicated.
I won't say animal activists because you're going to know exactly what they
are going to say. But, um...
In the end, I would say we can, I do believe that scientists are sort of most of the time behaving,
you know, they think about what they do.
Sometimes they are, they get too narrowly focused, which is why I really advocate
for getting more views into the process, because sometimes it can trigger other ways of seeing things.
But ultimately, it's going to be the scientists themselves, with the help of
their research ethics board, deciding whether they go for some kind of experimentation or not.
What we have been doing, for instance, in this kind of particular case in the
Human Man Project of data protection and all the data governance things… Well,
but that's not an issue, right?
I understand that… But we're also setting an ethical checklist where people
can go through and go through the branches.
For me, that's a different discussion because there we also face new regulation
by the European Commission on management of personal data, which,
of course, is very much driving that change.
I don't see that that's necessarily a good example if you look at responsible research.
It's a bit of a different topic for me. and what I was trying to get at,
and also you said it now, in some sense, the fundamental premise of your approach
is that in the end, humans know what is good.
That's what you're saying, right? Scientists in the end will know what is good
because they will think about what they do. That seems a very strong assumption.
It's my... I mean, they know what they want to do.
That's different. And...
And I don't think they all want to do things that I think personally are good.
But good and bad is very subjective. But also people are put in cultural situations
where it makes it difficult to act.
It makes it very difficult to… The way that it is difficult doesn't mean we
shouldn't try to address it.
So that's why I gave you an example where I say, well, as a researcher,
you must make ethical decisions on are the expected results worth the suffering
of this animal? That's an example, right?
It's a very concrete situation. And that's where people need guidance.
So how would you go about providing guidance from a responsible research perspective?
It's really trying to exactly evaluate what are the expected benefits.
Benefits, and the expected,
you know, I mean, depending on the people you are going to talk to,
no outcome is good enough to be worth an animal's life. No outcome.
For some people. For some people. No, no. For some people.
So that's what I mean, you know, right and wrong is very subjective and depending
on culture and on cultural beliefs.
For some people, very little is going to be... So.
But it doesn't solve. It's not going to solve your problem.
It's just going to help you get as many cards in your hands as possible about
what are the benefits, what are the downsides, and so on and so forth.
And eventually to give you an ethical training into thinking whether you buy
into consequentialist ethics or virtue ethics or duty ethics.
But as you know, ethicists from these different schools tend to kill each other.
You can't even go to an ethicist and ask them what they would do.
So in the end, it falls onto your… This is exactly where I want to get to.
Are you getting too much at the personal ethics? because, you know,
it's, and I accept that the area framework maybe speaks to the personal ethical stance of researchers,
but the challenge we really have is that we have a culture of science which
puts people in a double bind situation where,
you know, having a job, being able to support themselves and their families
puts them in a situation of having to do things which they may be ethically uncomfortable with.
And so what you want to do is create a research culture that doesn't create
the situation for people.
So it has to be at the level of the organization that you, well,
you can't avoid these clashes entirely,
but you can certainly, if you apply the area framework at the organizational level,
then you should be able to create a culture which is different.
So the point is, I think, so what I was trying to get at, as we saw earlier,
right, the core value of responsible research, as we now discussed it,
was a reflection, right?
And also we've seen that in reports to the European Commission. And I was just saying….
In the Wednesday conference, there was a lot of reflection on the final solution.
So obviously, reflection as such is not enough.
We have to insert certain priors into that discussion.
This might be human rights considerations. This might be one angle you can take, right?
It might be more ideological, political, religious considerations.
But there must be additional priors you insert in that debate,
even though it's difficult.
And that's the whole point, because it's difficult to want to deal with it,
right? John F. Kennedy, remember, we go to the moon because it's difficult.
So we kind of run away from it and just wave our hands and then diffuse responsibility,
say, oh, there's so many opinions in the room, we don't know what to do.
No, the real problems are there.
And so we must be willing to actually debate on foundational grounds these fundamental challenges.
And it also means that within research projects, we must have frameworks that
define what's right and what is wrong.
Otherwise, we're just wasting our time and running around in circles.
So I'm saying reflection is your core value is not enough, as history has shown.
We must go beyond that. And I don't give you an answer to what that should be,
but I'm saying that's a debate we must have as responsible citizens and as responsible scientists.
And I don't hear enough about that from ethics and ethics is a little bit about
the precautionary principle. Well, let's see what what happens?
That's not good enough anymore because we can have runaway technology,
the repercussions of which are irreversible.
So we must have a much more proactive stance. That's what I'm arguing for.
So I just wanted to feel out that domain, to say great, I understand,
but I feel it's not enough.
And I feel we're losing the game if we insist on this wait and see and let's
give everybody a voice in the process approach.
I think from an operational perspective, it has no value.
Effectively, collectively, you can end up involving, you know,
that's the problem of participatory democracy and all this kind of,
it's difficult to take decisions.
What I find that's where I sort of align with what you say is that I agree that
the ARIA framework is a lot about personal responsibility and thinking about your own work.
But I think that when analyzed properly,
it's all this process for me,
and that's why I say it's a shame it's only implemented at project level,
is that it also helps disentangling where it's really something an individual
scientist or lab head can take a decision about being reasonably informed.
But there are these kind of double bind situations or impossible decisions to make where it's not,
it should not be at the individual level and where collective mechanisms do
not exist and policy is lagging.
And and and so that's where i mean we've seen it in the human brain project
a lot is that you see things that should be done at a higher level than the
project in the way for instance the project the project officers are um.
Sort of running the project. But we don't have this mechanism by which we can
go back one step up on saying,
you know, this is an unacceptable way and actually an irresponsible way to run a project.
Like, for instance, this is a project that runs mostly on fixed-term contract
researchers, postdocs, with a few PIs.
And we are now in the third phase of the human brain project,
and they've never managed to have continuity of funding between two phases.
That's a problem. And especially between the rampant phase and the SGA one,
we had how many months, six months between the two, and how responsible a way
is that to run a scientific project?
When people don't get the money to pay their postdocs, postdocs go away.
And then, and I saw just the review letter that was finalized last week,
the report letter saying, oh, it's like, basically,
the reviewers and the European Commission are wondering why,
and are sort of alarmed that you haven't managed to spend all your budget in SGA1. It's like, duh.
I mean, so many people couldn't recruit for months.
And then when we got finally the money, we still have to recruit because people have gone.
So when I say, yeah, there are levels at the individual level, but then we are lacking.
So, I mean, I'm part of the guinea pigs who are, you know, sort of trying this
kind of science within full society implementation horizon 2020.
What that implication, that's what I totally mentioned to you,
right? This is an important one. I really like your example.
You say there's no responsible research without responsible governance, right?
And all the stakeholders in the governance process must then be part of the
responsible research process. This is also what you're saying, right?
But in some sense.
This is always an incomplete model because we work in hierarchies and somewhere
the hierarchy is closed and there is no more overview at that level, right?
So that means that is also best an approximation of how you might get to responsible research.
And there is also the issue that there are so many powerful lobbies with a lot
of money to pay for lobbying that are influencing at the commission level and that we are not going,
neither you nor I, going to be able to do.
I mean, you can't, so we are in a very imperfect world.
But we're in a position where, you know, sort of a large amount of money has
been devolved to a group of scientists, essentially,
to manage themselves and manage their research, which is quite a privilege and
in some senses an anticipated situation.
Maybe sort of CERN and things have been in situations. And as Christine said,
the big social experiment.
Yeah, exactly. So, I mean, and it seems to me that we do need to, as Paul said.
Think about different levels of which sort of responsible research can be considered.
So there's the question of what are our values and how do we agree some,
not necessarily consensus, but we at least look at our values and try and see
which ones we care about.
There's the question of what are the risks, and we need to anticipate the risks short and long term.
And then there's the question of how do we have the right governance structures
to ensure that we make decisions that are fitting with those values and address those risks.
Do we have the right mechanisms in a project like the HBP to do all those things?
Are there people who are thinking about values and looking for the right values
to guide what we're doing?
Are there people who are anticipating risks? I think They probably are,
but maybe it doesn't seem to me as an insider, I haven't seen that there's a
framework that summarizes all this. Is there one? Not really.
Boxes that now you start having those kind of there is
a sound operation procedure and kind of pipeline for
everything but it doesn't treat it never says anything about right or wrong
it's if the process has been respected then you must accept the outcome and
so it's it's been god yeah i used to be an engineer so it's kind of garbage in garbage out for me so,
So there is something missing.
You can't just be happy saying if all the procedures are in place and all the
process has been respected, then you have to accept the outcome.
Well, this is a good example, yeah, because ethical values actually become especially
important when your standard procedures don't work, right?
When the standard procedures don't work, it means everything is as expected
and there's apparently little challenges.
But ethical principles come in when you have trade-offs, when you have conflicts, right?
So this is where a whole procedural model can actually never solve that.
Exactly. Different values have to come in, and these values must be defined in some way.
And it's… I mean, what is terribly complicated is that there are so many disciplinary
fields represented in the Human Brain Project that you….
Even the professional ethics of those various fields don't align at all between themselves.
So you talk to the people who come from really the more informatics side,
and say, I've never had to fill an ethics application in my entire career.
It's like, why would I start now?
But the two states here, right? So the one end, your green project is a social
experiment in science funding and or science organization.
But maybe for the discussion of online ethics and research, it's just one example
we can look at. It's one example. There are many others.
And it might not necessarily be the best example, also given the kinds of problems
it has faced, actually, that have a lot to do with ethics from its inception to where it is now.
So maybe it's also worth a while to look a little bit, as do you,
if it's just a test case, a use case we can analyze.
And I think an important issue is, of course, researchers come in all sorts
of flavors, and there will be on one extreme those that are in a project of
that kind because it gives some resource.
And for the rest, and this is what some people report, it's a great project, but it got my money.
On the other extreme, you have people who are in the project because there's
more and more ideological objectives.
But that, of course, also means if you want these groups to reflect collectively,
that the frames in which they look at that process are so radically different,
it might be very difficult to make much progress.
Because if you look at the discussion on free will, where as an individual,
we must be able to make our decision based on a notion of good and evil and
good and bad, there is a notion of reason responsiveness, right?
So in free will, to be a moral decision maker, we must be able to monitor our own decision making.
We must be able to reflect on
those decisions, but we must be able to also explicitly externalize them.
We must be able to point to the reasons behind what we do. Right.
And so in that collective where you have so many different perspectives on one project,
it might be very difficult to come to a reason responsive analysis because the
reasons are very different and maybe more implicit in one case,
very explicit in another case.
So wouldn't that argue that this idea of let's reflect is actually,
in some sense, the best we can do in a very bad situation?
It's the best we can do in the best situation. And moreover,
like you said this morning, the scenario I presented was extremely crude.
But that's one of the few things like, if we don't talk about this kind of practical
questions, we don't even want to interact with you because you are wasting our time.
So, you know, so that's also part of...
We are there and for some people we are there in the Human Brain Project in
the same way that you are filling in an ethics application to get funding.
So we are there, we get a bit of the money, but further than that.
We do the ethics compliance.
Some people among us do the compliance, so to free the police of things.
If we can help with the science communication, which we don't really,
this is not really our work. I mean, I'm not a PR person, but then good.
Otherwise, take as little space as possible and just don't waste our time.
So there is also this kind of ill will that is very hard when there is already
a lot of demand in a project which is so bureaucratized for all sorts of reporting
and deadlines and stuff.
And you come back at things like, oh, let's think about ethics.
Like, I don't have time.
Right. Paul, we should wrap up because Christine has to get a plate.
Maybe we could finish by, the Human Brain Project has been running for five years now.
The commission is about to launch into another round of flagships.
They're going to fund some pilot studies for flagships in the next year.
So what lessons could you summarize from five years of HPP for this next round of flagships?
For me, I would say don't integrate all humanities,
arts, and social science into those big scientific projects because that frames
them as being purely utilitarians and it's not necessarily helpful.
I would also say the way they have been evaluating the large flagships is very subjective.
Because they are social experiments, but never at any point in these five years
has there been any social scientist involved in these projects mandated to actually
monitor these projects as social experiments.
So whatever conclusions are drawn are not drawn based on fieldwork and qualitative
research within the projects.
And because it was all launched from the premise that this is the right model
to address the big challenges.
And like I said, when you are framed within a certain, this is a very constraining frame.
So that's not questionable.
But it's sort of questioned because now they are sort of saying,
we may be using a different, smaller flagship, blah, blah, and so on,
which means they are not entirely satisfied with the way it's happened,
But they haven't mandated any research to be done in the way it has happened.
So I would say a bit more, you know, take all this a bit more seriously and
actually accept that even your own premises are questioned by whatever social
science and humanities you inject into those launch and essays you inject to those last projects.
And yeah that's
and another thing is um
it's all well and nice to create uh
work packages on sub or sub projects in charge of ethics and society to which
all these questions are sort of delegated but if you don't give them teeth to
uh actually go back to the to the to people who are not
necessarily willing to interact and collaborate and just like,
I'm not interested and I don't want to work with you.
I mean, we are powerless, so we end up like being unable to just go back to
the actual problem. Christine, if I want to pursue responsible research,
what should be Christine's law?
I mean for me it's always been and that's my very partial judgment,
I tend to be strictly critical of what I do and look back on it and I mean don't
spare yourself self-reflection is also about really trying to step out and look at what we're doing,
as, you know, do I have other motives than the ones I'm allowing myself to make explicit to myself?
Like, know thyself. We're back in Delphi, right?
Yeah. And the second, Dennis, if you have to make a prediction that Tony will
come and check in London four years from now on the state of the art in response to research,
what specific prediction would you like to see validated in a four-year time frame?
That's about by the end of the human brain. By the end of the human brain project.
So far, what we have seen validated is all the negative stuff.
When we told the medical informatics people at the beginning of the ramp-up
phase that if they went on the way they were going, ignoring consent and saying,
we are all going to anonymize. They were going to faceplant.
And they sort of faceplanted. So that was...
That's a bad prediction.
Yeah, we totally said, which is not exactly satisfying.
It's a prediction now. Prediction about the human brain project.
No, about... Responsible research. Responsible research. Responsible now.
I mean, like, utopian or realistic? Anything you want. Anything I want.
Tony will come and check it four years from now. Four years from now.
Now, I think it will have changed names.
Very good. Okay, Christina Cardes, thank you very much for this conversation. Very good.
The CSN podcast was produced by the Convergent Science Network of Biometrics
and Biohybrid Systems, a project funded by the European Sevens Research Framework Program.
For more interviews, recorded lectures, or upcoming conferences in the field
of biometrics and biohybrid systems, go to csnnetwork.eu.
Music.
And thank you for listening.