A theory of principles for science advisors

posted in: INGSA news 0

Guest blog from Dr Heather Douglas

On Wednesday Sept. 28, science advisors and scholars of science advice met to address the challenges of articulating principles for science advice, principles that would be applicable across institutional contexts and political cultures.  The World Science Forum at Budapest in November 2015 framed the challenge in a “call for concerted action of scientists and policy-makers to define and promulgate universal principles for developing and communicating science to inform and evaluate policy based on responsibility, integrity, independence, and accountability.” (World Science Forum 2015)  As the declaration from the forum noted, “the independence, transparency, visibility and accountability of those who receive and provide advice has never been more important.”  (ibid.)

What is needed is not just an articulation of what these concepts should entail in the practice of science advice, but also how these requisite goals fit together.  Ideally, our understanding of these aspects of science advice should cohere together as much as possible, illuminating each other.

In thinking about this challenge, I have come to see the core principles for science advisors are integrity, responsibility, and accountability.  In defining and describing these three principles, we will then be able to articulate what we should mean by independence (from what?) and transparency (for what to whom?), and ultimately, legitimacy.  Let me first describe the three core principles and then I will describe how they inform the others.

Scientific integrity is most helpfully defined as a proper respect for inquiry.  Having a proper respect for inquiry requires having an open mindedness about the outcomes of inquiry (forestalling commitments to predetermined results), allowing inquiry to proceed to the best of one’s expert judgment given the practical constraints with which one is operating (not directing it to a particular end), and ensuring that social and ethical values play only legitimate roles in the process of inquiry.  Respecting inquiry requires understanding that we should not be sure of the end results of an inquiry until that process is complete.

It would be ideal if all parties in the science advising process exhibited integrity in this way, including both the advisor and the advisee.  Having such a respect for inquiry helps ground shared trust, especially in the face of uncertain scientific outcomes.

Proper roles for social and ethical values include the direction of attention to matters of social and ethical concern, ethical restrictions on use of data and ethical subjects, and the use of values to determine evidential sufficiency, through the weighing of consequences of error.  Values should not supplant or stand in place of evidence.

Science advisors should also be aware of how values influence their reasoning and be clear with their advisees on the nature of that influence.

Accountability and responsibility are distinct but overlapping core principles.  Accountability concerns what science advisors are held accountable to and by whom they are held accountable.  Accountability involves identifiable mechanisms for holding a person to account.  Responsibility, on the other hand, concerns what an advisor is responsible for and whom they are responsible  to, even if there are no accountability mechanisms in place.  The difference between accountability and responsibility parallels the difference between legality and morality.  While there should be substantial overlap between these two domains, they are not equivalent.  One can be responsible for things for which one will not be held accountable, and one can be held accountable for things for which one is not ultimately responsible.  The lack of equivalence is important for proper functioning of institutions and defining roles within them.

Science advisors have a range of responsibilities.  First and foremost, science advisors are responsible for giving scientifically accurate advice. They also have a responsibility to explain their expert judgment regarding the available scientific evidence, to explain in their advice which evidence they found salient, and how it was assessed.  In addition, it may help in some cases to explain accounts of the evidence which were rejected (and why), and evidence that was set aside as irrelevant or inadequate (and why), particularly where there is a public dispute that references such evidence. Science advisors also have a responsibility to understand how values (including important social and ethical values) shape the definition of problems in advice and assessments of the sufficiency of evidence.  Such value-laden judgments are an essential part of science advice (there can be no value-free advice) and how values have played a role in the generation of advice should be made clear to the advisee (whether that is a policy-maker, a stakeholder, or the general public) (as noted in the discussion of integrity).  Finally, science advisors have a responsibility to be responsive to societal concerns where science advice is salient and potentially helpful.

These responsibilities are to complex sets of groups. Science advisors are responsible for  accuracy,  for the explication of value judgments, and  for responsiveness,  to their advisees,  to the expert community (which expects accurate representation of expertise), and to the public.  All three general groups need science advisors to perform these responsibilities to the best of their ability and depend on such performance in the use of science advice.

But these groups cannot all hold science advisors accountable for meeting these responsibilities.  The general public has no accountability levers directly to science advisors.  Instead, others have the capacity to hold science advisors accountable.  For example, it is the expert community (which will shift in composition depending on the topic of the advice) that can and should hold the advisor accountable for the accuracy of their advice, and should also hold science advisors accountable for the explication of value judgments (often only other experts can tell whether a particular judgment given a particular value commitment is reasonable).  Expert communities can and should call each other out for failing to meet these responsibilities.  When called out, the advisee should assess the advisor and decide whether further accountability measures (such as the loss of position) is warranted.

Finally, science advisors are accountable to their advisees for giving clear and understandable advice, and for explaining their expert judgments to their advisee.   They are also accountable to their advisees for being as clear as possible about the values that shape their advice, so that their advisees can properly interpret their advice.

Different modes of science advice will thus have different mechanisms in practice for accountability.  Accountability mechanisms for science advisors who work for particular advisees are more precise and exacting than mechanisms for science advisors who work through independent advisory committees, for example.  Despite the importance of both responsibility and accountability, it should never be the expectation or demand that a science advisor violate their integrity in order to meet a responsibility.  Rather, defending and exemplifying one’s integrity is a central responsibility for science advisors.

With these three, integrity, responsibility, and accountability, articulated, clearer accounts of independence, transparency, and legitimacy are possible.

Independence:  The key question for independence is independence from what?  Not every political influence on science advice is illegitimate—political concerns properly frame advising questions, social values shape evidential sufficiency assessments, and relevance concerns are often political.  Further, it is legitimate for social and ethical values to shape which research is done, and whether a research method or agenda is morally acceptable.  In addition, many science advisors find they are most effective if they are embedded in governance processes officially, rather than independent from them.  Issues of responsibility and accountability show that complete independence would be undesirable—we want responsive, responsible, and accountable science advisory systems.

However, we don’t want illegimate political interference.  Examples of such illegitimate interference includes practices that attempt to get a predetermined outcome from a science advisory process, e.g., through expert selection, problem definition, or dictating or altering results.  Such practices would clearly violate the core integrity principle.  Thus, independence is generally desirable to the extent that it is needed to protect integrity.  Dependencies that generate or encourage violations of integrity are the problem that needs to be avoided.

Transparency:  The central questions for transparency are what science advisors need to be transparent about and to whom.  The discussion of responsibility and accountability described above illuminates these questions, and will depend on the nature of the science advising mechanism.

Minimally, science advisors need to be transparent to their advisees.  Indeed, without such transparency, it is unclear what the use of science advice is.  In some science advising contexts, advice (and how it was generated) should be transparent to 1) the expert community, 2) relevant stakeholders, and/or 3) the general public.  What should be made transparent is minimally the expert judgment made (described above in responsibility).  Additionally, depending on the mechanism, the evidential basis should also be made transparent (or at least available).  Finally, in some advising systems, evidence or arguments found lacking should also be made transparent.

Transparency is crucial for accountability mechanisms, and the key issue is that as much transparency as is needed for accountability should be required.

Legitimacy:  (diversity, representation, accountability, responsibility, integrity, independence, process)

With this layered account of the principles of science advice, we can see where legitimacy comes from.  It comes from the core integrity of science advisors, their responsibility and accountability to their expert community and to their advisees, and is made manifest through the transparency of their work.

Depending on the nature of the science advising mechanism, legitimacy can be further bolstered by careful adherence to process (which demands responsibility, integrity, and transparency) and diversity in the participation in the process.  Diversity can be understood as both diversity of expertise/scientific background and diversity of social background (especially where social values are particularly salient and diverse).  Legitimacy arises from a complex set of factors which cannot be reduced to the evidential basis alone.

Thus, the structure of science advising mechanisms is of paramount importance.  While principles, such as integrity, responsibility, accountability, independence, and transparency can be articulated generally, often their specific instantiations depend on the particular science advising mechanism to which they are applied.  Fleshing out how the principles might work in different mechanisms and institutional contexts is the work of detailed guidelines that follow from principles.

Despite the importance of the institutional and contextual details, we still can have a coherent set of principles which capture concerns and provide insight for a range of science advising challenges. We don’t have to try to balance apparent tensions between accountability and independence, for example.  By carefully articulating what we should mean by these ideas in the practice of science advice, we can see how they work together in making science advice trustworthy.

Dr Heather Douglas is Associate Professor (Waterloo Chair in Science and Society) at the University of Waterloo