Climate Change Assessments:
Review of the Processes and Procedures of the IPCC

InterAcademy Council

U.N. Press Conference
Aug. 30, 2010

Opening Statement by
Harold T. Shapiro
President Emeritus and Professor of Economics and Public Affairs, Princeton University and Chair, InterAcademy Council Committee to Review the IPCC

Good morning. I would like to begin by thanking Dr. Robbert Dijkgraaf and Dr. Lu Yongxiang, co-chairs of the InterAcademy Council, for their leadership in organizing this review and arranging such a talented and dedicated committee that it has been my pleasure to lead. I am especially grateful to the very distinguished members of this committee, as well as our extraordinarily talented staff, for sacrificing a good part of their summer schedules not only to travel quite far for our meetings but also to complete this report under such a tight deadline.

As its name indicates, the Intergovernmental Panel on Climate Change, or more succinctly the IPCC, sits at the intersection of science and policy, and in many ways it represents a significant social innovation. Its massive, far-flung, and decentralized network of thousands of dedicated scientists who remain so committed to the assessment process is a significant achievement in its own right, as is the buy-in that it has received from the almost 200 government representatives who participate. Over the course of about every five or six years, this group of scientists, split into three Working Groups, produces an exhaustive assessment of what is known about the physical scientific aspects of climate change, its global and regional impacts, and options for adaptation and mitigation. The government representatives perform key functions such as setting the scope of each assessment, electing the Chair of IPCC and the Bureau that oversees the assessment process, and approving the language in the Summaries for Policy Makers that accompany the reports of each of the three Working Groups.

Overall, IPCC’s assessment process has been a success and served society well. The assessments have put IPCC on the world stage, raised public awareness of climate change, and driven policymakers to consider options for responding to climate change. Indeed, these were among the reasons IPCC was awarded a share of the Nobel Peace Prize in 2007.

All of this has been accompanied, however, by an increasingly intense debate in the media and political arena about climate change science and the costs of proposed climate policies, which in turn has meant much greater public scrutiny of IPCC. Controversies have erupted over the perceived impartiality of IPCC toward climate policy, and all of you are aware of the attention given earlier this year in the press to the revelation of errors in the last assessment. Meanwhile, the assessments themselves have grown in size and complexity. Most of our key recommendations are aimed at helping IPCC manage this increasingly complex process and doing so under the gaze of a public microscope.

To gain a fuller understanding of the complex IPCC assessment process and the challenges IPCC faces as an organization, we not only solicited material from the IPCC itself, but also from experts from many countries whom we interviewed who had participated in the IPCC process or who had publicly expressed criticisms of the process or who had a significant interest in the assessments themselves. We also distributed a questionnaire to hundreds of experts and other stakeholders with different views of the IPCC, and placed that questionnaire on the web so that the general public could comment. In addition, we held two public meetings in Amsterdam at The Royal Netherlands Academy of Arts and Sciences, and in Montreal at McGill University where we heard from IPCC and U.N. officials as well as scientists with varying views of the IPCC process. The committee drafted its report at a final meeting in Great Britain at the Kavli Royal Society International Center in Chichley. Subsets of the committee also held meetings in Brazil, China, and the United States in an effort to obtain as diverse a set of views as we could.

Before I get to our key recommendations, I would like to note what our charge was and how we approached our task. We were asked by the Secretary-General and the IPCC Chair to focus on the processes and procedures of the IPCC, and to make recommendations aimed at strengthening the process by which future climate assessments are managed and prepared. We were not asked to re-referee the last IPCC assessment or to review the basic science of human-caused climate change.

We recommend some significant reforms to the management structure that we believe will strengthen the day-to-day management and decision-making capability while at the same time maintaining oversight, transparency, and a variety of perspectives. There are a number of recommendations in the report so I urge you to read it in its entirety, but let me highlight a few of them for you here this morning:

First, we recommend that the Panel should elect a small executive committee to act on its behalf in order to maintain a year-round, ongoing decision-making capability. This committee should include members from outside IPCC or even outside the climate science community in order to give it more credibility and independence.

In addition, an executive director with the status of a senior scientist equal to that of the Working Group co-chairs, and with more autonomy and responsibility than the position of the current secretary, should be appointed to lead the Secretariat. A full-time chief executive serving alongside a part-time chair is a model of leadership employed at other organizations that we think would work well at the IPCC. Because the executive director would report to the chair and speak on behalf of IPCC, it should be someone in whom the chair can place high confidence.

To bring a fresh approach to each assessment process, we also recommend that the IPCC chair, the proposed executive director, and Working Group co-chairs all be limited to the term of one assessment. This recommendation is not in any way motivated by an evaluation of the current leadership of IPCC. Such an evaluation was beyond our charge. Rather, this recommendation has to do, as I said, with ensuring a variety or perspectives at the top as well as maintaining the overall vitality of the assessment process.

Another of our recommendations is that a rigorous conflict-of-interest policy be developed and applied to senior IPCC leadership and all authors, review editors, and staff responsible for report content. It was beyond our charge to review the conflict-of-interest controversies that have been reported in the press, but we did note that the lack of a conflict-of-interest policy was troubling to many of the stakeholders we heard from, and that many government and nongovernment institutions that conduct assessments or provide scientific advice have adopted such policies to assure the integrity of, and public confidence in, their results.

As you know, the request for our review was prompted in part by the few errors discovered in the last assessment, and thus part of our charge was to look at the review process and guidelines for assuring the quality of data used. We concluded that the review process is thorough but that stronger enforcement of existing IPCC review procedures could minimize the number of errors. This can be accomplished in part by encouraging review editors to fully exercise their authority to ensure that all review comments are adequately considered. At the same time, we made some recommendations for streamlining the review process to somewhat ease the burden on authors given the sheer number of reviews – there were 90,000 review comments on the last draft assessment, for example.

The use of non-peer-reviewed sources, which is sometimes called gray literature, also has been controversial and blamed for some of the errors. However, we found that such material, which can include technical reports, conference proceedings, observational data or model results, often is relevant and appropriate for inclusion in the assessment reports. IPCC has guidelines for the use of such sources, but these guidelines are vague and have not always been followed. We recommend that these guidelines be made more specific – including noting what types of sources are unacceptable -- and that they be more strictly enforced to ensure that unpublished and non-peer-reviewed literature is adequately evaluated and appropriately flagged in the reports.

We were also asked to examine how the full range of scientific views is handled. Although all reasonable points of view need to be considered in the assessment process, they do not all need to be given equal weight and what to include is at the end of the day a matter of professional judgment. At the same time, authors need to be careful of what is known as confirmation bias, where too much weight is placed on their own views relevant to others. Our report recommends that lead authors document that the full range of thoughtful scientific viewpoints has been considered, and that coordinating lead authors and review editors satisfy themselves that due consideration was given to alternative views that were properly documented.

The characterization of uncertainty also is an important issue. Again, IPCC’s guidelines in this area have not been consistently followed. The Working Group II report, for example, contains many statements that were assigned high confidence but for which there is little evidence. In future assessments, all Working Groups should characterize uncertainty in a consistent manner, qualifying their level of understanding of a topic based on the amount of evidence and level of agreement among experts, and quantifying the likelihood of an event occurring by using a probability scale, but only when there is sufficient evidence to do so.

There was general agreement among people we talked to and who responded to our questionnaire, as well as an acknowledgement by the IPCC itself, that its public response to the errors earlier this year was slow and inadequate. We understand a new communications officer is developing a communications strategy, which we urge be completed and implemented as soon as possible. This strategy should include a plan for rapid but thoughtful responses to crises and guidelines on how to speak on IPCC’s behalf while staying within the bounds of IPCC reports. The strategy also needs to emphasize transparency.

In fact, given the intense public scrutiny, we believe that a commitment to transparency across IPCC is absolutely necessary. The assessment process is difficult to understand, often even for those involved in it, and we feel that IPCC needs to be as transparent as possible if confidence in the process is to be maintained.

I have two concluding observations. First he Committee believes that most of its recommendations can be implemented during the fifth assessment and should, therefore, be seriously considered during the upcoming Plenary in Korea.

Second, although we believe our recommendations will strengthen the assessment process, we also understand that in the end, the quality of the results depends on the quality of the leaders at all levels who guide the assessment process. It is only by continuing to engage the energy and expertise of a large cadre of distinguished scholars, as well as the thoughtful participation of government representatives, that high standards are maintained and that truly authoritative assessments continue to be produced. IPCC deserves credit for adapting to lessons learned in past assessments, and we hope that it will remain flexible and creative in moving forward. Thank you. We will be glad to take your questions now.