ALL ABOUT THE CLIMATE CHANGE STUDIERS

Climate Change Assessments, Review of the Processes & Procedures of the IPCC
August 30, 2010 (InterAcademy Council)

Investigations proved conclusively that last year's tempest in a tea cup surrounding global climate change research was essentially all about nothing more than a few ill-tempered emails and some misplaced statistics. It nevertheless led to the thorough examination of Intergovernmental Panel on Climate Change (IPCC) methods and practices detailed below.

Most of the report is inside baseball but one valuable takeaway for the casual reader is the sheer volume of peer-reviewed information the IPCC digests before coming to its conclusions. It is a body that considers an enormous range of validated scientific research before making pronouncements like, for instance, that climate change is real, progressing and worsened by human spew.

Insights based on such voluminous research - in contrast, say, to rants based on the opinions of one or two pompous contrarians and a gut-level cynicism that science too complicated for the lay person to understand must be dubious - should surely be taken seriously.

Executive Summary

Climate change is a long-term challenge that will require every nation to make decisions about how to respond. The Intergovernmental Panel on Climate Change (IPCC) was established by the World Meteorological Organization and the United Nations Environment Programme to help inform such decisions by producing comprehensive assessments of what is known about the physical climate system, its global and regional impacts, and options for adaptation and mitigation. Sitting at the interface between science and politics, the IPCC assessment process has sustained a working dialog between the world’s governments and scientists since its inception in 1988.

Representatives of 194 participating governments agree on the scope of the assessment, elect the scientific leaders of the assessment, nominate authors, review the results, and approve the summaries written for policy makers. More than a thousand volunteer scientists evaluate the available scientific, technological, and socioeconomic information on climate change, and draft and review the assessment reports. The thousands of scientists and government representatives who work on behalf of the IPCC in this non- traditional partnership are the major strength of the organization.

Through its assessment reports, the IPCC has gained enormous respect and even shared the Nobel Peace Prize in 2007 for informing climate policy and raising public awareness worldwide.

click to enlarge

However, amidst an increasingly intense public debate over the science, impacts, and cost of climate change, the IPCC has come under heightened scrutiny about its impartiality with respect to climate policy and about the accuracy and balance of its reports. In response, the United Nations and the IPCC commissioned the InterAcademy Council to convene a Committee to review the processes and procedures of the IPCC.

The Committee found that the IPCC assessment process has been successful overall. However, the world has changed considerably since the creation of the IPCC, with major advances in climate science, heated controversy on some climate-related issues, and an increased focus of governments on the impacts and potential responses to changing climate. A wide variety of interests have entered the climate discussion, leading to greater overall scrutiny and demands from stakeholders. The IPCC must continue to adapt to these changing conditions in order to continue serving society well in the future. The Committee’s key recommendations for improving IPCC’s assessment process are given below.

click to enlarge

KEY RECOMMENDATIONS

The Committee’s main recommendations relate to governance and management, the review process, characterizing and communicating uncertainty, communications, and transparency in the assessment process. Other detailed recommendations on specific aspects of the assessment process appear in Chapters 2-4, and a complete list of recommendations appears in Chapter 5.

Governance and Management

The complexity and scale of climate change research and the associated assessment task have grown significantly over the last two decades, as have public expectations regarding the assessments. Yet the fundamental management structure of the IPCC has remained largely unchanged. The IPCC management structure comprises the Panel itself, which makes decisions about the structure, principles, procedures, and work program of the IPCC; the Bureau, which is elected by the Panel to oversee the assessment work; and a small Secretariat, which supports the work of the Panel and the Bureau. The Panel makes all of its major decisions at annual Plenary sessions. However, important decisions need to be made more often, and the Bureau has too limited a set of responsibilities and meets too rarely to meet this need.

Many organizations in the public and private sector have addressed the need for ongoing decision making by establishing an Executive Committee to act on their behalf. Similarly, the IPCC should establish an Executive Committee elected by and reporting to the Panel. An IPCC Executive Committee would act on issues—such as approving minor corrections to published reports, approving modest alterations in the scope of an ongoing assessment, ensuring effective communication—and any other task specifically delegated by the Panel. To respond quickly, the Executive Committee should be relatively small with ideally no more than 12 members. Its membership would include selected IPCC leaders as well as individuals from academia, nongovernmental organizations, and/or the private sector who have relevant experience and who are not connected with the IPCC or even climate science. Their participation would improve the credibility and independence of the Executive Committee.

Recommendation: The IPCC should establish an Executive Committee to act on its behalf between Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and 3 independent members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.

The IPCC Secretariat supports the Panel and Bureau by organizing meetings, communicating with governments, supporting the travel of developing-country scientists, managing the IPCC budget and website, and coordinating report publication and outreach. Although the number of staff has grown from 4 to 10 individuals, the growth in the magnitude and intricacy of the assessment task, advances in digital technologies, and new communications needs (see “Communications” below) have changed the mix of skills required of the Secretariat. An Executive Director is needed to lead the Secretariat, ensure that IPCC protocols are followed, keep in touch with the Working Group Co-chairs, and speak on behalf of the IPCC. As a peer of the Working Group Co-chairs, the individual selected as Executive Director would be capable of acting on behalf of the IPCC Chair. The Executive Director would also be a member of the Executive Committee.

click to enlarge

Recommendation: The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day operations of the organization. The term of this senior scientist should be limited to the timeframe of one assessment.

Review Process

Peer review is an important mechanism for assuring the quality of reports. IPCC’s peer review process is elaborate, involving two formal reviews and one or more informal reviews of preliminary text. The first complete draft is formally reviewed by scientific experts nominated by government representatives, observer organizations, and the IPCC Bureau. Lead Authors consider the review comments and prepare the second draft, which is reviewed by the same experts as well as government representatives. Two or more Review Editors for each chapter oversee the review process, ensuring that review comments and controversial issues are handled appropriately. However, the Lead Authors have the final say on the content of their chapter.

With the tight schedule for the revision process, authors do not always consider the review comments carefully, potentially overlooking errors in the draft report that might have been caught. Some errors will be missed in any review process; but with stronger enforcement of existing IPCC review procedures, the number of errors could be minimized. Staff support and clarification about the roles and responsibilities of Review Editors would help them carry out proper oversight.

Recommendation: The IPCC should encourage Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report.

For recent assessments, some governments made the second draft available for review by national experts and other interested parties, considerably opening the review process.

Although an open review potentially improves the report by increasing the level of scrutiny and widening the range of viewpoints offered, it also substantially increases the number of review comments.

Drafts of the Fourth Assessment Report drew 90,000 review comments (an average of a few thousand comments per chapter), stretching the ability of Lead Authors to respond thoughtfully and fully. A more targeted process for responding to reviewer comments could both ensure that the most significant review issues are addressed and reduce the burden on authors, who currently must document responses to all reviewer comments. In the targeted process envisioned, the Review Editors would prepare a written summary of the most significant review issues. While the Lead Authors would prepare written responses to these issues and all other non-editorial comments, they could focus their attention on the most significant matters.

click to enlarge

Recommendation: The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received. Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and no written responses to editorial comments.

Characterizing and Communicating Uncertainty

Uncertainty is characterized and communicated by describing how much is known about a topic (i.e., the quality and nature of the evidence available) and the probability that a particular event will occur. Each key conclusion in the Summary for Policy Makers is accompanied by a judgment about its uncertainty. For the fourth assessment, each Working Group used a different variation on IPCC’s guidance to describe uncertainty. Working Group I relied primarily on a quantitative likelihood scale (e.g., “extremely likely” indicates a greater than 95 percent probability that a particular event will occur). Working Group II relied primarily on a quantitative confidence scale (e.g., “high confidence” indicates an 8 out of 10 chance of being correct). Working Group III relied exclusively on a qualitative level-of-understanding scale (i.e., understanding is described in terms of the amount of evidence available and the degree of agreement among experts). The level-of-understanding scale is a convenient way of communicating the nature, number, and quality of studies on a particular topic, as well as the level of agreement among studies. It should be used by all Working Groups, as suggested in the IPCC uncertainty guidance for the Fourth Assessment Report.

Recommendation: All Working Groups should use the qualitative level-of-understanding scale in their Summary for Policy Makers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.

The Working Group II Summary for Policy Makers has been criticized for various errors and for emphasizing the negative impacts of climate change. These problems derive partly from a failure to adhere to IPCC’s uncertainty guidance for the fourth assessment and partly from shortcomings in the guidance itself. Authors were urged to consider the amount of evidence and level of agreement about all conclusions and to apply subjective probabilities of confidence to conclusions when there was high agreement and much evidence. However, authors reported high confidence in some statements for which there is little evidence. Furthermore, by making vague statements that were difficult to refute, authors were able to attach “high confidence” to the statements. The Working Group II Summary for Policy Makers contains many such statements that are not supported sufficiently in the literature, not put into perspective, or not expressed clearly. When statements are well defined and supported by evidence—by indicating when and under what climate conditions they would occur—the likelihood scale should be used.

click to enlarge

Recommendation: Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of well-defined outcomes only when there is sufficient evidence.
Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).

Communications

Communicating the results of IPCC assessments is challenging because of the range and complexity of climate science and response options and the increasing need to speak to audiences beyond scientists and governments. The communications challenge has taken on new urgency in the wake of recent criticisms regarding IPCC’s slow and inadequate responses to reports of errors in the Fourth Assessment Report. Such criticisms underscore the need for a media-relations capacity to enable the IPCC to respond rapidly and with an appropriate tone to the criticisms and concerns that inevitably arise in such a contested arena. In addition, IPCC leaders have been criticized for making public statements that were perceived as advocating specific climate policies. Straying into advocacy can only hurt IPCC’s credibility. A comprehensive communications strategy is needed to identify who should speak on IPCC’s behalf and to lay out guidelines for keeping messages within the bounds of IPCC reports and mandates. IPCC’s new communications and media relations manager is developing a communications strategy, and the Committee urges its rapid completion.
Recommendation: The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and which includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.

Transparency

Given the high stakes in the climate change debate and IPCC’s role of providing policy-relevant information, the IPCC can expect that its reports will continue to be scrutinized closely. Thus, it is essential that the processes and procedures used to produce assessment reports be as transparent as possible. From extensive oral and written input gathered by the Committee, it is clear that several stages of the assessment process are poorly understood, even to many scientists and government representatives who participate in the process. Most important are the absence of criteria for selecting key participants in the assessment process and the lack of documentation for selecting what scientific and technical information is assessed. The Committee recommends that the IPCC establish criteria for selecting participants for the scoping meeting, where preliminary decisions about the scope and outline of the assessment reports are made; for selecting the IPCC Chair, the Working Group co-chairs, and other members of the Bureau; and for selecting the authors of the assessment reports. The Committee also recommends that Lead Authors document that they have considered the full range of thoughtful views, even if these views do not appear in the assessment report.

If adopted in their entirety, the measures recommended in this report would fundamentally reform IPCC’s management structure while enhancing its ability to conduct an authoritative assessment. However, no matter how well constructed IPCC’s assessment practices may be, the quality of the result depends on the quality of the leaders at all levels who guide the assessment process. It is only by engaging the energy and expertise of a large cadre of distinguished scholars as well as the thoughtful participation of government representatives that high standards are maintained and that truly authoritative assessments continue to be produced. Moreover, the IPCC should think more creatively about maintaining flexibility in the character and structure of the assessment, including the number and scope of Working Groups and the timing of reports. For example, releasing the assessment of regional impacts substantially after the assessment of sectoral impacts would reduce the burden on the small community that carries out both assessments. It may also be desirable to release the Working Group I report a year or two ahead of the other Working Group reports. Although such issues are routinely raised and settled in the scoping process, the traditional approach may not be the best model for future assessments.

click to enlarge

Conclusions

IPCC’S PROCESSES AND PROCEDURES

The Committee concludes that the IPCC assessment process has been successful overall and has served society well. The commitment of many thousands of the world’s leading scientists and other experts to the assessment process and to the communication of the nature of our understanding of the changing climate, its impacts, and possible adaptation and mitigation strategies is a considerable achievement in its own right. Similarly, the sustained commitment of governments to the process and their buy-in to the results is a mark of a successful assessment.

Through its unique partnership between scientists and governments, the IPCC has heightened public awareness of climate change, raised the level of scientific debate, and influenced the science agendas of many nations. However, despite these successes, some fundamental changes to the process and the management structure are essential, as discussed in this report and summarized below.

Summary of Recommendations

Modernizing the management structure. Since its inception more than two decades ago, the governance and basic elements of the management structure of the IPCC have changed very little. Meanwhile, the magnitude and complexity of the assessment task has increased and new demands are being made for increased transparency and accountability. Best practices in other organizations provide a model for the IPCC to renew its governance and management structure.

Key elements of this structure include the establishment of an Executive Committee to act on behalf of the Panel between Plenary sessions, the appointment of a senior scientist as Executive Director to lead the Secretariat, and the institution of conflict of interest policies for major players in the IPCC assessment process. Moreover, the architecture of the Secretariat should be reevaluated to ensure that its responsibilities can be carried out effectively. As part of this reevaluation, the roles and responsibilities of key participants, including the IPCC Chair, should be clearly defined. A limit of one term for key IPCC leaders, including the IPCC Chair, Working Group Co-chairs, and the proposed Executive Director, would ensure a greater infusion of fresh perspectives on the assessments.

click to enlarge

Recommendations:

 The IPCC should establish an Executive Committee to act on its behalf between Plenary sessions. The membership of the Committee should include the IPCC Chair, the Working Group Co-chairs, the senior member of the Secretariat, and 3 independent members, including some from outside of the climate community. Members would be elected by the Plenary and serve until their successors are in place.

 The IPCC should elect an Executive Director to lead the Secretariat and handle day-to-day operations of the organization. The term of this senior scientist should be limited to the timeframe of one assessment.

 The IPCC should redefine the responsibilities of key Secretariat positions both to improve efficiency and to allow for any future senior appointments.

 The IPCC should develop and adopt a rigorous conflict of interest policy that applies to all individuals directly involved in the preparation of IPCC reports, including senior IPCC leadership (IPCC Chair and Vice Chairs), authors with responsibilities for report content (i.e., Working Group Co-chairs, Coordinating Lead Authors, and Lead Authors), Review Editors, and technical staff directly involved in report preparation (e.g., staff of Technical Support Units and the IPCC Secretariat).

 The term of the IPCC Chair should be limited to the timeframe of one assessment.

 The terms of the Working Group Co-chairs should be limited to the timeframe of one assessment.

click to enlarge

Strengthening the review process. The review process is a fundamental step for ensuring the quality of assessment reports. The Committee found that some existing IPCC review procedures are not always followed and that others are weak. In particular, Review Editors do not fully use their authority to ensure that review comments receive appropriate consideration by Lead Authors and that controversial issues are reflected adequately in the report. Staff support and/or clarification of the roles and responsibilities of Review Editors could help them provide the proper oversight. In addition, the large number of review comments may distract Lead Authors from fully addressing the most important issues. Having Review Editors identify the key issues that must be addressed would ensure that these issues receive due consideration. Allowing Lead Authors to document only their responses to noneditorial comments would reduce their administrative burden.

Recommendations:

 The IPCC should encourage Review Editors to fully exercise their authority to ensure that reviewers’ comments are adequately considered by the authors and that genuine controversies are adequately reflected in the report.

 The IPCC should adopt a more targeted and effective process for responding to reviewer comments. In such a process, Review Editors would prepare a written summary of the most significant issues raised by reviewers shortly after review comments have been received.

Authors would be required to provide detailed written responses to the most significant review issues identified by the Review Editors, abbreviated responses to all non-editorial comments, and no written responses to editorial comments.

Characterizing and communicating uncertainties.

IPCC’s guidance for addressing uncertainties in the Fourth Assessment Report urge authors to consider the amount of evidence and level of agreement about all conclusions and to apply subjective probabilities of confidence to conclusions when there was “high agreement, much evidence.” However, such guidance was not always followed, as exemplified by the many statements in the Working Group II Summary for Policy Makers that are assigned high confidence, but are based on little evidence. Moreover, the apparent need to include statements of “high confidence” (i.e., an 8 out of 10 chance of being correct) in the Summary for Policy Makers led authors to make many vaguely defined statements that are difficult to refute, making them therefore of “high confidence.” Such statements have little value. Scientific uncertainty is best communicated by indicating the nature, amount, and quality of studies on a particular topic, as well as the level of agreement among studies. The IPCC level-of-understanding scale provides a useful means of communicating this information.

Recommendations:

 All Working Groups should use the qualitative level-of-understanding scale in their Summary for Policy Makers and Technical Summary, as suggested in IPCC’s uncertainty guidance for the Fourth Assessment Report. This scale may be supplemented by a quantitative probability scale, if appropriate.

 The confidence scale should not be used to assign subjective probabilities to ill-defined outcomes.

 Quantitative probabilities (as in the likelihood scale) should be used to describe the probability of well-defined outcomes only when there is sufficient evidence. Authors should indicate the basis for assigning a probability to an outcome or event (e.g., based on measurement, expert judgment, and/or model runs).

 The likelihood scale should be stated in terms of probabilities (numbers) in addition to words to improve understanding of uncertainty.

 Chapter Lead Authors should provide a traceable account of how they arrived at their ratings for level of scientific understanding and likelihood that an outcome will occur.

 Where practical, formal expert elicitation procedures should be used to obtain subjective probabilities for key results.

Developing an effective communications strategy. In the wake of errors discovered in the Fourth Assessment Report, the IPCC has come under severe criticism for the manner in which it has communicated with the media. The lack of an ongoing media-relations capacity and comprehensive communications strategy has unnecessarily placed the IPCC’s reputation at risk and contributed to a decline in public trust of climate science.

Recommendation: The IPCC should complete and implement a communications strategy that emphasizes transparency, rapid and thoughtful responses, and relevance to stakeholders, and which includes guidelines about who can speak on behalf of IPCC and how to represent the organization appropriately.

Increasing transparency. Transparency is an important principle for promoting trust by the public, the scientific community, and governments. Interviews and responses to the Committee’s questionnaire revealed a lack of transparency in several stages of the IPCC assessment process, including scoping and the selection of authors and reviewers, as well as in the selection of scientific and technical information considered in the chapters.

click to enlarge

Recommendations:

 The IPCC should make the process and criteria for selecting participants for scooping meetings more transparent.

 The IPCC should develop and adopt formal qualifications and formally articulate the roles and responsibilities for all Bureau members, including the IPCC Chair, to ensure that they have both the highest scholarly qualifications and proven leadership skills.

 The IPCC should establish a formal set of criteria and processes for selecting Coordinating Lead Authors and Lead Authors.

 Lead Authors should explicitly document that a range of scientific viewpoints has been considered, and Coordinating Lead Authors and Review Editors should satisfy themselves that due consideration was given to properly documented alternative views.

Clarifying the use of unpublished and non-peer-reviewed sources. A significant amount of information that is relevant and appropriate for inclusion in IPCC assessments appears in the so called gray literature, which includes technical reports, conference proceedings, statistics, observational data sets, and model output. IPCC procedures require authors to critically evaluate such sources and to flag the unpublished sources that are used. However, authors do not always follow these procedures, in part because the procedures are vague.

Recommendation:

The IPCC should strengthen and enforce its procedure for the use of unpublished and non-peer-reviewed literature, including providing more specific guidance on how to evaluate such information, adding guidelines on what types of literature are unacceptable, and ensuring that unpublished and non-peer-reviewed literature is appropriately flagged in the report.

Engaging the best regional experts. The author team for each regional chapter in the Working Group II report is drawn largely from experts who live in the region. Yet some of the world’s foremost experts on a particular region live outside the region. This geographic restriction sometimes limits the expertise that may be drawn upon for the regional assessments.

Recommendation:

The IPCC should make every effort to engage local experts on the author teams of the regional chapters of the Working Group II report, but should also engage experts from countries outside of the region when they can provide an essential contribution to the assessment.

Expediting approval of the Summary for Policy Makers. The final language of the Summary for Policy Makers is negotiated, line-by-line, between scientists and government representatives in a grueling Plenary session that lasts several days, usually culminating in an all-night meeting. Both scientists and government representatives who responded to the Committee’s questionnaire suggested changes to reduce opportunities for political interference with the scientific results and to improve the efficiency of the approval process.

click to enlarge

Recommendation:

The IPCC should revise its process for the approval of the Summary for Policy Makers so that governments provide written comments prior to the Plenary.

Reducing the growing burden on the scientific community. A successful assessment achieves an appropriate balance between the benefits of the results and the opportunity costs to the scientific community, such as diverting resources from ongoing research projects.

Analysis of Global Change Assessments (NRC, 2007) found that high opportunity costs are a weakness of IPCC assessments. The Committee agrees, noting that each successive IPCC assessment has required greater amounts of human resources to assess the growing literature and to respond to the increasing number of review comments (e.g., see Figure 1.1). Without changes to the assessment process, the time may come when scientists reach the limit of their ability to produce a comprehensive assessment every five or six years. Scientists who responded to the Committee’s questionnaire had a number of ideas for reducing opportunity costs. Among the most common was making the assessment reports shorter and less comprehensive by focusing on key issues or examining only significant new developments (see also Agrawala, 1998b; Karoly et with them. Posting supplementary information on the IPCC website (see further discussion below) could encourage authors to write less and to stay within their page limits. Increasing the efficiency of the review process, as discussed above, would also reduce the burden on scientists.

Maintaining flexibility. To its credit, the IPCC has shown that it is an adaptive organization, applying lessons learned from one assessment to the next and improving its processes to address new policy needs. For example, the IPCC adjusted the scope of Working Groups II and III after the first and second assessments (IPCC, 1992; Watson, 1997); substantially revised its principles and procedures after the second assessment (IPCC, 1998; 1999); and introduced a revised set of scenarios of socio-economic, climate and environmental conditions after the fourth assessment (IPCC, 2008). The Committee urges the IPCC to use the recommendations of this report to continue to adapt its process and structures to accommodate future advances in scientific understanding and evolving needs of policy makers.

Implementation

At the request of the UN Secretary General and the IPCC Chair, this report was completed in time for discussion at the 32nd session of the IPCC Plenary. Most of the Committee’s recommendations can be implemented during the fifth assessment process and should be considered at the upcoming Plenary. These include recommendations to strengthen, modify, or enforce IPCC procedures, including the treatment of gray literature, the full range of views, uncertainty, and the review process. Recommendations that may require discussion at several Plenary sessions, but that could be implemented in the course of the fifth assessment, include those related to management, communications, and conflict of interest. Because the fifth assessment is already underway, it may be too late to establish a more transparent scooping process and criteria for selecting authors.

click to enlarge

ISSUES FOR FUTURE CONSIDERATION

In the course of this review, a number of issues arose that are not central to the assessment processes and procedures covered by the Committee’s task, but that affect the nature and quality of the assessment reports. These include the participation of developing countries and the private sector, access to data, the mandate of the Working Groups, and the timing of release of the assessment reports. Although the Committee came to no firm conclusions on these issues, they are raised here for consideration by the IPCC.

Participation of Developing Countries and the Private Sector

The level of participation of developing countries in the IPCC assessment process has been a concern since it was raised by Bert Bolin, the first IPCC Chair, in the early 1990s (Hulme and Mahony, 2010). Indeed, developing country participation has featured on the agenda of virtually every IPCC Bureau meeting from 1989 to 1996 (Agrawala, 1998b).

Full participation by developing countries is necessary to build worldwide trust, confidence, and ownership in the process; to help sustain a global community of climate scientists; to create broad-based political buy-in for the results; and to ensure that the assessment is framed in a way that accounts for the interests of all members and takes the fullest advantage of regional expertise (e.g., Lahsen, 2004).

Although capacity building is not in its mandate, the IPCC has made significant progress in increasing the participation of developing-country governments over the past two decades. In the first assessment, developing countries or countries with economies in transition accounted for 58 percent of the Panel membership; by the fourth assessment, the fraction had grown to 69 percent.

However, although their numbers have increased, their contribution to all stages of the IPCC assessment process remains relatively low. Similarly, some progress has been made in increasing the number of scientists from developing countries that participate in the IPCC assessment process. Nevertheless, more than three-quarters of authors still come from developed countries.

The goal of having proportional representation by developing countries, both at the government level and among scientists, is not disputed either by the IPCC or the Committee. But clearly there is still some way to go if the increased number of developing country participants is not to be construed by some as geographic window-dressing rather than meaningful participation. A number of individuals who were interviewed or responded to the Committee’s questionnaire observed that developing-country scientists often had limited understanding of developments outside of their region and/or did not do (or were not asked to do) their fair share of the work.

Most attributed this lack of participation to the unique difficulties faced by developing-country scientists. These include the exclusive use of English to communicate during the preparation of the Working Group reports, the lack of support by their home institutions, poor access to literature, and the relatively small number of qualified scientists from some developing countries (e.g., see Liverman, 2008).

click to enlarge

Overcoming these barriers will require an investment in scientific infrastructure by more nations, sustained investment in human capital in developing countries, and time. The recently launched IPCC fellowship program for vulnerable and developing countries, established with Nobel Prize funds, is a good example of how IPCC can play a direct role in developing the capacity of its climate research base. However, there are limits to what the IPCC can do to meet the capacity building needs of developing countries. The IPCC can play an indirect role by encouraging international development organizations (e.g., the World Bank, Gates Foundation, the UK’s Department for International Development), governments, and private companies to do what they can to build up human resources while carrying out their own objectives. The IPCC framework—in which scientists learn from their peers in other parts of the world while adding their own regional expertise and perspective—could be used as a model for training scientists from developing countries.

Development agencies and banks and other interested institutions could also help expand the scientific capacity available to the IPCC in other ways, including:

 Facilitating travel of developing-country scientists by funding mobility grants to and/or secondments (temporary placements) of developing-country Lead Authors to enable them to spend time in Technical Support Units or other appropriate institutions in developed countries to facilitate interaction, cooperation, and further human capital development;

 Establishing university-to-university partnerships to strengthen developing-country science;

and

 Establishing regional facilities in developing countries where authors from the region could spend time interacting and writing.

Private companies often investigate important issues related to climate change, particularly in the areas of adaptation and mitigation. Many companies are beginning to see climate change as an opportunity, rather than a threat.20 Their research and support of the process could significantly expand the available knowledge base concerning adaptation and mitigation options as well as the pool of well-qualified authors and reviewers. More fully entraining private companies into the assessment process increases the possibility of financial conflicts of interest, underscoring the importance of an IPCC conflict-of-interest policy.

Access to Information

Data are the bedrock on which the progress of science rests. The extraordinary development of new measuring techniques and new digital technologies has enabled climate scientists to assemble vast quantities of data. However, the large size and complex nature of these databases can make them difficult to access and use. Moreover, for various reasons many of these scientific databases as well as significant unpublished and non-peer-reviewed literature are not in the public domain. An unwillingness to share data with critics and enquirers and poor procedures to respond to freedom-of-information requests were the main problems uncovered in some of the controversies surrounding the IPCC (Muir Russell et al., 2010; PBL, 2010). Poor access to data inhibits users’ ability to check the quality of the data used and to verify the conclusions drawn.

Consequently, it is important for the IPCC to aspire toward ensuring that the main conclusions in its assessment reports are underpinned by appropriately referenced peer-reviewed sources or, to the greatest extent practical, by openly accessible databases. The Technical Support Units could play a key role in helping the IPCC work toward this goal.

In the future, the IPCC may want to consider implementing available technologies to improve its operational efficiency. Commercial databases and systems, for example, are available for managing nominations, citations, and drafts and revisions. Some emerging approaches also merit interest. In particular, the notion of a Wiki-style process was raised in presentations to the Committee21 and in responses to the questionnaire. A Wiki-style process is an electronic, web based system in which the available literature on climate change can be uploaded, critically reviewed, and synthesized with previous information in near real time. Some respondents have suggested testing the concept on a small scale, such as using Wiki pages to supplement the Working Group reports with information that is substantially more detailed than allowed by page limits and that is also more up-to-date. Others are working to develop the concept more fully.

12

Working Group Structure and Phasing of Reports

Although the IPCC adjusted the scope of Working Groups II and III after the first and second assessments (IPCC, 1992; Watson, 1997), the basic Working Group structure has remained consistent through all four assessments, despite some suggestions for change.

For example, Hulme suggested dividing the assessment into global science, regional evaluation, and policy analysis (Hulme et al., 2010). A number of respondents to the Committee’s questionnaire also offered suggestions for change, especially to Working Groups II and III, ranging from expanding their scope, to combining them, to eliminating them completely.

A key part of IPCC’s scoping process is the reevaluation of the scope and mandate of the Working Groups, based on lessons learned from the previous assessment and future needs. In the next scoping process, the IPCC is encouraged to explore structural options that may help address the increasingly multidisciplinary nature of the science, without being constrained by historical precedent.

click to enlarge

Regardless of the future structure of the Working Groups, it is likely that issues that cut across the Working Group mandates will arise throughout the assessment process, especially during the writing and reviewing of reports. Possible ways for fostering interactions among the Working Groups include designing key cross-Working Group issues into the scoping process, holding joint Working Group meetings as appropriate, and appointing reviewers from author teams in other Working Groups. Strengthening coordination across Working Groups where appropriate and productive would not only increase opportunities for knowledge transfer and synergy but would also provide a framework for integrating the various pieces of Working Group reports into the Synthesis Report.

Another issue that merits consideration by the Panel and the Bureau is the phasing of the Working Group reports. It is not clear to the Committee whether issuing all four reports of the assessment within one year is the most effective and efficient means of providing this information to policy makers. Advantages include ensuring that no Working Group report is outdated by the time the Synthesis Report is written. However, there are also disadvantages, particularly for Working Group II. In many regions there is a relatively small knowledge base in climate science and its impacts and also a relatively small cohort of available scholars. The Panel should consider whether the regional assessment should be released significantly later than the sector assessment in order to devote as many high-quality resources as possible to these important issues. In addition, it may be desirable to release the Working Group I report a few years ahead so the other Working Groups can take advantage of the results.

Given the short amount of time available for this review, the Committee could not address every issue of importance to the IPCC assessment process. Nevertheless, it is the hope of the Committee that this report will contribute to an ongoing dialog among IPCC stakeholders on a matter of importance to all humankind and that, as the IPCC embarks on its fifth assessment, the recommendations will encourage greater adherence to current procedures and strengthen IPCC’s assessment process and management structure.