The Aid Transparency Index assesses organisations’ overall commitment to transparency as well as the information they publish at the organisation level and for individual activities. The 2013 ATI represented an evolution in the index methodology, recognising changes in the global environment since the fourth High Level Forum and the progress donors had made in increasing aid transparency, both in terms of commitment and publication. The same methodology has been used in 2014. This section provides an overview of the methodology, including organisation selection, the indicators used and the scoring approach.

The 2014 ATI assesses 68 organisations, including the 67 assessed in 2013 and one new organisation – the Croatian Ministry of Foreign and European Affairs (Croatia joined the EU in 2013 and is therefore covered by the EU-wide commitment on aid transparency). Organisations are selected using three criteria, of which they have to meet a minimum of two:

1. They are a large donor (annual spend is more than USD 1bn);
2. They have a significant role and influence as a major aid agency and engagement with the Busan agenda;
3. They are an institution to which government or organisation-wide transparency commitments apply, for example members of the G7 or the Member States of the European Union.

The ATI uses 39 indicators, divided into those that measure commitment to aid transparency (three indicators) and those that measure publication of aid information (36 indicators). The commitment indicators collectively account for 10% of the overall weight while publication accounts for 90%. The publication indicators are further assembled into organisation level and activity level, which are then divided in sub-groups, based largely upon the groups used in the implementation schedule template for the Busan common standard.

The organisation-level indicators account for 25% of the overall weight, while the activity-level indicators account for 65%. Within these categories, the indicator sub-groups are equally weighted. As the number of indicators in each sub-group varies, individual indicators carry different weights. See full list of indicators and how they are grouped and weighted. The re-weighting tool allows user to test different weighting options and see the effect on the scores.

The score for each indicator ranges from 0–100. For 22 indicators, the scoring takes into account the format that the data is provided in, depending on the accessibility and comparability of the information and how consistently it is published (see chart 2 below). For example, information published in PDFs scores lower than information published in machine-readable formats. Information published to the IATI Standard, the most comparable format, can score up to 100 for each indicator, depending on the coverage of information and frequency of publication.

Data collection ran from 1 April–30 June 2014. Most information included in the ATI is gathered from what each organisation publishes – either on its website, the IATI Registry, national platforms such as the U.S. Foreign Assistance Dashboard or the OECD common standard website (for implementation schedules). A secondary data source is used to assess one indicator – the quality of Freedom of Information (FOI) legislation (more on this in the Technical Paper).

Information available via the IATI Registry was automatically collected and tested by a data quality tool. In 2014, IATI data needed to be available via the IATI Registry in order to be taken into consideration. Based on feedback from peer reviewers and donors, and following a public consultation, some of the data quality tests were improved in 2014 to ensure that they accurately measure the quality of IATI data in line with the agreed IATI Standard.

Information published to all other sources was collected via a survey. The survey is designed to assess the availability of information corresponding with the 36 publication indicators and the format in which it is published. All surveys were completed using information pertaining to the recipient country (or thematic sector, if the donor organises itself along those lines rather than by countries) receiving the largest amount of development flows by value from that donor agency (all development flows including ODA, OOF and non-traditional flows such as climate finance and south-south cooperation are taken into consideration for the purposes of the ATI).

To establish that information is consistently published at the activity level, five activities were randomly selected within the country or sector. The 68 organisations assessed were invited to review the surveys and provide feedback. Their responses were then independently reviewed by expert civil society organisations (CSOs) and by Publish What You Fund before the surveys were finalised. All the information collected for the ATI is stored on a publicly accessible platform called the Aid Transparency Tracker, including responses from donors and independent reviewers.

Publish What You Fund is grateful to the many people involved in producing this Index. Particular thanks goes to the peer reviewers who have advised on the approach and methodology. Their comments and suggestions were gratefully received. The 2014 reviewers included:

• Bill Anderson, IATI Secretariat
• Neissan Besharati, South African Institute of International Affairs, University of the Witwatersrand
• Laurence Chandy, Brookings Institution
• Molly Elgin-Cossart, Center on International Cooperation, New York University
• Brian Hammond, adviser, IATI Secretariat
• Alan Hudson, Global Integrity
• Rolf Kleef, Open for Change
• Ben Leo, Center for Global Development
• Marie Lintzer, Natural Resource Governance Institute
• Afshin Mehrpouya, HEC Paris
• Larry Nowels, independent consultant
• Paolo de Renzio, International Budget Partnership, Center on Budget and Policy Priorities

We are immensely grateful to the 55 organisations that independently reviewed our initial assessments of the donor organisations covered by the ATI.

• Evelin Andrespok, Estonian Roundtable for Development Cooperation (Estonia)
• Diana Bacheva, Access to Information Programme (for Bulgaria)
• Réka Balogh, Hungarian Association of NGOs for Development and Humanitarian Aid (for Hungary)
• Gordan Bosanac, Centre for Peace Studies/Croatian Platform for INternational Citizen Solidarity (for Croatia)
• Sylvain Browa, Save the Children USA (for UNICEF)
• Adele Broadbent and Wren Green, Council for International Development (for New Zealand)
• Ricardo Corona and Manuel Guadarrama , IMCO – Mexican Institute of Competitiveness (for IADB)
• Tigere Chagutah and Thembinkosi Dlamini, Oxfam South Africa (for AfDB)
• Coordination Sud and its members (for France)
• E sol Cho, ODA Watch (for Korea)
• CONCORD Aidwatch and its members
• Pedro Cruz, Platforma Portuguesa de ONGD (for Portugal)
• Koos de Bruijn and Mieke Olde Engberink, PARTOS (for the Netherlands)
• Luca de Fraia, on behalf of the Italian CSO Task Force on Development Effectiveness (for Italy)
• Rachel DePlaen and Bart Tierens, CNCD-11.11.11 (for Belgium)
• Amy Dodd, UK Aid Network (for UK)
• Casey Dunning, Center for Global Development (for U.S. Department of Defense)
• Andrea Girmanová, Platforma MVRO (for Slovakia)
• Lorna Gold, Trocaire (for Ireland)
• Evija Goluba, LAPAS – The Latvian Platform for Development Cooperation (for Latvia)
• Jo Marie Griesgraber, New Rules for Global Finance (for IMF)
• Laia Griñó, InterAction (for USAID)
• Tanja Hafner Ademi, Balkan Civil Society Development Network (for EC Enlargement)
• Minna Havunen, The Finnish NGO platform KEPA (for Finland)
• Aoi Horiuchi and Kanako Sugimoto, Japan NGO Center for International Cooperation (for Japan)
• Marjan Huč, SLOGA – Slovenian Global Action and Anita Ramsak, Institute Ekvilib (for Slovenia)
• Angela Kageni, Aidspan – Independent Observer of the Global Fund (for the Global Fund)
• Marc Keup, Action Solidarité Tiers Monde (for Luxembourg)
• Susanne Kirk Christensen and Julie Koch, Save the Children Denmark (for Denmark)
• Jack MacAllister, amfAR – The Foundation for AIDS Research (for U.S. PEPFAR)
• Fidanka McGrath, CEE BankWatch Network (for EBRD)
• Members of the AidWatch Working Group, SKOP (for Malta)
• Vitalice Meja, Reality of Aid Africa (for Gates Foundation)
• Jana Milerova and Katarina Sramkova, FoRS – Czech Forum for Development Cooperation (for Czech Republic)
• Scott Morris, Center for Global Development (for U.S. Department of the Treasury)
• Julius Norvila, Lithuanian Development Education and Cooperation Network (for Lithuania)
• Michael Obrovsky, Austrian Research Foundation for International Development and Sophie Vessel, Global Responsibility – Austrian Platform for Development and Humanitarian (for Austria)
• Leonardo Pérez-Aranda and Saya Sauliere, Oxfam Intermon (for Spain)
• Charles Perla, AidData (for China)
• Lydia Poole, independent consultant (for UN OCHA)
• Elvira Prohaska and Jan Stiefel, AidRating (for Switzerland)
• Jonathan Pyrke, Development Policy Centre (for Australia)
• David Redman, Bank Information Center (for World Bank IDA)
• Fraser Reilly-King, Canadian Council for International Cooperation (for Canada)
• Sarah Rose, Center for Global Development (for U.S. MCC)
• Jana Rosenboom, VENRO – Association of German Development NGOs (for Germany)
• Adela Rusu, FOND (for Romania)
• David Saldivar, Oxfam America (for U.S. Department of State)
• Claire Schouten, Integrity Action (for EC FPI)
• Zuzana Sladkova, CONCORD Europe (for EC DEVCO)
• Xavier Sol, CEE BankWatch Network (for EIB)
• Kerry Smith, Development Initiatives (for ECHO)
• Peter Sörbom, CONCORD Sweden (for Sweden)
• Yiouli Taki, INDEX: Research and Dialogue (for Cyprus)
• Amy Taylor, CIVICUS (for UNDP)
• Fotis Vlachos, PLATEAU – Hellenic Non-profit Organisation (for Greece)

We would also like to thank the following donor organisations that engaged in the 2014 ATI process by reviewing completed assessments for their respective organisations.

• African Development Bank (AfDB)
• Asian Development Bank (AsDB)
• Australian Agency for International Development (AusAID)
• Austrian Development Agency (ADA)
• Bill and Melinda Gates Foundation
• Canadian International Development Agency (CIDA)
• Croatian Ministry of Foreign and European Affairs
• CyprusAid
• Czech Development Agency
• Denmark, Ministry of Foreign Affairs
• European Bank for Reconstruction and Development (EBRD)
• European Investment Bank (EIB)
• Estonia, Ministry of Foreign Affairs
• European Commission, Service for Foreign Policy Instruments (FPI)
• European Commission, DG Development and Cooperation – EuropeAid (DEVCO)
• European Commission, DG Humanitarian Aid and Civil Protection (ECHO)
• European Commission, DG Enlargement (ELARG)
• Finland, Ministry of Foreign Affairs
• France, Ministry of Economy and Finance (MINEFI)
• France, Ministry of Foreign Affairs and International Development (MAEDI)
• French Development Agency (AFD)
• GAVI
• Germany, Foreign Office (AA)
• Germany, Ministry for Economic Cooperation and Development (BMZ)
• Greece, HellenicAid
• Hungary, Ministry of Foreign Affairs
• Inter-American Development Bank (IADB)
• International Monetary Fund (IMF)
• Irish Aid
• Italy, Ministry of Foreign Affairs
• Japan International Cooperation Agency (JICA)
• Japan, Ministry of Foreign Affairs
• Korean International Cooperation Agency (KOICA)
• Latvia, Ministry of Foreign Affairs
• Netherlands, Ministry of Foreign Affairs
• New Zealand, Ministry of Foreign Affairs and Trade (MFAT)
• Norway, Ministry of Foreign Affairs
• Poland, Ministry of Foreign Affairs
• Portugal, Camões Institute for Cooperation and Language (CICL)
• Romania, Ministry of Foreign Affairs
• Slovak Agency for International Development Cooperation (SAIDC)
• Slovenia, Ministry of Foreign Affairs
• Spain, Ministry of Foreign Affairs and International Cooperation
• Swedish International Development Cooperation Agency (Sida)
• Swiss Agency for Development and Cooperation (SDC)
• The Global Fund to Fight AIDS, Tuberculosis and Malaria
• United Kingdom, Department for International Development (DFID)
• United Kingdom, Foreign and Commonwealth Office (FCO)
• United Nations Children’s Fund (UNICEF)
• United Nations Development Programme (UNDP)
• United Nations Office for the Coordination of Humanitarian Affairs (UN OCHA)
• United States Agency for International Development (USAID)
• United States, Department of Defense (DOD)
• United States, Department of State
• United States, Department of the Treasury – Office of Technical Assistance
• United States, Millennium Challenge Corporation (MCC)
• United States, President’s Emergency Plan for AIDS Relief (PEPFAR)
• World Bank, International Development Association (IDA)
• World Bank, International Finance Corporation (IFC)

Publish What You Fund is indebted to all those who have provided advice on the methodology and inputs on the report. Special thanks to all those who participated in the public consultation for improving the data quality tests used for assessing IATI data – Riza Abdul and Akshay Sinha (UNDP), John Adams (DFID), Bill Anderson (IATI Secretariat), Jamie Attard and Julia Gray (Gates Foundation), Heather Hanson (MCC), Chongshan Liu (AsDB), Yohanna Loucheur (Canada DFATD), Astrid Manroth, Harika Masud and Qiyang Xu (World Bank) and Theo van de Sande (MinBuza). We are particularly grateful to Erin Coppin for methodological advice and support with data analysis, and to Martin Keegan for his invaluable contributions to the continued development of the Aid Transparency Tracker. Special thanks also go to David Banisar and Toby Mendel for their advice on scoring disclosure policies; to Steven Flower and Ben Webb for their advice on automated testing; to Pelle Aardema for inputs on the case study on open data portals; to Caroline Kroon and Theo van de Sande for the case study on the Netherlands; and to Liam Hine and the team at SVI Design for all their creative inputs. We are also grateful to Access Info Europe and the Centre for Law and Democracy for allowing us to use the RTI Rating as a data source. Finally, thank you to our Board of Directors and our U.S. Advisory Committee, all of whom have been involved in this project in some way. Any omissions or errors remain the responsibility of the authors.