The Challenges of Comparing IATI and OECD-DAC Data

We’ve just begun a project on transparency in humanitarian emergencies. The project very much focuses on data use needs amongst local and international actors on the frontlines of humanitarian emergencies. In the next few weeks we’ll embark on field trips to better understand the reality on the ground, but in the run up to our trips we took a look at the data available through the two main global reporting standards – IATI and the OECD-DAC.

Recently, IATI has begun tracking humanitarian funding, and having tracked funding flows before—albeit development funds—we thought it would be interesting to compare the amounts reported on IATI vs. OECD-DAC for humanitarian flows. As you might imagine, tracking humanitarian funding has been a bit different than development funding.

In doing this, we’ve found a few striking issues so far. Both IATI and the OECD-DAC use the same sector codes for emergency money (1. emergency response, 2. reconstruction relief and rehabilitation, and 3. disaster prevention and preparedness). This means that money that is categorised as “emergency response” in IATI should also be categorised as “emergency response” in the OECD-DAC. Because of this, one would assume that the numbers compared between the two platforms should match up pretty closely. As you’ll see, that’s not exactly what we found.

For this investigation we kept it simple. We looked at money spent in Iraq in 2017. We narrowed it down to some of the most important donors, and compared their emergency and overall spend in Iraq, as reported on IATI (accessing the data through d-portal), and through OECD-DAC. Here are the key findings:

1. Sometimes the data can confuse

The discrepancy in the IATI numbers for USAID and Canada have us pretty confused here. For both countries, the amount of money spent on emergencies is somehow more than the amount of money spent overall. Regardless of the emergency amount, money spent overall should be higher (as it includes emergency and also non-emergency spend). Also in the case of Canada, the overall spend reported through the OECD-DAC is significantly higher than that reported on in the IATI data. This means that funds being reported on OECD-DAC are either not being counted in IATI, or are just not being visualised properly on d-portal where we accessed the data. It’s not easy to get a quick answer for which one it is. [1]

2. It’s not always comparable, and we don’t know why

Germany and the UK’s Department for International Development (DFID) have a similar problem. As you can see, the numbers reported for Germany through the OECD-DAC are significantly higher than the IATI numbers–both for emergency and overall funds. DFID, on the other hand, has roughly equal emergency numbers between the two platforms, but somehow the overall spend reported on IATI is much higher than that on the OECD-DAC.

Again, there’s no quick way to figure out why this is the case, or to know which number–if any–is correct. Problems like these really undercut the trustworthiness of both of these platforms, and lack of trust is commonly cited as a reason for users not wanting to use IATI data.

3. Sometimes it’s missing

Here we have Japan, which given the amount of money it is reporting, is clearly an important donor in the region. However, because Japan does not report to IATI, we are unable to compare the results of the two platforms.

4. Sometimes it works

Finally, we have the EU, which has roughly equal numbers reported in both categories across both reporting standards.

This was just a quick experiment to see how the numbers added up, and as you can see, they don’t always. This is despite both IATI and the OECD-DAC using the same sector codes for emergency spending, meaning that they should line up pretty closely.

We don’t yet know the answer to why these results look like this–it could be anything from a visualisation issue or, for example, an under or over reporting of certain funds by donors. We’ll continue to investigate this as we work on our project, but we wanted to share these findings to see whether anyone in the aid transparency community has any ideas as to what’s happening here. In the meantime watch this space for more information as the project continues!

[1] Update: We’ve since learned that, at least in the case of USAID, this problem is due to d-portal’s visualisation of the data rather than an issue with the underlying data. It would appear that d-portal is double counting the amounts of certain activities when filtered through the emergency response marker.