Indicator Timeliness Methodology - consultation space

Consultations
#1

Timeliness is one of the three core Busan principles and accounts for a third of the Indicator Score. It has two components:

Frequency

This table seeks to measure how often a publisher updates their data. There is no simple answer as to what constitutes an update. Can any trivial edit be regarded as an update? As transactions are the most numerous element reported in IATI the adopted methodology assumes that a publisher has updated their data if a transaction with a more recent transaction date than previously published is detected across the publisher’s entire portfolio. The table records how many times in each of the last twelve months such changes have been detected.

Timelag

The time-lag statistics attempt to assess how up to date the data is at the point that it is refreshed. For instance a publisher may refresh their data monthly, but the refreshed data is in fact three months old. Alternatively a publisher may only refresh their data once a year, but when they do it contains current data that is less than one month out of date. Transactions are the most numerous and most regularly refreshed elements in reported IATI activities and they are therefore used to make this assessment. The table of statistics shows the number of transaction dates reported in each of the last twelve calendar months. The current month is shown for informational purposes, but excluded from the assessment.

#2

The red flag on Denmark’s data indicates that we report disbursements when they are booked to be made. This should not be reflected as an error. IATI should not induce incentives to suppress such information. Booking information is just as valid as payment-information from a bank – once a payment is booked, it can only be altered by booking another entry. And, using an external, governmental standard Financial-Management-System, we do not have access to bank-information. So, we really have no choice, but to use booking-information, and find it silly if we should delay our publication until the day of disbursement.

The frequency ought to reflect that our API is updated every night. We will look into any problems regarding the refresh-procedure for our response-cache’s, which might explain that files, sometimes, are not delivered within the (quite narrow) time-out limit of the current IATI data-collection, and will get back to the tech-team when we have a positive documentation on our side.

#3

OJ I think the point you make regarding booked disbursements is totally valid. We need to find a way to make the methodology work.

We have not measured frequency beyond monthly as this is currently IATI’s stated best practice. With the demand for more timely data increasing, particularly on the humanitarian front, it is worth considering whether to include weekly reporting into the frame. One for the next TAG?

#4

Yes - let’s try to move ahead. But my point was aiming at the count of actual updates, recorded by IATI. 8 in September and 17 in October - there is an error somewhere, since we update the DB every night.

But - we will get back, when we have revisited our current cache-sollution, and are able to document that the error isn’t on our side.

#5

Hi!
I’ve seen that the Spain - Ministry of Foreign Affairs and Cooperation Timeliness Indicator has changed now to zero.
http://dashboard.iatistandard.org/publisher/maec.html

Our last update was done in 2015 October. And we are planning a new one before the summer.

But, in any case, I think that at least the methodology should identify (1) publication.
Because it is not right that we are publishing less than Annual when we haven’t reached 2016 october.

Thanks in advance.
Alberto Amaro

#6

Hi @amaro and thanks for your comments. The timeliness assessment is based on transaction data so the dashboard software checks for transactions with more recent dates than was previously detected. If more recent transactions are found then a publisher is assessed as having updated their data. In addition the assessment looks at the updates for the most recent months on a rolling basis because the intention is to encourage the publication of as up-to-date information as is possible.

At the moment the dashboard stats are showing that the latest transaction date it can find in the published data for Spain - Ministry of Foreign Affairs and Cooperation is ‘10-05-2015’. However, if you think this is not correct then please just let us know. Hope that’s okay?

#7

Hi Wendy,
Thank you for your fast answer.

I’ll verify this date “10-05-2015” but I’m sure that your are right because, by the moment, our information system is configured according to the requirements of the DAC. So, the data are always related with the previous year final report.

But, please let me take advantage of this point, to insist on my arguments against this Timeliness methodology based entirely on the Transactions. Because, in my opinion, it’s not reflecting the reality and here we have an example.

We published in 2015 October additional information about the activities (as I’ve said before, referred to the DAC final report that we were preparing). And we made an effort to have an initial data publication in 2015 May and a second one (more completed and exhaustive) in 2015 Oct that, according this dashboard, never happened.

I agree with you that the Transactions are very important, but other IATI elements should be too. So, I think that the Methodology should be able of detecting (and weighting) the volume of elements updated. And, to be more ambitious, the coherence between this updated information.

#8

Thanks @amaro. The criteria for determining whether a publisher has updated their data or not has always been difficult to determine. ‘New’ transaction data was decided upon not only because they are the most populous element and are updated most frequently across the ecosystem but also because it would not generally be fair to declare that a publisher has updated their data if they have only made eg. a spelling correction in a description or added a full stop to a title. As a result we would always very much welcome any suggestions from any member of the IATI community as to the specific mechanisms for how a publisher update can fairly be judged to have updated their data?