Dashboard methodology for frequency of publishing

Consultations
#1

We have been discussing our frequency methodology with the World Food Programme. They do in fact refresh their data monthly, but not necessarily in each calendar month. Frequency appears to be simple but there are a range of scenarios that are more challenging for a machine, rather than the naked eye, to assess. To make this fairer and keep it simple we are proposing to adopt the following methodology:

Monthly = Data updated in 7 or more of past 12 full months AND data updated at least once in last 2 full months.
Quarterly = Data updated in 3 or more of past 12 full months AND data updated at least once in last 4 full months.

The effect of this is to improve the status of four publishers to Monthly (Canada, WFP, CAFOD and Find Your Feet) and one publisher to Quarterly (Samaritan’s Purse UK).

Comments most welcome

#2

This is a timely discussion given that several of us likely are starting to review their Aid Transparency Index results, where this may be an issue.

Indeed, we noticed that DFATD’s assessment on the Dashboard did not reflect the fact that we moved to a monthly publication in June - and that it would take some time to do so.

As such, we welcome this revision of the methodology that will better reflect publishers’ recent efforts.

#3

Thanks, Yohanna. We are in touch with the Index team. This is relevant as this is one of the few elements of the Index that has shared methodology with the Transparency Indicator.

#4

In fact, WFP has updated IATI transactional data every single month (at least in 2015) and we never fully understood why the current methodology does not reflect that. I thus welcome any change to the methodology that ensures a proper reflection of monthly updates, in particular considering that these monthly updates require a substantial effort from publishers.

#5

Otto it is true that WFP has reported transactional data for every month of 2015. However, frequency measures when the update takes place. It is not true that WFP updated its data during each month. For example your August transactions were detected by the dashboard on 31 August but your September transactions were only detected for the first time on 21 October. In other words no update was detected during September.

We are proposing this change to provide a fair, rather than ‘proper’ reflection.

#6

Hi Bill,

In 2015 we published at least one update every given month with the transactions for the previous month (see below dates).
As you can see September data was published 20/10, August data was published 28/9, July data published 20/8, etc.

We were never told why your evaluation tool does not detect some of these updates properly, thus we couldn’t react, e.g. improve our process if necessary…

Possible cause:
I already asked Wendy if by any chance some future transaction dates we published by mistake in several months for income could have caused these problems.
Otherwise I can’t understand how the tool detected August data in August for WFP if we published this transactional data only in September?
I also suggested, that an easy remedy to that fault would be to disallow/block future transactions from being published in the validator.

What do you think?

Rgds
Otto

  • Jan 30 (Dec data)
  • Feb 3 (Dec data)
  • Feb 9 (Jan data)
  • Mar 4 (Feb data)
  • Mar 9 (Feb data)
  • Mar 13 (Feb data)
  • Mar 18 (Feb data)
  • Apr 13 (Mar data)
  • May 13 (Apr data)
  • May 19 (Apr data)
  • Jun 1 (Apr data)
  • Jun 2 (Apr data)
  • Jun 11 (May data)
  • July 16 (June data)
  • Aug 20 (July data)
  • Sep 28 (Aug data)
  • Oct 20 (Sep data)
#7

Here is a link to our log, Each row shows the date and time of each logged observation, and the most recent transaction date observed at that time.

#8

This has now been added as a GitHub issue for planned implementation in the week beginning 2 November 2015: https://github.com/IATI/IATI-Dashboard/issues/343

#9

Dear Bill,

This is a timely discussion as we in UNICEF have just started to review our AID Transparency Index and even though we have started our monthly publications since June 2015 our frequency on the Dashboard is reflected as quarterly. Any rationale on why you have chosen to check for data updated in 7 or more of past 12 months in the revised methodology. Can it be for data updated in past 6 months? we would very much like to reflect our publication frequency as monthly.

regards, Sanjana