Results: discussion space and TAG 2016/17 path

Community Zone Results
#1

Hi, we are seeing a number of organisations who are now either voluntarily or contractually required to use the Results element of the IATI Standard. This element is probably the most under-developed of the IATI Standard, and yet open data on results remains a key aspiration of both IATI (it’s included in the new Vision and Mission) and the Sustainable Development Goals work.

I’d like to set up a discussion on the Results element here (recognising that this has been attempted before) which then feeds into a Results path at the next TAG meeting (in 2016 or early 2017?) to help develop both some consistency in how the element is used, share best practice and solve some problems.

Here’s why:

  1. There are pockets of expertise developing, for example in the Netherlands and amongst development resource providers, and it would be good to bring this together in one place in a very practical way.

  2. I think we need to develop a common understanding of what the word ‘results’ means, and what IATI results data can and can’t be used for.

  3. We need to gather and share evidence from Monitoring and Evaluation people within organisations, to check what results they want to record, and how they are want to record them. And also evidence from users that the information that they are getting from the Results element is meaningful and not misleading (ie through the attempted aggregation of apples and pears).

In terms of outputs from this discussion and particularly the TAG, I can see the following would be really useful:

  1. A recommendation to the IATI technical team on any updates needed to the Results element of the Standard to make it fit for purpose.

  2. I think it would be good to develop and publish a set of examples - real-world problems and solutions in order to share the knowledge more widely and to ensure much better quality data. So for example, who records what results in a group of organisations working on one project? How do you define attribution? How do you report qualitative results in a way that’s useful to data users?

What do you think?

Results: allow disaggregation
Results: improve consistency of result standard: aggregations-status
Results: improve consistency of result standard: baseline
Results: use of narrative elements
Results: statement to describe the purpose of IATI results data
Results: recognising partner contributions
Results: allow disaggregation
Results: represent more than quantitative data
Results: keep old results
Results: reporting how and why IATI results were or were not achieved
Results: Make the quality of evidence behind IATI results data transparent
Session for TAG2018 on results data - looking to collab
Results – allow disaggregations of results data (included 2.03)
Results – represent more than quantitative data (included 2.03)
Results – improve consistency of results standard (included 2.03)
Results – allow disaggregations of results data (included 2.03)
Results – add more indicator vocabularies (excluded 2.03)
Results – recognising partner contributions (excluded 2.03)
#2

Hi all,

I fully agree that a discussion on the results element is needed in order to improve it. We’ve been working on implementing an IATI compliant results framework for nearly a year now (see this blog) and have gotten quite a bit of feedback from our users, which are mostly the M&E people of an organisation.

Some questions that pop to mind right now are:

  1. How can we disaggregate our results for a certain indicator (e.g. we report on people trained, but we’d like to distinguish between men and women)?
  2. How can we link our outputs to outcomes or impacts? A nice example has been made by the WASH Alliance, see their results framework here.
  3. Why is the ‘Aggregation status’ field on result level? What happens if one indicator within a result is suitable for aggregation and another is not?
  4. Is it enough to have just an ‘Aggregation status’ (yes/no) field? A simple example is that you’d sometimes want to sum of several indicators to aggregate, and sometimes the average.

I’m not an M&E person myself, but in general I’ve found the M&E field more complex than I thought. So it would definitely be good to have some experts join in on the discussion.

Kind regards,
Kasper

#3

Just to give everyone a heads up that we are convening a working group on improving the IATI Results element led by Wateraid (@mikesmith). We’re aiming to present a draft proposal at the TAG to workshop with you all into a final proposal to hopefully include in the consultation for this year’s upgrade of the Standard. We’ll be sharing updates on this thread and would welcome others to contribute via this page too.

#4

Kasper - can you contact me at Bond? We are doing some work on the Results element this month to feed into TAG and it would be great to have you involved. Sarah @Kasper

Add document-link to Results indicator (included 2.03)
#5

Hi All - in preparation for the IATI TAG workshop on Wednesday 1330-1430, attached PrinciplesForIATIResults.docx (214.0 KB)
is the output of the consultation driven by Monitoring and Evaluation experts from UK CSOs, building on the Bond document "Publishing results to IATI” available here: https://www.bond.org.uk/resources/publishing-results-to-iati , and previously mentioned here: Sharing Results using IATI data standard: will it improve learning and accountability?

Within the attached document are 8 independent Principles concerning the IATI results standard, which have given rise to the following corresponding suggestions for modifications:

#6

Attached is a summary of all the technical aspects of the Principles:
SummaryTechForPrinciples.xlsx (18.8 KB)

#7

Updated summary table with typos fixed, and red text to indicate minimum suggestions discussed at the 2017 IATI TAG Standards Day: SummaryTechForPrinciples.xlsx (19.3 KB)