Results – recognising partner contributions (excluded 2.03)

[mikesmith]

Title
Results – recognising partner contributions

Standard
Activity

Schema Object
iati-activities/iati-activity/result
iati-activities/iati-activity/result/indicator

Type of Change
Addition to schema

Issue
• Currently attribution is only possible for entire projects, not individual results or indicators.
• Why is this a problem?: An increasingly large proportion of projects involve working in partnerships, be it from working with community groups, governments, NGOs, the private sector, academia, our funders etc. This presents a practical but solvable challenge where each reporting organisation in the partnership must represent these joint projects in their IATI data, making sure they reflect their relationship to the project and to others in the partnership using elements like participating-org and related-activity.

However, there is also a challenge for reporting results. There is currently no means to attribute specific project results to specific organisations within a partnership. This is leading to a number of undesirable reporting practices, for example, some organisations are unrecognised for their input, or all implementing organisations are assumed to have inputted equally, or the project is split into many smaller replicated projects in the data (one for each combination of implementing partners).

Each of these practices cause confusion when trying to understand and use IATI data. Further they can disempower some organisations through the reporting process and they can create high reporting burden and opportunity for error through forcing organisations to artificially split and repeat their reporting of project results (and corresponding finances) into multiple sub-projects just for the purpose of IATI.

There is a growing convention that each organisation reports only their own results. Further, where there are overarching results (e.g. where multiple organisations’ contributions cannot be separated) these results are reported by the lead organisation (as determined by e.g. funding agreement structure) (eg: section 1.2 of the Netherlands Ministry of Foreign Affairs (NMFA) guidelines: https://www.government.nl/documents/publications/2015/12/01/open-data-and-development-cooperation). This convention can help to avoid double counting and clarifies the interrelatedness of organisations’ activities.

However, this convention only partially solves the challenge. For example, even for overarching results, not all results apply to all members of a consortium. Here the ability to recognise contributions to results would avoid the additional reporting burden, risk of error, confusion and lack of traceability from artificially replicating and fragmenting projects.

Also some organisations also do not have the capacity to report directly to IATI (think of the tiny pop-up organisations that are funded to take on specific causes such as Ebola but that do not have computer/internet access). Rather than remove their voice from IATI publishing we suggest it is preferable that the funding/reporting organisation, which will already be collecting the required IATI information as part of their funding agreement in many cases, is able to report and recognise the organisations efforts.

Proposal
We propose that a “light touch” attribution should be possible by adding the names and/or IATI organisation identifier of participating organisations that are involved in a result or indicator.

add participating-org (0…*) to result/ and result/indicator
with “ref” attribute to link to iati organisation identifier

(see http://iatistandard.org/202/activity-standard/iati-activities/iati-activity/result/ and http://iatistandard.org/202/activity-standard/iati-activities/iati-activity/result/indicator/ for relevant sections of the standard)

Standards Day
Workshopped at the TAG 2017 and mentioned at the end of the Standards day as part of the results section. Although there was very little time to discuss the proposal, no criticism of the proposal was offered. Proposal has been on IATI Discuss since March 2017.

Links
• This topic is discussed here: Results: recognising partner contributions
• This topic addresses Principle 6 from a consultation driven by Monitoring and Evaluation experts from UK CSOs Jan – Mar 2017 – see Results: discussion space and TAG 2016/17 path. Technical suggestions were devised by technology specialists at the Nethope Athens conference March 2017. In all around 30 M&E and technical specialists were involved in this consultation and it builds on a previous consultation by Bond 2015-16 (https://www.bond.org.uk/resources/publishing-results-to-iati - also on discuss.iatistandard : Sharing Results using IATI data standard: will it improve learning and accountability? ).
• Related discussion on importance of allowing attribution at project level: http://support.iatistandard.org/entries/82377659-Add-activity-id-attribute-to-participating-org-element

[IATI-techteam (IATI Technical Team)]

This topic has been included for consideration in the formal 2.03 proposal

[andylolz (Andy Lulham)]

I don’t see mention of participating-org/narrative here, but presumably this will be added in order to be consistent with e.g. http://iatistandard.org/202/activity-standard/iati-activities/iati-activity/participating-org/ ?

Particularly relevant if the participating-org doesn’t report to IATI (and therefore doesn’t have a self-declared organisational identifier).

[IATI-techteam (IATI Technical Team)]

Notes from consultation calls w/c 3rd July

Outcome:
The proposal was reviewed by those on the call and there was no objection from the group.
Mike Smith and Bob Walkden agreed to explore usage of how this proposal will work over the next week. With the aim of providing practical examples to the IATI community.

Discussion:
There were some questions on how this element would be used, and how to avoid double counting. The IATI tech team clarified that the IATI principles of traceability within financials should also apply to IATI results data.
There was general agreement for the need to explore more use cases of how this element would work. Particularly how this element would allow publishers to delineate responsibility of meeting targets down, and then back up, the chain.

[amys (Amy Silcock)]

Thanks again to all who have contributed to this proposal. It would be good to hear more use cases from everyone. @mikesmith and @bwalkden I’m particularly looking to you as per our discussion, to bring any practical examples on this proposal.

[bwalkden (Bob Walkden)]

Hi,

I think there’s been a slight misunderstanding about what Mike and I were doing. During the call I asked for examples of how the designers of the results structure saw it working, particularly with respect to what I see as a risk of double-counting. I wasn’t offering to develop any examples myself – I was asking for examples.

Mike and I subsequently had a meeting to discuss our understanding of the model, and I think Mike has taken action to drop a note to Bill Anderson. Mike, please correct me if I’m wrong.

As far as qualitative results are concerned, my thoughts are as follows, largely influenced by what I’ve heard M&E people from Save and other organisations say – I’m an IT person, so not qualified to comment on operational use of these things as I don’t have the experience:

1. Yes, everything can be quantified. That doesn’t mean that everything should be quantified, or that the effort to quantify / measure is always justified. In particular, some smaller organisations would not be able to cost-justify it, and are not publishing to IATI, so it falls to their commissioning organisation to do it, and the commissioning organisations would like to recognise the smaller partners’ contributions.
2. The results model seems to be designed around the commissioning organisation setting targets, and the commissioned organisation reporting actuals against the targets. Therefore if the commissioning organisation sets quantified targets the commissioned organisation should report actuals using the same scale. This needs to be realistic – if the delivery partner can’t measure the thing for some reason then is there any point in setting a quantitative target? If the commissioning organisation prefers to set some qualitative targets they should be free to do so.
3. Some organisations may wish to report results that are not linked to targets. If so, I don’t see why they should not be able to report qualitative results as this gives some background and richness to the numbers.

I’d be very happy to hear other thoughts and opinions about this.

Regards,
Bob

[mikesmith]

Sorry for the delay, and yes, thanks @bwalkden we decided that the queries were not about the specific proposal but the results section of the IATI standard in general (eg double counting etc.). We’d like to suggest that guidelines are written for how to publish IATI data in general (eg following the traceability discussions, what IATI is and is not Our shared understanding of what IATI is, and is not, etc.), but guideline writing is separate to this proposal or any other in the 2.03 consultation and seems a much larger topic. I hope that helps.

[pelleaardema]

In line with https://discuss.codeforiati.org/t/our-shared-understanding-of-what-iati-is-and-is-not I think this specific proposal is adding complexity to the results standard, while the IATI community is still in the process of gaining experience with the use of the ‘results section’ in more complex networks.

I’d be in favour of gaining more experience first, before we try to fix (perceived) problems.