And I am fully supportive of those upcoming changes, Bill. Particularly as those changes are based on the robust user research carried out earlier this year.
In addition to the comments made by @reidmporter, @matmaxgeds and @bill_anderson, I doubt that it is possible to define a repository which will serve all the different use cases. A lot of functional design decisions would have to be made which are use-case dependent. E.g.:
- Do we split up transactions by their sector and country percentages in order to facilitate data use? If so, how? By sector, by region/country or both?
- How would we implement traceability? Would we attribute outgoing flows to the relative contribution of incoming flows or not?
- Etc.
Secondly, looking at the conclusions being drawn the last few years about the use of IATI data in country pilots, an very important problem is the completeness and accuracy of the published data. Wouldn’t we therefore better invest our very limited recourses in tooling, procedures, etc. enabling publishers to publish better quality data instead of trying to build another repository?
It would indeed be nice if the technical representation of IATI data would be separated from its semantic definition. A way to do this, would be to keep the current XML representation as the core representation and provide tooling to:
1 Convert non XML IATI data to IATI XML (e.g. convert CSV to IATI)
2 Retrieve non XML IATI data from IATI XML (e.g. convert IATI to CSV)
So I would keep the rich and proven XML format as the core technical representation for IATI data. The advantage is that we keep one canonical technical representation for all IATI data which has enough power to model all use-cases.
In such an approach it would i.m.o. be important to automatically check the consistency and conformance of the data being converted to the IATI. So the tooling should not only do the technical conversion, but should also do an automated quality check. Ideally there would be a centrally maintained data quality service, which could be used by all tool developers.
1 Like
We can’t blame OECD for our own blunders - we have designed the IATI standard in a way that allows for a conversion of DAC-formatted CRS++ data into a (simple) IATI-file, but we have made it impossible to go the other way; IATI activity-files cannot be converted into CRS++ and are thus unable to fit into, or feed into the statistical databases of the DAC.
What we can do (and I make the statement at any chance) is to advice the DAC to use IATI as a secondary datasource whenever they draft statistical analyses that trancends CRS++ (e.g. assessing flows from private charity-funds or working on the TOSSD measure for the future).
Hi @OJ_ I would be really interested to know about why IATI > CRS++ is not possible - is there anywhere I can read about it?
I made a paper for a session at one of the TAG’s in Canada - not at hand right now - but the obstacle identified at that time was the org_type etc. codes of IATI that can’t be translated into CRS channel-codes. Will need to update that paper in order to verify wether this problem is solved with version 2.03 of our standard.
Another issue should be noted as well: We must recognise that the DAC has a strong need for ‘single point of contact’ to governments of donor countries, for the dialogue and data-validation. Even though some of us are currently reporting on behalf of our Government, making sure that our IATI-reported ODA-volume reach 100% of our DAC-reported ODA, this is not the case for all donors, and might not continue to be the case for us; IATI is designed for reporting by individual organisations rather than Governments. There are advantages, certainly, but it will be an organisational challenge for IATI-formatted CRS-reporting.
Thanks @OJ_ really interesting to know - I found it in the 2015 schedule in case anyone else was there and can share the paper
1 Like
If you allow me,
I do support this, most of the data users depend mainly on some of the attributes(fields), we could have something such as “Basic” data or “Core” data and “Extended” or “Complete” data, this will:
-
Simplify exporting the “Basic” data to Excel format for the user
-
Simplify the publishing of the “Basic” data which minimize the invalid data and increase the reliability and quality of data
Hi @ibrahim,
I think this is a great idea. Does the recent work that was done on ‘IATI users’ help to define what data is needed by the different use cases? Or for the ‘basic’ needs, I can share the data that the various countries I work with would love to be able to easily access (in flat/excel) format, basically: a row per project: funder(s), implementer(s), total project value, local sector, start date, end date, grant/loan, disbursements to date, disbursement in last local FY, disbursements in next local FY, description (incl in local language), local contact name, local contact email. ADM1 geographic information, humanitarian/development.
If agreed, we could then work on making sure that that very restricted set of fields is comprehensive, not double-counted, uses local language.
Matt
Hi @matmaxgeds,
Thanks for your reply, to me these fields is fair enough, I only want to add a note for how to represent the multi-valued fields such as sectors or donors in a flat Excel file, I propose to separate them by either “,” or “/” or whatever else, this will help to import this in a database later, and keep that each row has one activity.
This sounds like ODA macro-management. I accept that this may be the most prevalent use of AIMS at present, but is this really the IATI vision? Is this really what in-country decision-makers need? I would go so far as to argue that ODA macro-management (for which double counting is an issue) is an edge case. Is IATI a bean-counting balance sheet, or part of what the data revolution promotes as data-driven decision-making?
I agree that we can provide more focused (even standardised) outputs from IATI data, but for every person satisfied with this use case you will find many to the contrary. We already have a ‘basic’ standard, common to all: it is defined by mandatory fields.
To return to my much-used UNDP example. Would you like to see the arithmetically clean $400m number they report to the CRS, or the $4bn they account through IATI for their activities on the ground?
Hi,
Is there a list of the mandatory fields somewhere? I tried here in the guidance but couldn’t see it. There is mention in the upgrades for specific mandatory fields?
[quote=“bill_anderson, post:21, topic:1047”]
Is this really what in-country decision-makers need?
[/quote] from my experience, yes - it is not everything they need, but a list of all the active and pipeline projects solves 80% of the needs for 80% of the users. At worst, it is a good starting point for them to be able to follow up further - at which point the use cases become too individual/unique to be able to cater for as a consistent group.
[quote=“bill_anderson, post:21, topic:1047”]
is this really the IATI vision?
[/quote] - I am not sure what the IATI vision is, but it would seem to me that first catering to the dominant user need would be a good start - I could be wrong that this is the dominant user need - and I think that DI commissioned some research last year working out who the users of IATI data were, is that something that is public yet? If not, a public mapping of the different user groups and their data needs sounds sensible to help define what the IATI vision is - and to guide future iterations of the standard/effort.
RE the UNDP example - of course we want the 4bn (UNDP mainly as an implementer, not as a funder), but are you suggesting that in the quest for simple export of basic fields in a flat format, we will not get the $4bn, but instead the $400mn?
I am agreeing (110%) that your use case can be met by a standardised export - and we should do this.
I’m disagreeing that your use case can be used to define the standard itself
Hi - also agreed, IATI is much more than just this one use case
aha - thanks @bill_anderson - based on that, I think @ibrahim and I are suggesting a slight expansion of the core/manadatory fields?
Agreed(100%)
Agreed(100%)
what I am proposing is that the standard has two parts: “Basic” and “Extended”, The “Basic” is the mandatory and the “Extended” is all the others.
Hi @ibrahim
Yes, I think that ‘basic’ and ‘extended’ are a good idea - although there might be ‘extended - results’ and ‘extended - openAG’ etc, i.e. multiple different extensions for different more specific use cases. The reason I like this idea is that it should make it easier to focus on comprehensiveness and usability in the key fields first - and at the same time, perhaps answer some of the needs of those currently campaigning for simplification, and those wanting more extensions. What worries me is that IATI already has some structures e.g. the different parts of the XML tree, and this would add another structure that runs across that which would add complexity.
Matt
As this is a matter for a Major upgrade I guess we have a couple of years to prepare a campaign around this.
Noted…campaign starting now 
Any ideas about the paper I thought that DI commissioned last year on IATI users?