IATI Datastore's new Query Builder is launched for testing

Notifications
#1

IATI has launched a new online tool offering easy access to development and humanitarian data published according to the IATI Standard. This complements IATI’s existing suite of online tools for data users such as d-portal.org, and allows for much more detailed queries and searches of published data than previously possible.

Try the new IATI Datastore Query Builder now.

About the IATI Datastore Query Builder

The new IATI Datastore Query Builder allows complex searches of spending and activities published according to the IATI Standard by donor governments, development finance institutions, UN agencies, non-governmental organisations, foundations and private sector organisations. The Query Builder uses data available from IATI’s new Datastore, released in September 2019.

By entering search parameters into the online tool, users can access development and humanitarian activities across 32 different fields (elements) of the IATI Standard. For example, users can filter by transaction type, activity status, or whether an activity is specifically addressing a humanitarian crisis.

1097×919

Improved access to IATI data

The new IATI Datastore Query Builder provides access to data that complies with the most up-to-date rules of the IATI Standard. The new tool will:

  • Provide data from organisations that have used version 2 of the IATI Standard.

  • Provide data that adheres to the Standard’s Schema. The IATI Schema provides the exact order and format in which publishers provide their IATI files. Technical information about the IATI Schema can be found on IATI’s Reference site.

  • Provide more fields for querying and searching IATI data.

CALL TO ACTION: Please test the IATI Datastore Query Builder 08th October - 05th November)

Users familiar with the IATI Standard are invited to test the new IATI Datastore Query Builder and provide feedback to the IATI Technical Team and Zimmerman and Zimmerman, which have worked closely together to develop this new tool. Specifically, users are invited to:

  • Build queries and review the content of the data returned to ensure it accurately matches the elements/parameters selected
  • Check that the data returned is usable in its selected format (CSV, JSON, XML)
  • Report any issues

Feedback should be provided by raising an issue with Github on the IATI Datastore project. Users who are unable to access Github are welcome to share their feedback by emailing support@iatistandard.org.

If after trying the tool users would like a demonstration of the Query Builder, they should send an email with specific questions to support@iatistandard.org and if there is sufficient demand, the Technical Team will set up a webinar demonstration during the testing phase.

What will happen after the new Datastore Query Builder testing phase?

After the testing period ends on 5th November, the team will review all suggestions for changes that were not addressed during the testing phase to inform the next phase of work on the tool.

Based on the scope of changes needed following the testing phase, a date for the official launch of the Datastore Query Builder will be announced, and the old query builder will be phased out six months afterward.

Thanks in advance to IATI’s dedicated community who are able to support the testing phase of the IATI Datastore Query Builder. For any questions please email: support@iatistandard.org.

6 Likes
#2

Thank you to the Tech Team and Zimmerman and Zimmerman for giving us an opportunity to test-drive the new Query Builder. I spend some time trying it, and also enrolled colleagues for additional sets of eyes.
While specific issues were raised via the Github platform (kudos for the input forms provided on Github, they were very helpful!), I thought I would share some of our input here in a more reader-friendly form, for discussion and future reference.

  • General reactions were quite positive. People were glad to have fairly easy access to a lot of data. One person reported that the QB was much nicer to use than QWIDS (the DAC tool).

  • Some people raised concerns about accessibility issues, for instance the insufficient contrast between the background colour of the QB, the input fields and the drop-down text. It may look fine on bigger & newer screens, but on other screens it can be quite hard to see the fields or the text. This sounds fairly easy to fix so hopefully something can be done. More generally, we would suggest to submit the QB to an overall accessibility assessment to improve the experience for all users.

  • Other concrete suggestions for improvements include:

  • adding filters for tags (especially SDGs) and policy markers (especially Gender Equality);
  • a “preview” function before generating the data file (to e.g. see errors in the selection of filters without having to download and open the file);
  • providing better namefiles, perhaps with time stamps, to distinguish them (useful when several files are generated in short order);
  • enabling combined searches by role and type of organizations.

We hope our input will be useful and look forward to the release of an even better Query Builder soon.

5 Likes
#3

I have received additional feedback from another colleague that I thought I’d share as-is. This comes from someone who does not know - nor care - about the IATI Standard; she just wants to find information on aid activities.

I know this “unexpert” use wasn’t the focus for this initial round of testing. And I don’t agree with all her remarks/suggestions (for instance, some things may be better addressed through the D-Portal). However, this feedback is worth considering as we continue to work on ways to increase the use of IATI data, via the QB and other tools.

  1. Would be preferable to not have the coding appear before the organization name, or at all (e.g. “NL-KVK-40530953: Amnesty International The Netherlands”). Is there a reason why it needs to be visible to the user?
  2. I wish that the field labelled “Search in title, activity or description” under Additional Filters appeared instead on the first screen, under Core Filters. This seems to be the most user-friendly field of all.
  3. The Query Builder “allows you to build common queries… in CSV, XML, and JSON format” might be important to users, but does not need to be so prominent.
  4. I tried to find out about secondary education in Vietnam involving international orgs. Five minutes later, I had no information and gave up.
  5. I wish that selecting a “sector category” would then prompt the “sector” field to show only the items that fall within the sector. Ditto selecting a recipient country: I tried Afghanistan and Mozambique, but none of those countries’ provinces appeared.
  6. “Transaction Provider Org” and “Transaction Receiver Org” (under Additional Filters) were terms that I am not familiar with. I did not know what to do with either, and the drop-down menus did not help.
  7. A proposed enhancement would be to avoid assuming the user knows things like the DAC Sector Codes or sub-national country regions.
  8. Another proposed enhancement would be to indicate which are mandatory fields, and which are optional.
3 Likes
#4

I think the query builder should be accessible to analysts who are not necessarily familiar with DAC and IATI terminologies and methodologies. It shouldn’t be too difficult to construct a short guidance note that explains core concepts.

I think it should be possible to guide “unexperts” through queries such as “secondary education in Vietnam” through a brief list of tips.

3 Likes
#5

@bill_anderson I have been putting together a User Guide for the Query Builder which until now focuses largely on functionality.

It does however give a brief outline of how to use each filter, and links to the appropriate guidance page or codelist.

Do you agree that this would be sufficient in helping to guide a non-expert through their query or do you think there should be very brief terminology guidance (a glossary even) within the user guide itself?

1 Like
#6

Congratulations to @IATI-techteam and @siemvaessen and colleagues for amazing work, the Query Builder is looking great! Thanks also for the opportunity to provide input. I can see that an enormous amount of work has gone into this, so hopefully you are now into the final stretch.

I just wanted to check my understanding – I know that you can download the full set of results in XML. However, that XML output is not currently IATI Standard XML. I think the plan is to make full IATI-XML data available in the near future, but it would be great if you’d be able to confirm that, to ensure that the Datastore can be used by applications expecting IATI data.

Many thanks!

(For example, the title for one random activity currently looks like this):

<arr name="title">
  <str>[{"lang":"en","narrative":"GCE: Maternal Immunization to protect infants against malaria"}]</str>
</arr>

rather than this (which would be valid IATI-XML):

<title>
  <narrative xml:lang="en">GCE: Maternal Immunization to protect infants against malaria</narrative>
</title>
4 Likes
#7

Perhaps even a ‘simple’ vs ‘expert’ switch which defaults to ‘simple’ for the unexpert.

1 Like
#8

@JoshStanley,

I’m happy to read through what you’ve written. It is not wrong to assume that if I can understand it than anyone should be able to understand it. I’m very far from being a technical expert. Just ask @siemvaessen.

Kind Regards,
Michelle

2 Likes
#9

Stop saying that :slight_smile:

1 Like
#10

Ok so I’m an IATI subject matter expert but the IT technical stuff still kicks me in the #*$&. :blush:

2 Likes
#11

Ooh… I’m a fan of this idea.

1 Like
#12

Could you share them item by item on the Github issue tracker by any chance…?

1 Like
#13

As I said as intro, I don’t necessarily agree with all of them - or don’t necessarily agree they should be addressed for initial release - which is why I didn’t put them on Github. But if you want to have all suggestions in a single location for future reference, happy to put them on Github.

#14

Not sure about this. I wonder if the “simple” queries should be re-directed to the D-Portal instead?
Though yeah, perhaps users could have options to make things more or less complicated (e.g. show/hide guidance, show/hide codes)

#15

It would be great to see the user stories etc of what the two different tools (QB vs D-portal) are targetting e.g. at the moment, d-portal has limited search options, so the role for the QB might be that it allows searching via all IATI fields (as made available by the DSv2) - this would be consistent with the suggestion that “simple” queries should be sent to d-portal, whereas the QB is for complex queries, from non-technical (non-programmer) users. Or maybe it is something different, but it would be good to share what the aim is for how these different IATI tools will coexist as it might make answering these questions much easier, also the ones on whether to e.g. have searching by finance_type, or untying_status etc in the QB interface.

2 Likes
#16

My constant reminder that I’m not a “technical” person stems from my US/private sector background where “technical” generally refers to IT/computer folks whereas someone with a deep knowledge of something is a subject matter expert or SME.

I don’t want to slip hairs here by starting a big discussion on the use of “technical”. But I do think what we are getting at is the use of the datastore for people who are not deeply versed in the data elements and their quirky definitions now how to extract info using an API. Perhaps “Datastore Guidance for non-IATI Specialists”

But I still offer my services as a reviewer. I’ve only been working in this space for 2 years and I still remember how painful and steep the learning curve is for IATI. I still struggle really.

Kind Regards,
Michelle

1 Like
#17

Thanks @YohannaLoucheur

#18

Re-raising this question from @markbrough earlier in the thread.

1 Like
#19

Yes, it’s still planned and it’s currently being assessed by the Datastore team.
We will update once we have a final roadmap and eta status (which we’re working on at the moment)

3 Likes