IATI Data Removal Guidance: input by 26 June

Notifications
#1

The IATI Technical Team has written guidance for data owners to explain the process for removing data published to the IATI Standard.

In the process of writing this guidance, we looked at 360 Giving’s data takedown policy to see how others have approached similar guidance on data removal; we also discussed this with the suppliers of the tools referenced in the document, as well as IATI Governing Board Focal Points Melinda Cuzner and Leo Stolk.

We would like to share the IATI data removal guidance with you in advance to make sure it is clear and straightforward. Please respond to this thread by Friday 26 June if you have clarification comments and we will make sure we address them. We will then publish this on our main website and link to it from the relevant publisher guidance pages as well.

Data retention on the Data Store
#2

I think there is something I am missing here, please can I check that I follow…the use case for this guidance is:

a) IATI data (not linked documents etc as they are not stored)
b) that a publisher has published, changes their mind, and wants to remove from public view
c) in less than the (maximum) 24 hours than it would take for it to be automatically removed from the various IATI tools under normal refresh processes
d) or in the situation (which is unclear whether it exists?) where some of the IATI core tools start caching data to smooth out a publishers IATI files hosting non-purposefully going offline - and in that case there would also need to be a system to automatically push changes to those systems as well

Is that correct? Do we have any metrics for how often this set of criteria is met?

In all cases, if the publisher stopped hosting the file, within 24 hours, (or a tiny bit longer if tools are caching), the data would be removed anyway?

It also doesn’t seem sensible to me that so many of these features are built into the registry - that seems to add another layer of complexity compared to a publisher just removing the offending file from where they host it? Or is this difficult for some publishers?

#3

Regarding data retention for each core tool, I wonder if it’s clearer and more future-proof to say data will be removed from all core tools within 24 hours. Then this guidance doesn’t need to be updated if core tools change their policy slightly.

I wonder if it’s also possible to include guidance for third party consuming applications. Something like the following:

  • You should aim to remove stale data within 24 hours
  • You must remove stale data within 7 days
#4

Thanks @matmaxgeds and @andylolz for your comments. We have provided responses to your points below, which hopefully answer your questions. Do let us know if that sounds good to you and we will proceed with adding the data removal guidance to our website at the end of this week.

@matmaxgeds in terms of how often the criteria or such cases might occur, the answer is that there are very few (we don’t have an exact estimation). But it is important to have the guidance out there for publishers when needed, even if there are few cases.

In all cases, if the publisher stopped hosting the file, within 24 hours, (or a tiny bit longer if tools are caching), the data would be removed anyway?

Yes, that is correct. The guidance is there to make this clear to the user.

It also doesn’t seem sensible to me that so many of these features are built into the registry - that seems to add another layer of complexity compared to a publisher just removing the offending file from where they host it? Or is this difficult for some publishers?

This is a sensible point, but the guidance follows current processes and we need to make it clear to the publishers how the IATI Registry works and what options publishers have if they want to remove data. If there are changes to the Registry, then the guidance will be updated.

@andylolz this guidance is on data removal, which is separate from data retention, which sets policies on, for instance, how long a dataset is retained for in a specific application. We have mentioned that we can look at this in the future, but at the moment the priority is on data removal. We decided not to include guidance for third party applications as they are out of our scope of responsibility.

#5

@IATI-techteam Thanks for your response.

Apologies for my choice of terminology. The section I was referring to is entitled “What happens when IATI data is removed?” This section includes details on how frequently each secretariat-managed consuming application refreshes its local cache of IATI data (i.e. how long each application retains data).

#6

@andylolz thank you for the clarification and no need to apologise! For the section you are referring to on how frequently each Secretariat-managed consuming application refreshes, we thought that it would be clearer for users who might not be familiar with all IATI tools to list them individually under the category “consuming applications managed by the Secretariat”. Our preference is to keep them listed separately. Do let me know if you have any other questions or anything else requires clarification.

1 Like