Revision of IATI Standard Upgrade Procedures

Standard Management Issues for Members Assembly
#1

Here is a draft discussion document on proposed changes to our upgrade procedures that will be submitted to the Member’s Assembly in October. The more that we can agree on before we get to Rome, the more likely we are to come away with a framework that works for all. I would urge everyone who has expressed interest in this matter to engage.

Could you make major comments in this thread (quoting sections here when relevant) and restrict comments on the Google Doc to finer/pedantic points.

This document is very different from the one we presented in Dar es Salaam. Hopefully it takes on board the further learnings we have gone through with 2.03 over the past months.

The biggest change is to make a clear distinction between content and technical implementation. We propose that we should formally approve the content - the explicit use cases for each proposal - before going anywhere near schemas, xml, etc.

Other changes include

  • New responsibilities for the Governing Board in initiating and approving decimal upgrades and overseeing integers
  • Procedures for involving the Governing Board in resolving disputes.
  • Specifying the necessary phases of an upgrade without prescribing the timeline (which should be determined on a case-by-case basis to take into account likely complexity and TAG and MA meeting timings).

I look forward to an open and constructive discussion.

4 Likes
Improvements to the IATI Datastore CSV Query Builder now live
#2

Hi Bill,

This looks good to me. Requiring proposals to be submitted as real world usecases – and distinguishing between the content consultation and technical consultation – should make it far easier for non-technical stakeholders to engage with the upgrade process.

4 Likes
#3

Hey Bill,

There’s some good evolution here. Some initial thoughts:

It might be useful to defer to the Semantic Versioning (SEMVER) framework for defining what can happen at MAJOR.MINOR.PATCH numbering points (moving to the language of major change, minor change and patch might also help build understanding with those who don’t immediately map those concepts for integer and decimal)

When thinking about a ‘beta’ of each version for testing, ‘Release Candidate’ might be a good terminology to use.

Does all of the documentation fall into additional rules and guidelines?

  • It seems to me it might be worth separating rules (which are normative, and affect ‘validity’) from guidelines (which might be thought of as advisory, and affect ‘utility’ or ‘quality evaluation’)?
  • If there is documentation that falls outside this governance process, is that clearly identified.

Terminology wise, I wonder if intention and implementation might better capture the point than content and implementation. I tend to find there is a three stage process from intent (we want to do X with data), via content (what kind of data so people hold, what can be extracted from existing systems, what can be queried effectively), through to implementation (what exactly should the data element look like).

I think moving away from tight timings makes sense with the reality of a process, but it would be good to keep some principles on minimum consultation periods, just to manage expectations about how long people will get to review major or minor changes.

The details of ‘step vi’ final approval may need some working out. In OCDS the governance groups responsibility at that point is just to confirm the process has been followed, but is not to have any say on the substantive content of the standard. I think there is a risk that as governing board were involved in setting intention, their involvement at this step could risk raising substantive implementation issues that should have been picked up earlier.

Happy to join a more in depth discussion as useful.

4 Likes
#4

Many thanks, @TimDavies

We agree that this is a better solution and have made appropriate changes to the proposal

Agreed and changed

All our documentation should make this distinction. Rules should be a MUST and guidelines a SHOULD or be RECOMMENDED. We will check consistency.

#5

I think you mean “MUST”!

+1 for semver and RC – that’s great.

#6

Thanks Bill.

On rules and guidelines, I was also thinking of a wider separation in terms of governance.

I.e. should changes to guidelines only be possible with a new version of the standard, or could they be updated outside that versioning process?

#7

Personally, I’d suggest that it should be possible to update Guidelines through a Patch Upgrade, so:

Rules may be:

  • Added - at a Decimal or Integer Upgrade, through the applicable consultation process
  • Modified (relaxed restrictions) - at a Decimal or Integer Upgrade, through the applicable consultation process
  • Modified (enhanced restrictions) - at an Integer Upgrade, through the applicable consultation process
  • Removed - at a Decimal or Integer Upgrade, through the applicable consultation process

Guidelines may be:

  • Added - at any point, subject to applicable notification and/or consultation
  • Modified (relaxed restrictions) - at any point, subject to applicable notification and/or consultation
  • Modified (enhanced restrictions) - at any point, subject to applicable notification and/or consultation
  • Removed - at any point, subject to applicable notification and/or consultation
#8

Section 3 talks about backwards-compatibility. It may also be useful to add something about forwards-compatibility - this has been relevant with 2.03.

Suggest that:

  1. All Patch Versions within a Decimal Version must be backwards-compatible.
  2. All Decimal Versions within an Integer Version must be backwards-compatible.
  3. All Decimal Versions within an Integer Version should be forwards-compatible where possible.
  4. An Integer Version generated through an Integer Upgrade must not be backwards-compatible with the previous Integer Version.

In the currently proposed wording: 1 is explicit (3.c.), 2 is explicit (3.d.), 3 is new and 4 is implicit (3.a. - SemVer requires this).

In terms of definitions:

Backwards Compatible
Let α be a Version with a Version Number and β be a Version with a higher Version Number.

β is Backwards Compatible with α if all conformant Datasets at α are conformant at β.

Forward Compatible
Let α be a Version with a Version Number and β be a Version with a higher Version Number.

α is Forward Compatible with β if for all conformant Datasets at β:

  • Let γ be a conformant Dataset at β
  • Let δ be γ but with all elements and attributes defined through an Upgrade Process from α to β removed
  • The elements and attributes defined in δ have the same meaning at α as in γ at β
#9

I agree with this approach @hayfield

#10

Hi Bill,
Thanks for this revision of the IATI upgrade procedure. It is a great improvement compared to the original upgrade procedures, because of the clear distinction between ‘what’ and ‘how’ and by making more explicit what the role of the MA is.

The proposed upgrade procedure makes a distinction between the handling of minor and major upgrades. I am wondering if this distinction is necessary. The difference between a minor (decimal) and a major (integer) upgrade is its backward compatibility. Substantial functional changes may occur in both major and minor upgrades (e.g. the addition of the Humanitarian elements in minor upgrade 2.03). So doesn’t this mean that the same upgrade procedure can be used for minor and major upgrades?

Because all functional changes will affect IATI members, I think it is vital that the MA has the final say of both minor and major upgrades. Both minor and major upgrades ought therefore to be initiated and approved by the MA. Initiation of a minor or major upgrade could for instance take place in a yearly cycle in which the MA determines the work plan for the next year and approves the terms of reference for the next minor or major release.

It is up to the governing body to oversee the execution of the work plan and monitor if the work plan is executed in accordance with the terms of reference. When executing the work plan, there can be consensus about how to implement a change proposals in the standard or not. For all change proposals for which there is consensus, the governing body approves and releases the change proposals.

When there is no consensus, the discussion is often about the interpretation of the required functionality or about the impact on non-functional aspects of the standard (e.g. usability, data-quality, etc.). Therefore, it is i.m.o. important that in these cases the MA has the final say, regardless whether or not it is an minor or major upgrade.

#11

It took me a while to get my head around forward-compatibility but I now get it and will add this proposal. This TV analogy works better for me:

The introduction of color television allowed backward compatibility since new receivers could receive black-and-white signals generated by old transmitters. It allowed forward compatibility since black-and-white TV sets still could receive a signal from a new transmitter.

3 Likes
#12

The standard is, by its nature, complex. Tools and services that simplify both the publishing and use of data must be the core priorities of the technical team tasked with maintaining the standard.

Tools and services may need to be upgraded after introducing a new IATI version, but in itself tools and services have no consequence for the governance of the upgrade process. To avoid confusion I would suggest to remove this point from the principles section.

#13

In discussing the upgrade procedures it is essential to (better) define the roles and functions of the ‘technical lead’ as well as the ‘technical team’.
In the current SOP of IATI they are loosely described as a function of DI, without defining their specific functions and or their relations with other parts of the secretariat, or with (chair) TAG, MA or (Chair) GB.

#14

The only reason for distinguishing between MA and Board for Major and Minor was pragmatic. With the MA only meeting once a year it means every upgrade, however trivial, is effectively spread over a two-year period.

#15

@theo.sande don’t you think this is best done in the institutional arrangements - where I see you have indeed raised this point

Currently the technical team’s role is the administration, coordination and technical implementation of upgrades. The technical lead oversees the work of the team, chairs team discussions on candidate selection and engages in consultations with the aim of reaching consensus (or establishing a clear lack thereof).

Members of the team, including the lead, also engage in consultation discussions in their personal capacity on issues within their own working experience.

#16

I take your point @Herman, you are quite correct (although I was referring to tools and services that simplify the publishing and using data in general, not how they are upgraded)

I do, however, think that, somehow, the implementation of the standard needs to be taken into account within the governance.

There is a paradox:

  • every publisher and user would like the simplest possible standard to meet their needs
  • many publishers and users have very different needs

Can we capture this in another way?

#17

A yearly update cycle (1 major or minor upgrade each year) might still be possible. At the yearly MA meeting the MA can decide about the terms of reference for the next years upgrade and decide about the conflict items of the previous year upgrade.

All the change proposals for which there is consensus, can be released on authority of the board. No need to discuss those again in the MA. So an upgrade release does not have to wait on the MA meeting.

After decision on the conflict items by the MA, it might be possible to release those items directly after the MA meeting as a patch release on the minor or major release.

#18

Papers for the Members’ Assembly need to be submitted on Tuesday 19th September so please could you submit any comments before 18:00 UTC on Monday 17th September.

#19

@bill_anderson Yes, this is exactly where a lot of discussions about the standard boil down to. The discussion about this inherent tension between the complexity and usability of the standard is now largely a matter of opinion of individuals with an interest in the standard.

Your proposal to separate the ‘what’ and ‘how’ in the upgrade procedure will help to more explicitly strike the balance between complexity and usability. But I think that will not be enough since we still need to decide for each technical implementation of a user need, if the proposed solution is workable.

What could help i.m.o. , is a set of criteria/principles/rules to determine if a solution does not unnecessarily contribute to the complexity of the standard, but still satisfies the user needs.

Examples of these rules (open for debate of course) could be:

  • solutions must be relevant for a majority of users within at least one constituency (to avoid adding complexity which benefits only a couple of publishers)
  • if an alternative to implement a user need already exists in the standard, the standard is not changed (to avoid technical redundancy, making processing logic more complex).
  • etc.

In case of conflicts, these criteria can than be used to advise the MA whether or not to accept a change proposal.

#20

I agree with the proposals.

In particular I like the separation of the What and How, and the need to explain changes in terms of user need first. That means there will be initial discussion around the extent and value of the user need rather than starting with less accessible discussions about details of tech implementation.

That single change I think goes quite a way to address concerns expressed by people in this forum about the purpose of the standard and how it can continue to meet user needs as they evolve.

The recent decimal upgrades have focussed on a couple of user “epics” on humanitarian and results. I think we can learn from those experiences about how to frame the user need questions better.

I think the GB rather than the MA should initiate minor changes. Adding the MA to this process would be too inflexible. The focus on user needs helps to ensure minor changes are meeting real needs that can be understood across the wider IATI community.

The principle of consensus is important here, and it would be useful to explore further community principles as Herman has suggested.

Additionally, could we make better use of an extensions approach to test user needs better? Take an Alpha/Beta/Live approach to changes to help understand the user need better. That might help to see how big the user base for particular elements might be, addressing Herman’s concern about scale.