pyIATI has definitely not been abandoned and the reasons for its existence are as present today as they ever have been!
Back in early 2017 we posted our long-term plans to make IATI technical infrastructure more robust, more scalable and more sustainable. This was also presented at the TAG2017. The key challenges are a set of aging technical products (in multiple languages) built quickly to prove the concept of IATI at a time when the number of publishers and datasets was a fraction of what it is now. The refactoring work to pay down the technical debt this approach inherently created as a side effect happened in places but, even now we’re left with a large codebase that has at least 3 approaches (in 3 languages) to downloading a dataset.
We were really encouraged when pyIATI itself even seemed to receive rapturous applause on Discuss and again duing last year’s TAG.
The first commit was indeed back in early 2017 and since then, pyIATI has had at most 6 months of active development which could roughly be categorised into three main chunks:
- February 2017 (initial prototyping)
- July-September 2017 (main development)
- November (refactoring and work on scalability for new IATI versions).
If only we’d been working on it since January 2017!
It is being refactored into existing IATI tools where time allows. It currently exists in the schema test runner (cited as the robust standard upgrade implementation process we’ve had). Additionally significant work has gone into including it into an upgrade of the query builder (which currently runs on very old-school PHP ) which is due to be completed as soon as we have the website in front of the MA!
pyIATI will also be a key part of work to incorporate the IATI Standard reference docs into further iterations of the new IATI website. It will also form part of validation functionality due to be added to the Registry in coming months.
We would have loved to have come further with it - integrated it into more tools, have a full readthedocs site, solved all of IATI’s technical issues etc - but given we are an incredibly small dev team (at least by industry standards) of 2 full time developers for the amount of products we have and building a robust reference implementation of the IATI Standard uncovered more gremlins than we thought: each that need designing for in a robust and tested way.
All in all, we have a tool that is designed for use (well, our use as tool maintainers), offers a good vehicle we have for integrating single-point-of-responsibility code across IATI products and can even do stuff right now!
Since last Autumn, IATI’s Governing Board have set priorities to work on the 2.03 upgrade and a new IATI website, which has been the focus of dev time. Nonetheless, we do try to keep pyIATI ticking along - indeed releasing pyIATI v0.4.0 was indeed on the cards for today’s ‘Maintenance Monday’ until we had uncovered Discuss needed a critical update as well as some issues with Discuss Twitter logins
Indeed some of the README documentation could do with a review given what is possible with IATI and even the much cited ‘experimental’ tags added at the start of the project (sometimes it sucks to be transparent right?!) are due for removal now. Some (mostly dev) library requirements are indeed out by a few minor versions but even the best of us suffer from this
We see pyIATI as the future of IATI code. More features are planned (subject to MA and governing board approval) in coming months when we can focus dev attention onto ensuring validation functionality is robust enough to work with all datasets. Alongside this, we’ll continue to seek out every opportunity to integrate it into other tools where we can.