Of all the features and aspirations of FOLIO, my favourite has always been the promise to empower libraries and be the antithesis of vendor lock-in. We’ve been hard at work on this project a while now and much of what we’re building delivers on this—it’s modular and de-coupled, APIs and interfaces are specified so you can take the parts you want, leave the ones you don’t, and build new pieces where you want to innovate. Or encourage your favourite vendor to. And of course, the core is all open source such that you can run your own deployment as an institution or even become a SaaS vendor.
But to really avoid lock-in we need to offer a streamlined experience for moving a FOLIO system between varying deployments. So far we’re still working on getting to the first deployment so this hasn’t necessarily been front of mind, but I feel it’s worth taking a moment to see what would be required to make this pleasantly frictionless—I doubt we’ve boxed ourselves in but I’d like to run through and see how it might work and put this on the radar.
To my mind, it seems like we need two main things to achieve portability. Okay, they’re not necessarily small but still, surely it’s not this easy? Please help fill in what I’ve missed so we can start to get a realistic picture.
Per-module data serialization
Since data and configuration are persisted in storage managed by Okapi modules, wouldn’t it suffice to move a system if there were a standard way to request a dump from each module of any persisted data and a way to rehydrate it on the other end?
We could make supporting something like a /serialize and /populate endpoint a requirement of any storage module. This is very different from something like a bulk export API where we want to be transparent and expose, say, user data to be processed by other systems as a part of a migration. Instead, this exports everything the module controls in a way very specific to FOLIO, this module, even this version, with the goal of replicating this tenant’s data exactly on another deployment of the same. In fact, it should probably include metadata to ensure it is only imported by a compatible module.
Aside from needing a ‘maintenance mode’ or something for Okapi so we can keep the data consistent while this process is running, I don’t think much stands in our way of implementing a feature like this?
Standard platform description
It doesn’t do much good to have very specific data that exactly replicates a FOLIO setup without the ability to re-create a platform that can import it. For example, the data stored in mod-configuration could come from a variety of Stripes modules and a different version may not be able to consume it.
For Stripes in particular we have a notion of a platform—a collection of modules a tenant might run and their versions. Each tenant then has a configuration detailing which of these are enabled. I believe we have some similar notions on the Okapi side of things. From this it should be readily possible to list all modules relevant to a particular tenant and their versions. And we could offer an Okapi module that would parse such list and highlight which modules are not available.
With the description of your source you (or your vendor) can create a destination environment to match. With a list of unsupported modules on your destination you know what you’ll need to change before you can move. Those changes (migrating between modules (perhaps even ones exposing the same interface) and between versions) are, I believe, a different and important other topic. While the upgrade path should always be clear, there may not always be a path to downgrade and inter-module migration will likely need to have explicit support and not be possible in all cases or may require manual steps.
Given that constraint (same modules, same versions), is it clear we can do this painlessly? Do we need other information about the source system to ingest its data?
