Project update – Oct 2020

Murmurations is evolving. The first version 0.1 of Murmurations is up and running and several organisations have created Murmurations profiles to help test the concept. You can create your own profile and see it pop up on the demo map, in the demo directory and if you have RSS your posts will be added to an aggregated mix of news posts in the demo aggregator.

But version 0.1 was only ever really a proof-of-concept (POC) and so, having received encouraging feedback and gathered more input on the users’ needs, we have now embarked on a complete rebuild of Murmurations.

The update will be a major new version, which we are calling version 1.0, as detailed in this overview matrix and the sequence diagrams, which show the processes and flow between the different entities involved. The update extends the functionality of Murmurations to make it a ‘real’ protocol – i.e. a basic set of rules which anyone can use to define their own specific schemas in order to develop maps, directories and aggregators, of any kind, in any sector, on any platform.

Although the POC works, and is extensible via add-on schemas, it was not really built to scale. It was written in PHP and contained a set ‘base schema’ (the fields you need to fill in to create a profile), largely for testing and to prove the concept. However, after discussing this with various users it soon became clear that there is, and always will be, significantly differing opinions about which fields should be included in any schema, how fields should be named and validated, and which types of data they should accept. These ontological questions underscored the need for separation between the protocol itself and the schemas.

Version 1.0 of Murmurations takes a new approach, which is designed to scale, is built using JSON and JSON-Schema, leverages Git to deliver a flexible, extensible Field and Schema Library on which others can build, and a simple REST API to provide access to the libraries and the Index. Version 1.0 also leverages Schema.org wherever possible – but leaves the ‘context’ of any field open for the schema creator to define. For example ‘Mission Statement’ is not defined in Schema.org, so we can reference Wikipedia instead to provide the full context of this field. In a Murmurations it would be expressed in JSON as:

{
    "name": "mission-v1",
    "definition": {
      "title": "Mission Statement",
      "description": "A short statement of why the entity exists, what its overall goal is: what kind of product or service it provides, its primary customers or market, and its geographical region of operation.",
      "type": "string",
      "maxLength": 1000,
      "context": "https://en.wikipedia.org/wiki/Mission_statement"
    }
  }

Enabling Schema creators (i.e. people that want to build contextual, searchable maps and directories) to select and share standardised fields will greatly increase the interoperability of any data they collect – and therefore, the possibilities of deeper more valuable interrogation of their data. Enabling schema creators to create new contextual and inter-related fields of their own specification, provides a mechanism to build semantic triples and opens up Murmurations for unanticipated uses.

Developing an intuitive user interface for schema creators to build simple ‘profile creation’ forms is going to be a key element of V1.0 – and one which could help encourage the right balance between the creation of too many new, but similar fields (great for specificity, bad for interoperability) and the re-use of existing fields (great for interoperability, bad if you need greater specificity i.e. you want to limit a field to a specific set of options in a dropdown, for example). One idea to combat this is to list “used by x” under each field in the Field Library (where x = the number of profiles using this field –  as illustrated in the draft ‘Create / Edit Profile’ UI below) in order to encourage coalescence around a standard set of most used fields.

The plan is to build a flexible Profile Creation form, in javascript + HTML + CSS, which can work as a replacement to the standalone profile generator on Murmurations.network and also be embedded within a standard CMS, for example as an update to the Murmurations WordPress Plugin. Getting this, and the updated demos working nicely will be critical to the uptake of Murmurations – so if you have skills to share in this area, or know someone that does, please join us.

We also plan to require authentication to access the Index and its API to enable monitoring and, more importantly, once the network grows to implement rules for the use of the protocol. Whilst this may not sound very open, by carefully defining the right set of rules in conjunction with the users, we aim to deploy a semipermeable membrane around Murmurations in the form of a Membership Agreement. This will protect Murmurations and ensure it remains a commons, as opposed to a free-for-all, which anyone (aka Google) could spider and then monetise by incorporating everyone’s Murmurations data into their own knowledge graph. In this way we hope that Murmurations might potentially liberate the semantic web for the regenerative economy.

There is so much potential to this idea – and much to do to make it viable, but we are now getting to the point where the new plans for V1 are taking shape in public, on Git, so it’s easy to follow and get involved. Our main need right now is for someone with excellent javascript and CSS and/or possibly React skill to jump in and help us build out the UI.

You can see the overview of the protocol, the roadmap, the library and the project board to get a feel for things – and please get in touch or give us a shout in the forum if you can help, or are interested in building an initial schema for a decentralised map of your sector.