Network Ecologies

Technical Plan

3.1 Summary of Digital Outputs and Digital Technologies

There will be four digital outputs for the project. Two are Open Source contributions to the academic community as public infrastructure. The third output will be a series of digital prototype productions of a transmedia publication to demonstrate the research contribution to the field of Open Access and digital publishing. The fourth outcome will be a prototype workflow for the generation of custom OER (in cooperation with the OER team of the Afro-European Mokolo initiative).

3.1.1 Open Source Public Infrastructure

The following software release would look to be compatible with the following example software frameworks. Data Futures (Westminster University), A-machine (Hybrid Publishing Consortium), Scalar (University of Southern California), Booktype and Superdesk (Sourcefabric), Fiduswriter, Transpect (le-tex), Tamboti (Heidelberg Research Architecture) and Superglue.

3.1.2 Enhanced Transmedia Citation and Reference Management

In such citation and reference management, if text is understood in computational terms as a linear string similar to a point in the linear position of the timeline of a video or the trace of a game thread, then these points can be identified, cited, referenced and used in the context of a publication. The design of such a management system is complex, as the fixed properties of print media are no longer the only parameter and the publication becomes more of a recipe for combining a revision point in a variety of distributed media. These features would be explored in a usable Beta prototype software implementation as part of the rapid prototyping research process.

3.1.3 Platform Independent Publication (PIP) Type

This is a software implementation to test the the Platform Independent Publication type, using existing open document standards. Through the project’s research, co-creation and design research methods, these standards would be implemented in to facilitate their use by academic practitioners.

3.1.4 Transmedia Publications: Hybrid Publishing Consortium Prototypes

A series of hybrid publications would be released as digital publications, making use of the existing HPC material and context. Some of the digital assets would manifest as hybrid digital print objects via print-on-demand digital printing.

3.2 Hybrid Publishing Consortium, Data Futures and xm:lab—Open Source Publishing Infrastructures

The long term collaboration and objectives of our research network (including HPC, Data Futures, and xm:lab) is to support Open Source infrastructures for transmedia, multi-format, scholarly publishing. Using single source architectures to output format such as; EPUB, HTML5, ODT, DOCX, PDF, PDF for print-on-demand. The stages in the publishing workflow and life cycle can be described as follows:
  1. Document Validation—writing and authoring
  2. Document Editing—text, citations, metadata and images
  3. Media Editing
  4. Layout Design—typesetting and templates; semi-automatic and automatic
  5. Publishing
  6. Publication Collections—libraries, bookshops, academic and OER repositories
  7. Transmedia Publishing API

In this research project we focus on three of these stages: validation, asset management, and a transmedia API.

3.2.2 Platform Independent Publishing (PIP) Standards

The objective of the project is to contribute to the working implementation of an open standards-based, transmedia-structured document, for multi-format publishing. A document-interactive validator will be a valuable tool for use by many document repositories containing unstructured documents that are not usable outside of very limited contexts. HTML5 as opposed to TEI or XML is emerging as the industry standard as an ‘intermediary file format’ with associated metadata and document settings, although this still has many problems such as footnote management. The benefits of machine-readable documents are that users can stay within their preferred writing environment while carrying out validation with the use of an API. The PIP standard allows for a variety of semi-automated processes, known as dynamic publishing—layout, multi-format conversion, distribution, rights management, file transfer, translation workflows, document updates, payments and reading metrics.

3.2.3 Innovations vs. Comparable Open Source Initiatives

Our project is defined by these features: it’s service is not a platform or markup language, and it works via an API so it is more like a feature set to add to other platforms. Points of comparison with other platforms include an interactive validator for platform independent, transmedia document for multi-format publishing; multi-format conversion (by connecting to existing A-machine, multi-format transformation engine); and multi-format design template generation and use. Neither current platforms (Fiduswriter, Booktype, Readthedocs, Sharelatex) nor markups and converters (LaTeX, Pandoc, or the python documentation generator Sphinx) meet all three criteria. The project offers a new publication GUI for transmedia OA and OER content, interactive inline validation GUI, web real-time updating GUI Javascript Web technologies, and design template libraries for multi-format layouts.

3.2.4 Software Development: Design Research, Working with the Technology Stack

The software development model establishes a three stage process of ‘Design Research’ for the working group. Firstly, research involves close development with the external open source expert contractors, secondly, testing; thirdly, the running of prototype publication productions as an advanced form of testing. Using this technology stack (validator, assets management and API) provides the design team with structured content and a development environment on which they can then experiment, test, and build templated design layouts for new types of publications. The API model has been chosen as it allows integration with existing open source platforms and encourages user adoption by minimizing workflow disruption caused by platform changes or reskilling.

3.2.5 Validation for Document and Publication Structuring

This task creates a validation system for documents and publication files and asset structuring to have semantic information standardized for multi-format digital publishing, including layout, document structure, output format specific settings, and meta-descriptive information. To make a single source structured document, the validation system gives interactive feedback directly via a GUI (to prompt users to make decisions related to structural markup) or via an API. Interactive Validator
A validator system ensures that the publication workflow structures documents and publication files according to standards and markup requirements that suit single source publishing workflows. As a result of a successful validation process, documents are machine readable and can be used in a variety of semi-automated publishing workflow processes. The validator has to cover different types of structuring that can be customized, including: typesetting (such as headers, bold, superscript, citation styles, line spacing after paragraphs, etc.); semantic document or publication structuring (picture credits, inline quotes, footnotes, pagination, chapters, and section, etc.); media inclusions (images, audio, video, equations, game sequence, etc.); document and publication file type and publication-ready outputs (PRO)-specific settings (image color profiles, print settings, job description files (JDFs), image resolutions, media queries, custom user output format design or content settings); multi-format document settings (i.e. instructions for conversion of a single source document into a specific format that involves information loss, such as a plain text file); a limited set of descriptive metadata (revision history, author, title, date, etc.).

The validator is also required to be interactive with a GUI and to work in an API environment. The interactive feedback is needed because a purely automated validation and structuring process would not be able to make the correct decisions and the files would end up with incorrect formatting. Instead, with an interactive system for user feedback, the user can intervene and make decisions when prompted to do so. Example cases where user feedback is needed divides into two categories. Firstly, workflow processing errors, for example invisible hyperlinks sitting in a document, which can be flagged up for removal in the way that a word processor’s ‘Navigator’ function allows a user to get an overview of a document. Secondly, common user errors, including not correctly marking up unordered lists, or using empty paragraph breaks rather than paragraph styles to make space between paragraphs. Examples of Interactive Structuring Systems
The type of interactive validator we will be developing will be for inline interaction within a web based word processing authoring environment. Here is an example from the commercial product for structured writing called ‘EasyDITA’: Other types of interactive validation systems produce a report, for the user to respond to, by editing their document and then resubmitting for further validation. Examples are the International Digital Publishing Forum (IDPF) EPUB Validator ( and W3C HTML Markup Validation Service (

Important features of such systems include differentiating between a document and publication, where different structural and metadata requirements are needed. The validation markup and rule set options are: 1. markup set updatable and customizable; 2. heuristics rule sets; 3. centralized or shared markup profiles. Users interact with the validator via the following: 1. GUI; 2. API; 3. command line.
Requirements, Specification and Standards for the Validator. A sample of categories for ‘validation’ and ‘single source’ file recipes includes: Validator Schematic


3.3 Publication Asset Storage

The publication asset management component of the technology stack will use a MongoDB-based NoSQL infrastructure, which will provide a highly scalable and proven platform for heterogeneous file types together with multiple application programming language bindings. Shared management facilities and efficient use of potentially large numbers of distributed virtual machines makes MongoDB ideal for prototyping and refining a very flexible asset management system. The partners have significant experience in developing loadable javascript-based environments for GUI workflow development with MongoDB, and experience supporting them on major projects with Heidelberg, Princeton, and Westminster University since 2011, as well as providing asset management for several commercial publishing applications. Avoiding off-the-shelf solutions such as Drupal or Wordpress will reduce the project's vulnerability to third party support requirements, will simplify Open Source licensing, and will improve overall sustainability of the framework produced.

An important aspect of the work already conducted by the partners on automated transformation of digital collections is the ability to create and tune metadata exports to multiple XML standards representations, including both MODS and VRA Core4. In addition, a number of workflows already exist to parse asset structures and automate the creation of MongoDB data structures. Together, these sub-components will provide the project with a mature environment for distributed teams of contributors, allowing them to build fully-searchable digital collections of publications' components and create Library of Congress-compliant metadata. The resulting structure can remain as BSON in MongoDB or be exported into other ecosystems such as Tamboti (which uses the native XML eXistdb). Additionally, MongoDB's multiple application programming language bindings enable it to be interfaced rapidly with other components such as Zotero, and allow structured APIs to be implemented for additional project-specific elements of the technology stack.

These tools will be used to build sub-components of the technology stack with the following functionality:

3.4 Transmedia Publishing API

An Application Program Interface (API) is the way in which our systems’ modular components can communicate with other systems on the internet securely. This means that the functionality we are researching and developing (validation, publication asset structuring, templates) can be integrated into these other systems that we are connecting to. The proposed API allows access to internal modules (Validator, Assets, Design Template Library, and Multi-format Transformer) as well as to external services. Additionally, the use of the API allows the project to use external functionality. For example, link to the software ecosystem called A-machine, with its multi-format transformation software, currently developed by the Hybrid Publishing Consortium. Using the API model of service provision best suits the introduction of solutions to common problems in the Open Access workflow. The API model is one where external features and services like citation management platforms like Zotero and university library distributors such as EBSCO can be integrated. Another benefit of the API model for service distribution is that users can stay within their familiar writing environments. This has the added benefit of encouraging adoption of new services like ours. For these major reasons we see using an API as being the best way to benefit the Open Access community.

3.4.1 Functionality Areas and Use Cases for the API

The API will have four functionality areas:
  1. Validation rule set. An external document editing system will be able to have our rule set applied to its documents to raise appropriate flags when a questionable content section is encountered (and to have these flags passed to other systems, and, if appropriate, allow changes and edits to be made to the remote document). The result would be that remote documents could be structured for multi-format conversion.
  2. Assets and asset structuring meta description framework. The effect here is that remote systems could store their content on our system, then make use of that content in different ways to, for example, extract all citations or images and captions from an archive and then extract and use these components on a granular level.
  3. Templates for authoring and using templates. Templates could be added to the system for use in making multi-format publications using the transformer engine.
  4. Transformer. A multi-format conversion engine. After a set of content has been approved by the validator, it could then be sent to the transformer with instructions to use a specific template and be outputted to a number of formats (EPUB, responsive HTML, HTML Book-in-browser, PDF, PDF for print-on-demand [PoD] etc).

The API is used inThe system API can connect to the following types of systems: content management systems (CMSs) like Wordpress or Drupal; OA and OER repositories with a digital asset management (DAMS) functionality; any online web based text editor or word processor, for example Fiduswriter software; template repositories such as CDN Bootstrap ( or multi-format format and distribution channels such as the open source app formatting software PugPig (; third party content services, such as citation management, picture and video repositories such as Zotero, Tamboti or Pandora. The use cases for the API can be varied: book publishing, publication repository, publishing from archives, legacy batch conversion, journal submission processes, web and peer review journals [Open Journal System (OJS)].


3.5 Technical Methodology

At the project’s core, we would use co-creation and design research methods to establish the software requirement to address the two central research concerns—transmedia referencing and citation, and document portability. Our research approach includes the following elements:

3.5.1 Comparative Software Tools Assessment

A comparative assessment of available approaches and open source tools, including:

3.5.2 Standards and Formats

For documents OASIS, W3C and IDPF open formats are used. Primarily this will be HTML5 and EPUB3, with attention to emerging EDUPUB W3C and IDPF initiatives announced in a January 2014 W3C call with contributions from publishers Pearson and O’Reilly. For citation and referencing purposes this would make use of a combination of Dublin Core (DC), Open Archive Initiative (OAI), and open Citation Style Language (CSL). Additionally, we would use a set of collections and bibliographic management meta description frameworks, Visual Resources Association (VRA) Core 4.0 and Metadata Object Description Schema (MODS) from the US Library of Congress and Text Encoding Initiative (TEI) for text.

3.5.3 Hardware and Software

Software used in the project will be MongoDB, Ruby, Transpect from le-tex, Scalar, A-machine framework, Linux, NodeJS, Linux, GitHub. Book scanner from the Public Library project

3.5.4 Data Acquisition, Processing, Analysis and Use

Core data for the project would come from the ‘Typemotion’ publication and exhibition. Other data would come from partners as well as from stakeholders. Data analysis would be carried out in consultation with industry stakeholders such as Westminster Data Futures project, HPC, Fiduswriter, and le-tex.

3.6 Technical Support and Relevant Experience

Core technical lead comes from the Westminster Data Futures project, HPC, Sourcefabric, Fiduswriter, and le-tex.


3.7 Preservation, Sustainability and Use

3.7.1 Preserving Data

Data Futures specializes in long term data preservation. Data and code would be stored on the Data Futures’ distributed and secure long term digital preservation network.

3.7.2 Ensuring Continued Access and Use of Digital Assets

Published material and software code would be published under open licenses. With published materials being in Green Open Access repositories covered by CLOCKSS (Controlled LOCKSS) support. Code would be published on the open repository GitHub and registered with the UK and EU Open Source research repositories. For example:

This page has paths:

This page references: