Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/fdawgs/docsmith

RESTful API for converting clinical documents and files
https://github.com/fdawgs/docsmith

doc docker documents docx fastify nhs nodejs pdf pm2 rest restful rtf

Last synced: 15 days ago
JSON representation

RESTful API for converting clinical documents and files

Host: GitHub
URL: https://github.com/fdawgs/docsmith
Owner: Fdawgs
License: mit
Created: 2021-01-14T13:08:25.000Z (about 4 years ago)
Default Branch: main
Last Pushed: 2025-01-01T03:58:41.000Z (21 days ago)
Last Synced: 2025-01-01T04:26:04.536Z (21 days ago)
Topics: doc, docker, documents, docx, fastify, nhs, nodejs, pdf, pm2, rest, restful, rtf
Language: Rich Text Format
Homepage:
Size: 24.7 MB
Stars: 20
Watchers: 2
Forks: 2
Open Issues: 10
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE

Awesome Lists containing this project

README

        

# Docsmith

[![GitHub release](https://img.shields.io/github/release/Fdawgs/docsmith.svg)](https://github.com/Fdawgs/docsmith/releases/latest/)

[![CI](https://github.com/Fdawgs/docsmith/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/Fdawgs/docsmith/actions/workflows/ci.yml)

[![Coverage status](https://coveralls.io/repos/github/Fdawgs/docsmith/badge.svg?branch=main)](https://coveralls.io/github/Fdawgs/docsmith?branch=main)

[![code style: Prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg?style=flat)](https://github.com/prettier/prettier)

> RESTful API for converting clinical documents and files

## Overview

Docsmith is a RESTful API, built using Node.js and the [Fastify](https://fastify.dev/) web framework, that can convert a range of files:

| Input | Output | Notes                                        |

| ----- | ------ | -------------------------------------------- |

| DOC   | TXT    | DOT file variant supported                   |

| DOCX  | HTML   | DOCM, DOTM, and DOTX file variants supported |

| DOCX  | TXT    | DOCM, DOTM, and DOTX file variants supported |

| HTML  | TXT    | XHTML file variant supported                 |

| PDF   | HTML   |                                              |

| PDF   | TXT    | Scanned documents supported using OCR        |

| RTF   | HTML   | Images are removed[^1]                       |

| RTF   | TXT    |                                              |

[^1]: The underlying UnRTF binary converts images and stores them using an incremental naming scheme of `pict001`, `pict002`, and so on. This poses a confidentiality and clinical risk as concurrent requests will overwrite each other's images, which could result in a patient's image being placed in another patient's document. To mitigate this, Docsmith removes all images from the output HTML

### Why Docsmith?

Docsmith was created in my spare time outside of work after identifying the need for an open-source document conversion service at Yeovil Hospital (ran by [Somerset NHS Foundation Trust](https://www.somersetft.nhs.uk/)).

Being open-source, with the ability to be self-hosted, enables a data processor (i.e. an NHS trust) to confirm that a service is not storing and logging files with confidential patient identifiable data (PID) in them, which is essential for preventing potential GDPR breaches. This is something that the majority of existing closed-source document conversion services cannot offer. Docsmith was built to remedy this.

Before Docsmith, Yeovil Hospital was using expensive proprietary conversion tools that would regularly produce unreadable documents with issues such as text running off the page, paragraphs overlapping each other, and Windows-1252 to UTF-8 character encoding problems. GP surgeries in Somerset and Dorset would receive these corrupted documents through [MESH](https://digital.nhs.uk/services/message-exchange-for-social-care-and-health-mesh) and be unable to read them. This resulted in time and money wasted either posting or faxing them again, opening up the potential for further data breaches.

Docsmith enables a data processor to use a comprehensive, GDPR-compliant, open-source document conversion service. In comparison with equivalents in the market today it completes this vital task at a fraction of the cost (free!), whilst also ensuring a higher level of security and privacy for the data subjects.

## Prerequisites

These are only required if running the API outside of Docker:

-   [Node.js](https://nodejs.org/en/) >=20.0.0

-   Linux only: `poppler-data` >=0.4.9

-   Linux only: `poppler-utils` >=20.12.0

-   macOS only: `poppler` >=20.12.0

-   Linux and macOS only: `unrtf` >=0.19.3

## Setup

Perform the following steps before deployment:

1. Download and extract the [latest release asset](https://github.com/Fdawgs/docsmith/releases/latest)

2. Navigate to the extracted directory

3. Make a copy of `.env.template` in the root directory and rename it to `.env`

4. Configure the application using the environment variables in `.env`

5. Place additional trained data into `ocr_lang_data` directory (optional, [info can be found here](./ocr_lang_data/README.md))

> **Note**

> Set the following environment variables in `.env` to meet NHS England's recommendation to retain six months' worth of logs:

>

> -   `LOG_ROTATION_DATE_FORMAT="YYYY-MM-DD"`

> -   `LOG_ROTATION_FREQUENCY="daily"`

> -   `LOG_ROTATION_MAX_LOGS="180d"`

## Deployment

### Standard deployment

1. Run `npm ci --ignore-scripts --omit=dev` to install dependencies

2. Run `npm start`

The service should be up and running on the port set in the config. Output similar to the following should appear in stdout or in the log file specified using the `LOG_ROTATION_FILENAME` environment variable:

```json

{

	"level": "info",

	"time": "2022-10-20T07:57:21.459Z",

	"pid": 148,

	"hostname": "MYCOMPUTER",

	"msg": "Server listening at http://127.0.0.1:51173"

}

```

To test it, use [Insomnia](https://insomnia.rest/) and import the example requests from `./test_resources/insomnia_test_requests.json`.

### Deploying using Docker

This requires [Docker](https://docker.com) installed.

1. Run `docker compose up` (or `docker compose up -d` to run in the background)

### Deploying using PM2

If this cannot be deployed into production using Docker, use a process manager such as [PM2](https://pm2.keymetrics.io/).

1. Run `npm ci --ignore-scripts --omit=dev` to install dependencies

2. Run `npm i -g pm2` to install pm2 globally

3. Launch the application with `pm2 start .pm2.config.js`

4. Check that the application has been deployed using `pm2 list` or `pm2 monit`

#### To install as a Windows service:

If using a Microsoft Windows OS utilise [pm2-installer](https://github.com/jessety/pm2-installer) to install PM2 as a Windows service.

> **Note**

> PM2 will automatically restart the application if `.env` is modified.

## Usage

### Accessing API documentation

API documentation can be found at `/docs`:



The underlying OpenAPI definitions are found at `/docs/openapi`.

## Contributing

Contributions are welcome, and any help is greatly appreciated!

See [the contributing guide](./CONTRIBUTING.md) for details on how to get started.

Please adhere to this project's [Code of Conduct](https://github.com/Fdawgs/.github/blob/main/CODE_OF_CONDUCT.md) when contributing.

## License

`docsmith` is licensed under the [MIT](./LICENSE) license.