https://github.com/idiocc/idio
@Goa/Koa Web Server Bundled With Essential Middleware, SSR And Router.
https://github.com/idiocc/idio
Last synced: 7 months ago
JSON representation
@Goa/Koa Web Server Bundled With Essential Middleware, SSR And Router.
- Host: GitHub
- URL: https://github.com/idiocc/idio
- Owner: idiocc
- License: agpl-3.0
- Created: 2019-05-12T15:06:49.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2020-03-12T08:35:55.000Z (over 6 years ago)
- Last Synced: 2025-10-21T19:59:17.821Z (8 months ago)
- Language: JavaScript
- Homepage: https://www.idio.cc
- Size: 4.97 MB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# @idio/idio
[](https://www.npmjs.com/package/@idio/idio)
`@idio/idio` contains Koa's fork called Goa — web server compiled with _Closure Compiler_ so that its source code is optimised and contains only 1 external dependency (`mime-db`). Idio adds essential middleware to Goa for session, static files, CORS and compression and includes the router. As the project grows, more middleware will be added and optimised.
This is a production-ready server that puts all components together for the ease of use, while providing great developer experience using JSDoc annotations for auto-completions. _Idio_ is not a framework, but a library that enables **idiomatic** usage and compilation of the server and middleware.
```console
idio~:$ \
yarn add @idio/idio
npm install @idio/idio
```
## Example Apps
There are some example apps that you can look at.
1. [File Upload](https://github.com/art-deco/file-upload.artdeco.app): a front-end + back-end application for uploading photos. [Demo](https://file-upload.artdeco.app/) requires GitHub authorisation without any scope permissions to enable session middleware showcase.
1. [Akashic.Page](https://github.com/art-deco/akashic.page): a service for managing email and web-push subscriptions, with JS widgets and Mongo database connection.
## Table Of Contents
- [Example Apps](#example-apps)
- [Table Of Contents](#table-of-contents)
- [API](#api)
- [`async idio(middlewareConfig=, config=): !Idio`](#async-idiomiddlewareconfig-middlewareconfigconfig-config-idio)
* [`MiddlewareConfig`](#type-middlewareconfig)
* [`Config`](#type-config)
* [`Idio`](#type-idio)
* [ConfiguredMiddleware](#type-configuredmiddleware)
- [Middleware](#middleware)
* [Static](#static)
* [Session](#session)
* [CORS](#cors)
* [Compression](#compression)
* [File Upload](#file-upload)
* [Front End](#front-end)
- [Additional Middleware](#additional-middleware)
- [Custom Middleware](#custom-middleware)
- [Router Set-up](#router-set-up)
- [SSR](#ssr)
- [NeoLuddite.Dev](#neoludditedev)
* [`NeoLudditeOptions`](#type-neoludditeoptions)
- [WebSocket](#websocket)
- [Copyright & License](#copyright--license)
## API
The package is available by importing its default function and named components:
```js
import idio, { Keygrip, Router } from '@idio/idio'
```
## async idio(
`middlewareConfig=: !MiddlewareConfig,`
`config=: !Config,`
): !Idio
Start the server. Sets the `proxy` property to `true` when the NODE_ENV is equal to _production_.
- middlewareConfig !MiddlewareConfig (optional): The middleware configuration for the `idio` server.
- config !Config (optional): The server configuration object.
The app can be stopped with an async `.destroy` method implemented on it that closes all connections.
There are multiple items for middleware configuration:
__`MiddlewareConfig` extends FnMiddlewareConfig__: Middleware configuration for the `idio` server.
| Name | Type | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| static | (!StaticOptions \| !Array<!StaticOptions>) | _Static_ middleware options. |
| compress | (boolean \| [!CompressOptions](https://github.com/idiocc/idio/wiki/Compression#type-compressoptions)) | _Compression_ middleware options. |
| session | !SessionOptions | _Session_ middleware options. |
| cors | [!CorsOptions](https://github.com/idiocc/idio/wiki/Cors#type-corsoptions) | _CORS_ middleware options. |
| form | !FormDataOptions | _Form Data_ middleware options for receiving file uploads and form submissions. |
| frontend | !FrontEndOptions | _Front End_ middleware allows to serve source code from `node_modules` and transpile JSX. |
| neoluddite | !NeoLudditeOptions | Records the usage of middleware to compensate their developers' intellectual work. |
| csrfCheck | !CsrfCheckOptions | Enables the check for the presence of session with `csrf` property, and whether it matches the token from either `ctx.request.body` or `ctx.query`. |
| github | (!GitHubOptions \| !Array<!GitHubOptions>) | Sets up a route for GitHub OAuth authentication. The returned middleware will be installed on the `app` automatically so it doesn't need to be passed to the router. |
| jsonErrors | (boolean \| !JSONErrorsOptions \| !Array<!JSONErrorsOptions>) | Tries all downstream middleware, and if an error was caught, serves a JSON response with `error` and `stack` properties (only if `exposeStack` is set to true). Client errors with status code _4xx_ (or that start with `!`) will have full message, but server errors with status code _5xx_ will only be served as `{ error: 'internal server error '}` and the app will emit an error via `app.emit('error')` so that it's logged. |
| jsonBody | (boolean \| !JSONBodyOptions) | Allows to parse incoming JSON request and store the result in `ctx.request.body`. Throws 400 when the request cannot be parsed. |
| logarithm | !LogarithmOptions | Options to record hits in _ElasticSearch_. |
The types for starting the server include the address, port and router configuration.
__`Config`__: Server configuration object.
| Name | Type | Description | Default |
| ------ | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------- |
| port | number | The port on which to start the server. | `5000` |
| host | string | The host on which to listen. | `0.0.0.0` |
| router | !_goa.RouterConfig | The configuration for the router. | - |
After the app is started, it can be accessed from the return type.
__`Idio`__: The return type of the idio.
| Name | Type | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| __url*__ | string | The URL on which the server was started, such as `http://localhost:5000`. |
| __server*__ | 
!http.Server | The server instance. |
| __app*__ | !Application | The Goa application instance (with additional `.destroy` method). |
| __middleware*__ | !ConfiguredMiddleware | An object with configured middleware functions, which can be installed manually using `app.use`, or `router.use`. The context will be a standard Goa context with certain properties set by bundled middleware such as `.session`. |
| __router*__ | !Router | The router instance. |
All middleware can be accessed from the `middleware` property, so that it can be installed on individual basis on specific routes, if it's not used app-wise.
ConfiguredMiddleware extends MiddlewareObject: Idio-specific properties of the middleware object.
| Name | Type | Description |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| form | !_multipart.FormData | An instance of the form data class that can be used to create middleware. |
| session | !Middleware | The session middleware to be installed on individual routes. |
| frontend | !Middleware | The frontend middleware. |
| csrfCheck | !Middleware | Configured CSRF check middleware. |
| jsonErrors | (!Middleware \| !Array<!Middleware>) | Middleware to server errors as JSON. |
The example below starts a simple server with session and custom middleware, which is installed (used) automatically because it's defined as a function.
SourceOutput
```js
const { url, app,
middleware: { session, form },
router,
} = await idio({
// Developers' payment scheme neoluddite.dev
neoluddite: {
env: process.env.NODE_ENV,
key: '0799b7f0-d2c7-4903-a531-00c8092c2911',
app: 'idio.example',
},
// Idio's bundled middleware.
session: {
algorithm: 'sha512',
keys: ['hello', 'world'],
prefix: 'example-',
},
static: {
use: true,
root: 'upload',
},
form: {
config: {
dest: 'upload',
},
},
// Any middleware function to be use app-wise.
async middleware(ctx, next) {
console.log('//', ctx.method, ctx.path)
await next()
},
})
app.use(router.routes())
router.get('/', session, (ctx) => {
ctx.body = 'hello world'
})
router.post('/upload', session, async (ctx, next) => {
if (!ctx.session.user) {
ctx.status = 403
ctx.body = 'you must sign in to upload'
return
}
await next()
}, form.single('/upload'), (ctx) => {
// db.create({
// user: ctx.session.id,
// file: ctx.req.file.path,
// })
ctx.body = 'Thanks for the upload. Link: ' +
`${url}/${ctx.file.filename}`
})
```
```
http://localhost:5000
// GET /
hello world
```
## Middleware
Idio's advantage is that is has the essential middleware, that was compiled together with the server, so that the packages are reused and memory footprint is low.
### Static
🗂 [Explore Static Middleware Configuration](../../wiki/Static)
Used to serve static files, such as stylesheets, images, videos, html and everything else. Will perform mime-type lookup to serve the correct content-type in the returned header.
```js
const { url, app } = await idio({
static: {
root: 'example', use: true,
},
```
```js
// or multiple locations
static: [{
root: ['example'], use: true,
}, {
root: ['wiki'], use: true,
}],
}, { port: null })
```
```css
/** http://localhost:57537/app.css */
body {
font-size: larger;
}
```
Show Response Headers
```http
Content-Length: 29
Last-Modified: Thu, 18 Jul 2019 14:34:31 GMT
Cache-Control: max-age=0
Content-Type: text/css; charset=utf-8
Date: Thu, 05 Mar 2020 13:30:57 GMT
Connection: close
```
```http
Content-Length: 114
Last-Modified: Sat, 28 Dec 2019 18:07:31 GMT
Cache-Control: max-age=0
Content-Type: image/svg+xml
Date: Thu, 05 Mar 2020 13:30:59 GMT
Connection: close
```
### Session
👳♂️[Explore Session Middleware Configuration](../../wiki/Session)
Allows to store data in the `.session` property of the context. The session is serialised and placed in cookies. When the request contains the cookie, the session will be restored and validated (if signed) against the key.
```js
const { url, app } = await idio({
session: { use: true, keys:
['hello', 'world'], algorithm: 'sha512' },
async middleware(ctx, next) {
if (ctx.session.user)
ctx.body = 'welcome back '
+ ctx.session.user
else {
ctx.session.user = 'u'
+ (Math.random() * 1000).toFixed(1)
ctx.body = 'hello new user'
}
await next()
},
})
```
The session data is encrypted with base64 and signed by default, unless the .signed option is set to false. Signing means that the signature will contain the hash which will be validated server-side, to ensure that the session data was not modified by the client. The default algorithm for signing is sha1, but it can be easily changed to a more secure sha512.
```js
// GET /
"hello new user"
/* set-cookie */
[
{
name: 'koa:sess',
value: 'eyJ1c2VyIjoidTg2LjciLCJfZXhwaXJlIjoxNTgzNTAxNDU5ODQzLCJfbWF4QWdlIjo4NjQwMDAwMH0=',
path: '/',
expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
httponly: true
},
{
name: 'koa:sess.sig',
value: '5hRueSOyLuhp6nZvOi4TcziXNiADlaIhE6fJHruR-I8cGtEVDYCgNe9t3LS0SyV-SEN1kPa8ZwIz-a91GWPw-A',
path: '/',
expires: 'Fri, 06 Mar 2020 13:30:59 GMT',
httponly: true
}
]
// GET /
"welcome back u86.7"
```
### CORS
👮♀️[Explore CORS Middleware Configuration](../../wiki/Cors)
To enable dynamic communication between clients and the server via JavaScript requests from the browser, the server must respond with `Access-Control-Allow-Origin` header that sets the appropriate allowed _Origin_. This middleware is easy to use on production and development environments.
CORS sourceThe Output
```js
const { NODE_ENV } = process.env
const { url, app } = await idio({
async example(ctx, next) {
console.log('//', ctx.method,
ctx.path, 'from', ctx.get('Origin'))
ctx.body = 'hello world'
await next()
},
cors: {
use: true,
origin: NODE_ENV == 'production' ?
'http://prod.com' : null,
allowMethods: ['GET', 'POST'],
},
})
```
```js
// GET / from https://3rd.party
{
vary: 'Origin',
'access-control-allow-origin': 'http://prod.com',
date: 'Thu, 05 Mar 2020 13:31:01 GMT',
connection: 'close'
}
// GET / from http://prod.com
{
vary: 'Origin',
'access-control-allow-origin': 'http://prod.com',
date: 'Thu, 05 Mar 2020 13:31:01 GMT',
connection: 'close'
}
// OPTIONS / from http://prod.com
{
vary: 'Origin',
'access-control-allow-origin': 'http://prod.com',
'access-control-allow-methods': 'GET,POST',
date: 'Thu, 05 Mar 2020 13:31:01 GMT',
connection: 'close'
}
```
### Compression
🗜[Explore Compression Middleware Configuration](../../wiki/Compression)
When the body of the response is non-empty, it can be compressed using `gzip` algorithm. This allows to save data transmitted over the network. The default threshold is `1024` bytes, since below that the benefits of compression are lost as the compressed response might end up being even larger.
Compression sourceThe Output
```js
const { url, app } = await idio({
async serve(ctx, next) {
console.log('//',
ctx.method, ctx.path)
ctx.body = packageJson
await next()
},
compress: {
use: true,
},
})
```
```js
// GET /
{
'content-type': 'application/json; charset=utf-8',
vary: 'Accept-Encoding',
'content-encoding': 'gzip',
date: 'Thu, 05 Mar 2020 13:31:01 GMT',
connection: 'close',
'transfer-encoding': 'chunked'
}
```
### File Upload
🖼[Explore Form Data Middleware Configuration](../../wiki/Form-Data)
Browser will submit forms and send files using `multipart/form-data` type of request. It will put all fields of the form together and stream them to the server, sending pairs of keys/values as well as files when they were attached. The _Form Data_ middleware is the **[Multer](https://github.com/expressjs/multer)** middleware specifically rewritten for Koa that can handle file uploads.
File Upload sourceThe Output
```js
const { url, app, router, middleware: {
form,
} } = await idio({
form: {
dest: 'example/upload',
},
})
app.use(router.routes())
router.post('/example',
form.single('bio'),
(ctx) => {
delete ctx.file.stream
ctx.body = { file: ctx.file,
body: ctx.request.body }
}
)
```
```js
{
file: {
fieldname: 'bio',
originalname: 'bio.txt',
encoding: '7bit',
mimetype: 'application/octet-stream',
destination: 'example/upload',
filename: '106e5',
path: 'example/upload/106e5',
size: 29
},
body: { hello: 'world' }
}
```
### Front End
🌐[Explore Front End Middleware Configuration](../../wiki/Front-End)
Web applications are always full stack and involve both back-end together with front-end. Whereas all previously described middleware was for the server only, the front-end middleware facilitates browser development, as it allows to serve source code from the `node_modules` directory and transpile JSX. Modern browsers support modules, but JavaScript needs to be patched to rename imports like
```js
// was
import X from 'package-name'
// becomes
import X from '/node_modules/package-name/src/index.mjs'
```
This is achieved by resolving the `module` field from `package.json` of served packages (with fallback to the `main` field, but in that case `require` statements will not work).
```js
const { url, app } = await idio({
frontend: {
use: true,
directory: 'example/frontend',
},
})
```
```jsx
import { render, Component } from 'preact'
class MyComp extends Component {
render() {
return (
Hello World!
)
}
}
render(MyComp, document.body)
```
Using the simple configuration from above, and a JSX file, the browser will receive the following patched source code. The middleware will also look for requests that start with the /node_modules path, and serve them also. The pragma (import { h } from 'preact') is also added automatically, but it can be configured.
```js
import { h } from '/node_modules/preact/dist/preact.mjs'
import { render, Component } from '/node_modules/preact/dist/preact.mjs'
class MyComp extends Component {
render() {
return (h('div',{className:"example"},
`Hello World!`
))
}
}
render(MyComp, document.body)
```
The idea here is to provide a basic mechanism to serve front-end JavaScript code, without inventing any module systems, adapting to _CommonJS_, or transpiling old features. We simply want to execute our modern code and browsers are more than capable to do that, without us having to run complex build systems on the development code. Our simple JSX parser is not rocket science either and works perfectly well without building ASTs (but check for minor limitations in Wiki).
## Additional Middleware
There are some small bits of middleware that can be used in server as well, but which are not essential to its functioning. They are listed in 📖 [Wiki](../../wiki/Additional_Middleware).
- `csrfCheck`: Ensures that the `csrf` token from session matches one in the request.
- `jsonErrors`: Allows to serve errors as _JSON_, which is useful for APIs.
- `jsonBody`: Parses requests with the `application/json` content type into `ctx.request.body`.
- `logarithm`: Record hits in _ElasticSearch_.
- `github`: Sets up _GitHub_ OAuth routes.
## Custom Middleware
When required to add any other middleware in the application not included in the _Idio_ bundle, it can be done in several ways.
1. Passing the middleware function as part of the _MiddlewareConfig_. It will be automatically installed to be used by the _Application_. All middleware will be installed in order it is found in the _MiddlewareConfig_.
```js
import idio from '@idio/idio'
const APIServer = async (port) => {
const { url } = await idio({
// 1. Add logging middleware.
async log(ctx, next) {
await next()
console.log(' --> API: %s %s %s', ctx.method, ctx.url, ctx.status)
},
// 2. Add always used error middleware.
async error(ctx, next) {
try {
await next()
} catch (err) {
ctx.status = 403
ctx.body = err.message
}
},
// 3. Add validation middleware.
async validateKey(ctx, next) {
if (ctx.query.key !== 'app-secret')
throw new Error('Wrong API key.')
ctx.body = 'ok'
await next()
},
}, { port })
return url
}
export default APIServer
```
```
Started API server at: http://localhost:5005
--> API: GET / 403
--> API: GET /?key=app-secret 200
```
2. Passing a configuration object as part of the _MiddlewareConfig_ that includes the `middlewareConstructor` property which will receive the reference to the `app`. Other properties such as `conf` and `use` will be used in the same way as when setting up bundled middleware: setting `use` to `true` will result in the middleware being used for every request, and the `config` will be passed to the constructor.
```js
import rqt from 'rqt'
import idio from '@idio/idio'
import APIServer from './api-server'
const ProxyServer = async (port) => {
// 1. Start the API server.
const API = await APIServer(5001)
console.log('API server started at %s', API)
// 2. Start a proxy server to the API.
const { url } = await idio({
async log(ctx, next) {
await next()
console.log(' --> Proxy: %s %s %s', ctx.method, ctx.url, ctx.status)
},
api: {
use: true,
async middlewareConstructor(app, config) {
// e.g., read from a virtual network
app.context.SECRET = await Promise.resolve('app-secret')
/** @type {import('@typedefs/goa').Middleware} */
const fn = async (ctx, next) => {
const { path } = ctx
const res = await rqt(`${config.API}${path}?key=${ctx.SECRET}`)
ctx.body = res
await next()
}
return fn
},
config: {
API,
},
},
}, { port })
return url
}
```
```
API server started at http://localhost:5001
Proxy started at http://localhost:5002
--> API: GET /?key=app-secret 200
--> Proxy: GET / 200
```
## Router Set-up
After the _Application_ and _Router_ instances are obtained after starting the server as the `app` and `router` properties of the returned object, the router can be configured to respond to custom paths. This can be done by assigning configured middleware from the map and standalone middleware, and calling the `use` method on the _Application_ instance.
```js
import { collect } from 'catchment'
import idio from '@idio/idio'
const Server = async () => {
const {
url, router, app, middleware: { pre, post, bodyparser },
} = await idio({
// 1. Configure middlewares via middlewareConstructor without installing them.
pre: {
middlewareConstructor() {
return async function(ctx, next) {
console.log(' <-- %s %s',
ctx.request.method,
ctx.request.path,
)
await next()
}
},
},
post: {
middlewareConstructor() {
return async function(ctx, next) {
console.log(' --> %s %s %s',
ctx.request.method,
ctx.request.path,
ctx.response.status,
)
await next()
}
},
},
bodyparser: {
middlewareConstructor() {
return async (ctx, next) => {
let body = await collect(ctx.req)
if (ctx.is('application/json')) {
body = JSON.parse(body)
}
ctx.request.body = body
await next()
}
},
},
}, { port: 5003 })
// 2. Setup router with the bodyparser and path-specific middleware.
router.post('/example',
pre,
bodyparser,
async (ctx, next) => {
ctx.body = {
ok: true,
request: ctx.request.body,
}
await next()
},
post,
)
app.use(router.routes())
return url
}
```
LoggingResponse
```
Page available at: http://localhost:5003
<-- POST /example
--> POST /example 200
```
```js
// server response:
{ ok: true, request: { hello: 'world' } }
```
Also checkout the [_Router_ package](https://github.com/idiocc/router) that allows to automatically initialise routes from a given directory, and watch for changes in them during development. This means you don't have to refresh the server manually after a change to a route.
```js
const w = await initRoutes(router, 'routes', {
middleware,
})
if (process.env.NODE_ENV == 'prod') watchRoutes(w)
```
## SSR
_Idio_ supports Server-Side rendering of JSX components ([same restrictions](https://github.com/idiocc/idio/wiki/Front-End#todo--jsx-limitations) apply as for front-end). You can easily mark up your back-end pages using full-scale HTML, or basic placeholders in which you can then render your front-end app.
```jsx
import idio, { render } from '@idio/idio'
const { url, app, router } = await idio()
router.get('/', (ctx) => {
ctx.body = render(
Example
Hello World!
, {
addDoctype: true,
pretty: true,
})
})
app.use(router.routes())
```
```html
Example
Hello World!
```
## NeoLuddite.Dev
This web server integrates with [NeoLuddite](https://neoluddite.dev): the package monetary reward scheme. It's currently in beta, and this section will be relevant when it's open to the public.
Every time you invoke certain functionality in a package somebody has written (e.g., `koa-static` for static files, `koa-session` for creation of session), via _Idio_, your usage will be counted and your balance in _Ludds_ on the neoluddite server will be transferred to the software engineer as a reward for his/her intellectual work. Contact license@neoluddite.dev for any requests.
```js
const { url, app,
middleware: { session, form },
router,
} = await idio({
// Developers' payment scheme neoluddite.dev
neoluddite: {
env: process.env.NODE_ENV,
key: '0799b7f0-d2c7-4903-a541-10d8092c2911',
app: 'idio.example',
},
// ...
}
```
The usage will be billed for apps running in production mode, therefore the `env` variable is needed. Setting the `app` has no effect but allows to break down statistics by web application on the portal. See the license section for more info.
__`NeoLudditeOptions`__: Options for the neoluddite.dev client.
| Name | Type | Description | Default |
| -------- | --------------- | -------------------------------------------------------------------------------------------------------- | ------------------------ |
| __key*__ | string | The API key received from the app. | - |
| env | string | The environment (e.g., `dev`/`staging`). The production env must be indicated as `prod` which is billed. | - |
| host | string | The hostname of the server. | `https://neoluddite.dev` |
| app | string | The name of the application. | - |
## WebSocket
We've implemented a library to upgrade requests into _WebSocket_ connections. You can read more at the actual [package page](https://github.com/idiocc/websocket). _Idio_ simply exports this method via its API. You need to configure it yourself.
```js
import idio, { websocket } from '@idio/idio'
const { url, app, server } = await idio()
// clients stores current connections against ID
const clients = websocket(server, {
onConnect(clientId) {
// the value is a function to send messages
clients[clientId]('intro-event', 'Hello Client!')
},
})
```
## Copyright & License
GNU Affero General Public License v3.0
Affero GPL means that you're not allowed to use this web server on the web unless you release the source code for your application. This is a restrictive license which has the purpose of defending Open Source work and its creators.
To be able to use the server, just set up a monthly payment on [Open Collective](https://opencollective.com/nodetools) for any amount of your choice.
All original work on middleware and _Koa_ are under MIT license. See [Goa Page](https://github.com/idiocc/goa/) for the list of packages and modules used in compilation of the Goa server, and the [`package.json`](/package.json) file for dependencies of this project (todo: create wiki page w/ licenses table).
