Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/GeraldWodni/kern.js
web-framework for web-applications, also features multi-domain content management with redundancy elimination
https://github.com/GeraldWodni/kern.js
Last synced: about 1 month ago
JSON representation
web-framework for web-applications, also features multi-domain content management with redundancy elimination
- Host: GitHub
- URL: https://github.com/GeraldWodni/kern.js
- Owner: GeraldWodni
- License: gpl-3.0
- Created: 2014-06-05T13:55:35.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2024-05-29T12:15:46.000Z (4 months ago)
- Last Synced: 2024-06-21T14:28:39.014Z (3 months ago)
- Language: JavaScript
- Size: 1.7 MB
- Stars: 11
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-starred - GeraldWodni/kern.js - web-framework for web-applications, also features multi-domain content management with redundancy elimination (others)
README
kern.js
=======
Web-framework for web-applications, also features multi-domain content management with redundancy elimination .
## Under Construction
Currently *KERN ][* or *kern 2* is written in PHP (30k+ lines), this is a rewrite for node.js .
As I was often asked to make the PHP version open source, this new rewrite will be free software from the beginning,
if you are unhappy with GPLv3, write me an email, we can arrange a dual-license for your business case.
## Install
npm and bower are required to be installed globally. Then install the dependencies:
```
npm install
bower install
npm start
```
## tl;dr - gets you started in 10 minutes
Very simple example to get a website up and running after installation:
In the websites directory, create a folder named by a domain pointing to your server i.e. *example.com* .
Create a file *site.js* in the directory *websites/example.com* with the following content:
```
module.exports = {
setup: function( k ) {
k.router.get("/", function( req, res ) {
res.send("Is it really that simple?");
});
}
};
```
now run `npm start` and you should reach kern.js via browser getting "Is it really that simple" as plain response.
As you might be heavily dissatisfied by the format of this plain text let's add some markup.
Create a folder `websites/example.com/views`, and add the following `layout.jade` file:
```
doctype html
html
head
title kern.js - tl;dr
link(rel="stylesheet", type="text/css" href="/css/index.css")
body
h1 kern.js - tl;dr
p If I would read the other sections instead I'd be able to accomplish astonishing stuff with kern.js ;)
```
now adopt *site.js* to serve jade instead of plain text:
```
module.exports = {
setup: function( k ) {
k.router.get("/", function( req, res ) {
k.jade.render( req, res, "home" );
});
}
};
```
Still interested? Read on! :D
## Websites Hierarchy
Each directory in the websites-folder represents a website and should offer a valid domain name.
When files are required (server- or client-side), they are searched for in a hierarchical pattern.
For example when www.example.com/images/lol.png is requested, it is searched for in the following order:
- www.example.com/images/lol.png
- example.com/images/lol.png
- com/images/lol.png
- default/images/lol.png
The first match is sent as response to the client.
The hierarchy can be influenced by a site-config (see section "Site Config"), i.e. assume the following configuration *example.com/config.json*:
```
{
".*": {
"hierarchyUp": "wodni.at",
}
}
```
would yield the following search pattern:
- www.example.com/images/lol.png
- example.com/images/lol.png
- wodni.at/images/lol.png
- at/images/lol.png
- default/images/lol.png
This search order is also applied to site-modules, less-include and most other subsystems.
Overall the hierarchy allows you to eliminate lots of redundancy for your websites, and makes it easy to reuse existing functionality.
## site.js (each website's main script)
*TODO*
## Site Modules
*TODO*
## Administration Modules
Administration modules are site-modules who are protected and confined into to administrative section of a site.
They will only be accessible and shown in the navigation when a user's permission contain the module's link.
### Custom Administration module
You are advised to look at the default administration modules as they are employing a simpler API.
However if you want to maintain control of rendering and only use a basic crud-router, here is how the old user module did it:
```javascript
k.rdb.crud.router( k, ["/", "/edit/:id"], k.rdb.users, {
readFields: function ( req ) {
var fields = {
name: req.postman.username("name"),
permissions: req.postman.linkList("permissions")
};
var password = req.postman.password();
if( password && password.length > 0 )
{
if( !req.postman.fieldsMatch( "password", "password2" ) )
throw req.locales.__( "Passwords do not match" );
if( password.length < k.rdb.users.minPasswordLength )
throw req.locales.__( "Password too short, minimum: {0}" ).format( k.rdb.users.minPasswordLength );
fields.password = password;
}
return fields;
}
});
function renderAll( req, res, values ) {
k.rdb.users.readAll( req.kern.website, function( err, items ) {
if( err )
return next( err );
items.forEach( function( item ) {
item.escapedLink = encodeURIComponent( item.link );
});
k.renderJade( req, res, "admin/users", k.reg("admin").values( req, { messages: req.messages, items: items, values: values } ) );
});
}
k.router.get( "/edit/:link?", function( req, res ) {
k.rdb.users.read(req.kern.website, req.requestData.escapedLink( 'link' ), function( err, data ) {
if( err )
return next( err );
renderAll( req, res, data );
});
});
k.router.get( "/", function( req, res ) {
renderAll( req, res );
});
```
## Data
*TODO*
### data.js (website's main data-description)
Contains the data-model description for a website. Returns an object containing CRUDs, which can be received in any site-script by calling k.getData().
Example file:
```
module.exports = {
setup: function( k ) {
var connection = k.getDb();
var groups = k.crud.sql( connection, {
table: "groups",
foreignKeys: {
user: { crud: k.users, unPrefix: true }
}
} );
var devices = k.crud.sql( connection, {
selectListQuery: "SELECT `devices`.`id`, `devices`.`name`, `groups`.`name` AS `groupName` FROM `devices` INNER JOIN `groups` ON `groups`.`id`=`devices`.`group` ORDER BY `groupName`, `name`",
table: "devices",
foreignBoldName: 'groupName',
foreignKeys: {
group: { crud: groups }
}
});
return {
devices: devices,
groups: groups
};
}
}
```
### Database
*TODO*
### Redis
*TODO*
## postman, getman, requestman & filters
Input sanitization for:
- getman: GET parameters, parameters from the url-query string i.e. `/search?needle=foobar`
- postman: POST parameters (form-encoded POST)
- requestman: express-js url-parameters i.e. `'/api/items/get/:id'`
### getman & requestman
getman and requestman are called in plain sequence within a request i.e.:
```
k.router.get("/articles/:id", function( req, res, next ) {
k.requestman( req );
db.query( "SELECT * FROM articles WHERE id=?", [ req.requestman.uint('id') ], function( err, data ) {
if( err ) return next( err );
if( data.length == 0 )
return k.httpStatus( req, res, 404 );
k.jade.render( req, res, "article", data[0] );
});
});
```
### postman
postman on the other hand needs to wait for the request-body to be processed, so a callback needs to be specified:
```
k.router.post("/article/new", function( req, res, next ) {
k.postman( req, res, function() {
db.query( "INSERT INTO articles (time, title, text) VALUES( NOW(), ?, ? )",
[ req.postman.singleLine( 'title' ), req.postman.text( 'text' ) ],
function( err, result ) {
if( err ) return next( err );
res.json( { success: true, id: result.insertId } );
});
});
});
```
if postman receives 'application/json' as Content-Type, it replaces req.body with a parsed JSON object.
### filters
All three methods use the same sanitization filters.
Each filter is impleemented as a function which expects the only argument to be the name of the parameter.
If no parameter-name is passed, the function will assume its own function-name as parameter-name.
i.e. `req.postman.id()` is the same as calling `req.postman.id('id')`.
Here is a list of the currently implemented filters. Currently they are maintained for English and German. Feel free to submit your own language.
- address: `/[^-,.\/ a-zA-Z0-9äöüßÄÖÜ]/g`
- allocnum: `/[^a-zA-Z0-9äöüßÄÖÜ]/g`
- alpha: `/[^a-zA-Z]/g`
- alnum: `/[^a-zA-Z0-9]/g`
- alnumList: `/[^,a-zA-Z0-9]/g`
- boolean: `/[^01]/g`
- color: `/[^#a-fA-F0-9]/g`
- dateTime: `/[^-: 0-9]/g`
- decimal: `/[^-.,0-9]/g` and replace ',' with '.'
- email: `/[^-@+_.0-9a-zA-Z]/g`
- escapedLink:decodeURIComponent and `/[^-_.a-zA-Z0-9\/]/g`
- filename: `/[^-_.0-9a-zA-Z]/g`
- hex: `/[^-0-9a-f]/g`
- id: `/[^-_.:a-zA-Z0-9]/g`
- int: `/[^-0-9]/g`
- link: `/[^-_.a-zA-Z0-9\/]/g`
- linkItem: `/[^-_.a-zA-Z0-9]/g`
- linkList: `/[^-,_.a-zA-Z0-9]/g`
- password: no manipulation
- raw: no manipulation
- singleLine: `/[^-_\/ a-zA-Z0-9äöüßÄÖÜ]/g`
- telephone: `/[^-+ 0-9]/g`
- text: no manipulation
- uint: `/[^0-9]/g`
- url: `/[^-?#@&,+_.:\/a-zA-Z0-9]/g`
- username: `/[^-@_.a-zA-Z0-9]/g`
- renameFile: replace non-ascii chars by synonyms i.e. 'ä' becomes 'ae'
## Users & Sessions
*TODO*
### Session Cookies
Once a session is started, a cookie (default name: `kernSession`) is sent and updated in every request to extend it for another timeout(see `KERN_SESSION_TIMEOUT` period). The cookie options can be set using `sessionCookies`, defaults to: `{ sameSite: "strict" }`.
To allow returning from an external processing i.e. payment providers, an external cookie can be set using `req.sessionInterface.setExternalCookie( req, res )`.
After the external process has concluded, the cookie should be destroyed using `req.sessionInterface.destroyExternalCookie( req, res )`.
### Persistent Logins
Using a combination of [best practice (2006)](https://web.archive.org/web/20180819014446/http://jaspan.com/improved_persistent_login_cookie_best_practice) and an [updated strategy described here (2015)](https://paragonie.com/blog/2015/04/secure-authentication-php-with-long-term-persistence#title.2), the persistent login works as follows:
1. When the user successfully logs in with _Remember me_ checked, a `persistentLoginCookie` is issued in addition to the standard session cookie.
2. The `persistentLoginCookie` contains a series identifier and a token. The series and token are unguessable random numbers. They are stored in the persistentSessions table.
3. When a non-logged-in user visits the site and presents a `persistentLoginCookie`, the series and token are looked up in the database.
1. If the pair is present, the user is considered authenticated. The used token is replaced in the database and the cookie.
2. If the series is present but the token does not match, a theft is assumed. The user receives a strongly worded warning and all of the user's persistent logins are deleted.
3. If the series is not present, the `persistentLoginCookie` is ignored.
#### Cookie
The cookie consists of two random values which are SHA256 hashed:
- A `series` (64 chars long)
- A `token` (64 chars long)
- They are sepated by `-`.
Example: `5db1fee4b5703808c48078a76768b155b421b210c0761cd6a5d223f4d99f1eaa-b7d8d2de6ec20d8ac3b75ea3a9054b9f9079d5947c52dbb230356014934cccac`
#### Table
```sql
CREATE TABLE persistentLogins(
series VARCHAR(64) NOT NULL, -- unique series
hashedToken VARCHAR(64) NOT NULL, -- token value, but hashed
username VARCHAR(64) NOT NULL, -- associated user name
expires DATETIME NOT NULL, -- old persistentLogins are removed
PRIMARY KEY(series)
);
```
## Hooks
*TODO*
## Site Config
A website-directory can contain a config.json file.
Example file:
```
{
"ganymed": {
"hierarchyUp": "example.com",
"mysql": {
"user" : "username",
"database" : "development",
"password" : "asd",
"multipleStatements": true
},
"autoload": true
},
".*": {
"hierarchyUp": "example.com",
"mysql": {
"host" : "1.2.3.4",
"user" : "username",
"database" : "producation",
"password" : "asd",
"multipleStatements": true }
}
}
```
The root object's keys add as guards to the configuration objects. The keys are evaluated as regular expressions against the system's host name.
First matching key's object is used as configuration for the website (*Attention:* I am not very happy with this behavior, might be changed to extend all matching objects, or even respect other configurations found in the hierarchy's search order.
### Configuration Options
- **autoload** *boolean* executes the site's setup-function on kern startup if true
- **mysql** *object* specifies the mysql-connection, see [node-mysql](https://github.com/felixge/node-mysql/)
- **hierarchyUp** *string* used as next hierarchy-step in the search order, see *Website Hierarchy*
- **custom** *object* you can add other configuration values for your modules
### Enviroment Variables
Containers (i.e. in cloud environments) are usually configured using environment variables.
When making `kern.js` ready for kubernetes, the following variables were added. Please feel free to suggest additional variables and state a reason as well as a usecase.
- `KERN_STATIC_HOST`: Force the use of this `Hostname`, ignoring the HTTP-Header. Usefull if running as a service and meant to be accessed via a ClusterIP
- `KERN_LOAD_ONLY_HOSTS`: Comma separated list of websites that are loaded, others are skipped
- `KERN_STATIC_LOCALE`: Force the use of this `locale`, ignoring the HTTP-Header.
- `KERN_CLI_SECRET`: Secret used by the websync-container to access the `CLI` api. This is meant to read userdata and store it in the repo.
- `KERN_CLI_PORT`: While you are protected by a secret, why not randomize the `CLI` port. This is security by obscurity but makes it a bit harder for script kiddies. That being said: this port should never be exposed outside of your pod.
- `KERN_SESSION_TIMEOUT`: session timeout in seconds. Defaults to `3000` (50min).
- `KERN_AUTO_LOAD`: if set to `"false"`, autoLoad will be skipped for all websites
#### Database
If you have configured a database, you can insert stubs for the actual values and override them with the following environment variables.
This is usefull if you have access data stored in a kubernetes secret or the like. The names are the same as used by mysql and mariadb containers.
- `MYSQL_HOST`
- `MYSQL_DATABASE`
- `MYSQL_USER`
- `MYSQL_PASSWORD`
#### Static content
##### prefixCache
Generate previews of files by prefix.
Example how internal image-previews are generated.
```js
prefixCache( "/images-preview/", "/images/", function( filepath, cachepath, next ) {
imageMagick( filepath )
.autoOrient()
.resize( 200, 200 + "^" )
.gravity( "Center" )
.extent( 200, 200 )
.write( cachepath, next );
});
```
If a more fine grain control is required to generate previews, here is how to generate previews for different filetypes, as well as how to serve static images if a custom preview is hard to generate (i.e. for spreadsheets):
```js
const pictures = [".jpg", ".jpeg", ".png", ".gif"];
const extPictures = [".pdf", ".eps"];
const movies = [".mov", ".mp4"];
const allowedTypes = [].concat(pictures, extPictures, movies);
k.static.prefixCache( "/server-preview/", path.join( config.syncPrefix, 'server/' ) , function( filepath, cachepath, next ) {
const extension = path.extname( filepath ).toLowerCase();
if( allowedTypes.indexOf( extension ) < 0 )
return next( new Error( `Unsupported file type '${extension}'` ) );
/* use 10th frame as preview */
if( movies.indexOf( extension ) >= 0 )
filepath = filepath + "[10]"
/* use 1st page of pdf */
else if( [".pdf"].indexOf( extension ) >= 0 )
filepath = filepath + "[0]"
imageMagick( filepath )
.autoOrient()
.resize( 200, 200 + "^" )
.gravity( "Center" )
.extent( 200, 200 )
.write( cachepath, next );
}, {
router: k.router, /* mount directly on current path */
pathRewrite: t => {
const extension = path.extname( t ).toLowerCase();
/* map known mimes */
if( /\.odt|\.doc.?|\.rtf|\.txt/.exec( extension ) )
return { redirect: "/images/mimetypes/text.png" };
if( /\.ods|\.xls.?|\.csv/.exec( extension ) )
return { redirect: "/images/mimetypes/spreadsheet.png" };
if( /\.odp|\.ppt.?/.exec( extension ) )
return { redirect: "/images/mimetypes/presentation.png" };
if( /\.mp3|\.wav?/.exec( extension ) )
return { redirect: "/images/mimetypes/audio.png" };
if( /\.zip|\.7z|\.tar|\.gz?/.exec( extension ) )
return { redirect: "/images/mimetypes/archive.png" };
/* custom previews */
const match = /(.*)\.tw-([a-z0-9]{3})\.(jpg)/gi.exec( t );
if( match )
return {
source: `${match[1]}.${match[2]}`,
targetExtension: `.${match[3]}`,
};
/* previews */
if( allowedTypes.indexOf( extension ) >= 0 )
return t;
/* generic image */
return { redirect: "/images/mimetypes/generic.png" };
},
});
```