{"id":19040378,"url":"https://github.com/glory0432/Synchronise-Backend-Express","last_synced_at":"2025-09-07T02:30:43.671Z","repository":{"id":250596354,"uuid":"834887118","full_name":"ChrisChan1117/Synchronise-Backend-Express","owner":"ChrisChan1117","description":"Integration backend as a service to connect web services to your app","archived":false,"fork":false,"pushed_at":"2024-08-30T09:29:04.000Z","size":26156,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-02T07:10:59.158Z","etag":null,"topics":["backend","integration","javascript"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChrisChan1117.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-28T16:57:06.000Z","updated_at":"2024-10-12T03:47:58.000Z","dependencies_parsed_at":"2024-11-08T22:35:00.384Z","dependency_job_id":null,"html_url":"https://github.com/ChrisChan1117/Synchronise-Backend-Express","commit_stats":null,"previous_names":["chrischan1117/server","chrischan1117/synchronise-backend-express"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisChan1117%2FSynchronise-Backend-Express","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisChan1117%2FSynchronise-Backend-Express/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisChan1117%2FSynchronise-Backend-Express/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisChan1117%2FSynchronise-Backend-Express/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChrisChan1117","download_url":"https://codeload.github.com/ChrisChan1117/Synchronise-Backend-Express/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232163089,"owners_count":18481580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","integration","javascript"],"created_at":"2024-11-08T22:22:25.088Z","updated_at":"2025-09-07T02:30:37.628Z","avatar_url":"https://github.com/ChrisChan1117.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# [Synchronise](https://www.synchronise.io)\n![HomePage of Synchronise](https://raw.githubusercontent.com/synchroniseiorepo/server/master/public/images/githubImage1.png)\n\n## Table of contents\n* [Introduction](#introduction)\n* [Documentation](http://synchroniseiorepo.github.io/documentation)\n* [Install and Run on your own server](#install-and-run-on-your-own-server)\n* [Contribute](#contribute)\n\n## Introduction\nSynchronise is an integration backend as a service. It helps you connect web services to your app, without having to configure everything yourself or even maintain a server. It simplifies the integration process by standardising any APIs, and turning them into objects called \"Components\".\nComponents can be combined together in a \"Workflow\" to create more complex actions. Steps after steps you can create very powerful Workflows that will save you time and money.\n\nImagine a \"Workflow\" that \"registers a user on your database\", \"send an email with mailgun\" and \"create a customer account on stripe\", all that without having to write a single line of code.\n\nThe entire codebase is provided as is, under the MIT licence except the database containing sensitive data of existing customers. This means you are free to copy, modify and distribute Synchronise however you want. Software should be shared, and I am convinced the work the community will achieve, will always surpass my own dedication to this platform.\n\nHere are the 3 features available so far:\n![Components on Synchronise](https://raw.githubusercontent.com/synchroniseiorepo/server/master/public/images/githubImage2.png)\n![Workflows on Synchronise](https://raw.githubusercontent.com/synchroniseiorepo/server/master/public/images/githubImage3.png)\n![Custom Components on Synchronise](https://raw.githubusercontent.com/synchroniseiorepo/server/master/public/images/githubImage4.png)\n\n## Install and Run on your own server\n### Fork it or Clone it\n```shell\ngit clone https://github.com/ChrisChan1117/server\n```\n\n### Install the databases\nSynchronise needs 3 separate databases to avoid collissions.\nFirst of all, create 3 new Redis databases. You can either do this locally by downloading [Redis](http://redis.io/download), or you could create a FREE account at [Redis Labs](https://redislabs.com/). Disclaimer: I am not affiliated with Redis Labs in anyway.\n\nThe 3 databases you need to create are:\n- \"data\" Contains the actual data of Synchronise (users, components, projects...)\n- \"session\" The session of the users\n- \"events\" The events database is dedicated to the PubSub system for refreshing the web interface live.\n\nOnce you have done that, go to the file \"/helpers/assets.js\" and modify the settings accordingly using the credentials of your redis databases.\n\nLook for the lines starting like this:\n```javascript\n/***** DATABASE *****/\n// These are the credentials to connect to the databases\n// PRODUCTION DATABASE\nif(process.env.PRODUCTION){\n\tvar rcEvents = {\n```\n\nAnd that's all you need to do. The system will automatically create and populate the different data stores without any other actions from your side.\n\n### What about MySQL or Oracle?\nTo achieve the best efficiency I decided to build Synchronise entirely around Redis. I did not consider a second, the idea of MySQL because it did not make sense at the time.\n\nNow, this does not mean you MUST use Redis to run Synchronise. You could use MySQL with just a few modifications of the code. The elements that you need to be modify are all contained in the file \"helpers/assets.js\".\n\nThe name of the file \"assets\" is not relevant I know that, but changing it now would require quite some work. If someone wants to modify it everywhere, feel free to rename it to something like \"config.js\" for example.\n\nIf you want to modify the databases system, look for a line starting like this in the file \"helpers/assets.js\":\n```javascript\nvar publishRedisAdapter    = redis(rcEvents.port, rcEvents.host, { return_buffers: false, auth_pass: rcEvents.pass });\n\tpublishRedisAdapter.on(\"error\", function(error){\n```\nThen modify the adapters accordingly to your new architecture. Make sure your adapters responds to all the methods.\n\nFor example the publisher adapter needs a pub method:\n```javascript\npublishRedisAdapter.pub\n```\nIf you have any difficulties adapting a new database system, give me a shout [@synchroniseio](https://www.twitter.com/synchroniseio) and I'll help you as fast as I can.\n\n### Configure your server\nThere are some elements you need to setup before you can start using Synchronise server.\n\nGo to the file \"/helpers/assets.js\" and modify the settings to your needs. They are all explained in the file so you should not have much difficulty understanding what they do. The settings you should modify are from the line:\n```javascript\nexports.LIMIT_REQUESTS_FREE_PLAN = 10000;\n```\nto the line\n```javascript\n\tvar rcData = {\n\t\thost       : \"localhost\",\n\t\tport       : '6379',\n\t\tpass       : \"foobared\",\n\t\tdisableTTL : true,\n\t\tsecret     : \"foobared\"\n\t};\n}\n```\n\nAll good? Now let's run your server!\n\n### Run your server\nThe entry file for the server is located in the \"bin\" folder\n```shell\nnode bin/www\n```\n\nYour server should now be running at\n\n```\nhttp://localhost:3001\n````\n\nBecause this is the first time you run the server, there are no admin on the system. You MUST, create the first user using a MANUAL signin system. Basically, use email and password, not the social login. This first user you create will have all the permissions, including the superadmin panel.\n\nAll users after the first one, will have normal \"user\" permissions. You can modify the permissions of a user by going to the superadmin panel.\n\n### Optional SSL\nIf you want to serve your server over SSL you will need to enable it in the file \"helpers/assets.js\".\n\nEnable SSL:\n```javascript\nexports.SHOULD_USE_SLL = true;\n```\n\nYou need to add 2 new files \"certificate.pem\" and \"privatekey.pem\" in the root folder of Synchronise so that it now looks like this:\n\n...\n\n[folder]views\n\n[newFile]certificate.pem\n\n[newFile]privatekey.pem\n\n[file]package.json\n\n...\n\n## Deploy Synchronise on AWS\nDeploying Synchronise on AWS takes a bit more work, but not too much I promise!\n\n### Spring clean\nThe first thing to do is to get rid of all the modules that AWS does not like.\n\nFor some reasons, EC2 instances don't want to run if some NPM modules are provided. I think this is for security reasons, but we could not find any clear evidence of that. These packets are not mandatory for Synchronise to execute properly, so it is safe to remove them.\n\nIn the folder of Synchronise, execute the following command to remove all the modules AWS does not like:\n```shell\nnpm remove oniguruma; npm remove fsevents; npm remove bufferutil; npm remove utf-8-validate;\n```\n\nNow that you have cleaned the folder, you need to consider whether you want to deploy on a single instance, or if you want to deploy on multiple instances for scalability purposes.\n\n### Single Instance\nLet's face it, Synchronise does not play well with Elastic Beanstalk, because it uses some custom configurations for the WebSockets.\n\nHowever, I still recommend to create an EBS application because it simplifies the deployment process. You will use EBS for its deployment script, not for its scalability purpose.\n\nGo and create a new app on AWS Elastic Beanstalk.\n\nNow that this is done, go in the main folder of Synchronise and find the folder named \".aws\". Inside it you will find a file called \"credentials\". Put your own AWS credentials there.\n\nNote that you should also put your AWS credentials in the file \"/helpers/assets.js\" when you configure Synchronise.\n\nLook for a line starting like this:\n```javascript\nexports.AWSCredentials = {\n```\n\n### Getting ready for deployment.\nConfigure your Elastic environment with:\n```shell\neb init\n```\nMake sure your have commited all changes of Synchronise and execute\n```shell\neb deploy your_application_name\n```\n\nYour application should be running now at the address given by AWS.\n\n### Configuring deployed app\nAlmost there, one last step before it works properly. We need to tell Synchronise it is running in production mode.\n\nIn ElasticBeanstalk go to \"Configuration\" -\u003e \"Software Configuration\" -\u003e scroll down to \"Environment Properties\".\n\nAdd a new property with the name \"AWS\" and the value \"true\".\n\nApply your changes and wait for relaunch of your instance. Finger crossed, everything should work fine now.\n\n### Multiple Instances\nComing soon...\n\n## Contribute\n### File structure\n- backend      : Backend functions. In an MVC architecture this would be the Controller\n- bin          : Setup of the app and scripting\n- helpers      : Functions that are just doing one task, also contains the credentials of the software\n- libraries    : Some NPM modules needed modifications to be compatible. We placed them here to avoid them being updated when we run NPM commands\n- models       : Contains all the data models and their methods\n- node_modules : Contains all the node modules\n- public       : Contains all the files that will potentially be directly served to the browser at some point and that are available at the root of the website\n    - css      : Contains all the CSS files of the website\n    - js       : Contains all the Javascript files of the website\n        - dependencies : Contains all the libraries that haven't been created by the Synchronise team\n        - helpers      : Just helpers, there is one for manipulating the url for example\n        - libraries    : Contains all the libraries created by the Synchronise team\n    - fonts    : Contains all the fonts of the website\n    - images   : Contains all the images of the website\n    - js-react : Contains all the React.JS components in their uncompiled version. This folder has the same structure than the JS folder and all the content of js-react will be compiled and transferred to the JS folder when executing the required babel script\n    - libs     : Contains all the versions of the Synchronise client library\n- routes       : Contains all the routes of the website\n- view         : Contains all the views of the website\n\n### Realtime Interface Update\nHold tight, this one is challenging. It is not complicated, but confusing at first. Our constraints were speed, data footprint and memory footprint on the server.\nThis is not your usual Realtime PubSub because we wanted to ensure it is optimized for our case. We estimate you can run up to 2500 clients with a t2.micro instance on AWS.\n\nIf you have difficulty with understanding the realtime system, give us a shout @synchroniseio on Twitter.\n\nWhen we created the platform we wanted to make sure the interface reacts to changes as it happens.\nSynchronise comes with a built-in websocket, encapsulated in an object called Synchronise on the client side.\nThe Synchronise object comes with functions that simplify the work with the backend and the websocket.\nSynchronise.Cloud.run()\nSynchronise.User.current()\nSynchronise.User.login()...\n\nThe idea is that the interface should refresh itself automatically when a change happen on the server.\nOur realtime system works around a pub-sub architecture. The client code in Javascript subscribes to events happening on the server.\n\nIn this example\nSC = Synchronise Library Client\nS  = Server\n\n- SC ------\u003e S      When the page loads, the Client registers to the server and subscribes all functions for realtime\n- SC \u003c------ S      The Server provides a unique ID for the realtime session\n- SC \u003c------ S      When something happens on the data model, The server notifies all clients that are still connected, that en event happened for type of data X\n- SC ------\u003e S      Client says \"I'm still alive, give me the updates for the type of data X\"\nSC \u003c------ S      Server provides updated data to the client\n\nThis is very similar to Flux for React.JS. Flux wasn't mature enough when we first started Synchronise. We decided to create our own system.\nThe subscription is made at the function level, allowing to have a very small amount of functions needing to update every time.\n\nHere is an example:\nLet say we are displaying a list of user on the client using the function \"getListUser\". We want that function to be called every time a new user has signed up.\n\nOn the client side\nThe Javascript function is executed to collect the current users. That same function also subscribes to any changes happening with users by specifying the \"realtime\" parameter\nSynchronise.Cloud.run(\"getListUsers\", {realtime: true, sort: \"name\"...}, {\n    success: function(users){\n        // Display the users on the interface\n    }\n});\n\nOn the server side\nEvery time a new user signs up, let say in the function \"createUser\" of the backend \"user\", we call\nresponse.success(\"userCreate yeah!\", 200, {}, [idOfUsersConcernedByTheUpdate]);\n\nThe response.success function will calls all concerned clients, and tell them to ask for new data about the \"getListUsers\" function.\n\n- \"userCreate yeah!\": This is any data we want to send back to the client. This has no impact on the realtime subscription\n- 200: This is the status code of the execution. Because it is a success it is 200. It could be anything we want.\n- {}: Contains data that are going to filter the realtime subscription. More on that later.\n- [idOfUsersConcernedByTheUpdate]: The is an array of user ID that are concerned by the update. If we wanted all users to know about the update we would make a list of all registered users on the website. This is useful in order to only ping the users that are actually concerned by the update of the data. It is very unlikely that everyone is concerned by the same update of data.\n\nCalling response.success is not enough to trigger a realtime subscription. You also need to subscribe functions on the super admin interface.\nTo do so, login on Synchronise (on your own instance) and go to http://yourOwnInstanceUrl/superadmin\n\nYou will be facing  an interface with two blocks: \"Add new subscription\" and \"Existing subscriptions\"\n\nOn the \"Add new subscription\":\n- \"Room name\" is the name of the function that will trigger an update. In our previous example it was \"createUser\"\n- \"Subscription name\" is the name of the function that will be recalled when the update occurs. In our previous example it was \"getListUsers\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglory0432%2FSynchronise-Backend-Express","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fglory0432%2FSynchronise-Backend-Express","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fglory0432%2FSynchronise-Backend-Express/lists"}