{"id":13554919,"url":"https://github.com/assemblee-virtuelle/Semantic-Bus","last_synced_at":"2025-04-03T07:33:24.790Z","repository":{"id":37420352,"uuid":"58863034","full_name":"assemblee-virtuelle/Semantic-Bus","owner":"assemblee-virtuelle","description":"object flow treatment,  data transformation","archived":false,"fork":false,"pushed_at":"2024-04-12T14:33:24.000Z","size":52179,"stargazers_count":55,"open_issues_count":69,"forks_count":9,"subscribers_count":20,"default_branch":"master","last_synced_at":"2024-04-14T09:04:59.092Z","etag":null,"topics":["data-mining","data-transformation","semantic-data-transformation","worflows","workflow-sharing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/assemblee-virtuelle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-05-15T13:27:34.000Z","updated_at":"2024-04-07T18:22:41.000Z","dependencies_parsed_at":"2024-04-14T04:52:42.803Z","dependency_job_id":null,"html_url":"https://github.com/assemblee-virtuelle/Semantic-Bus","commit_stats":{"total_commits":1307,"total_committers":23,"mean_commits":56.82608695652174,"dds":0.5899005355776588,"last_synced_commit":"7a127ce7d99b00426b9d970e1d342e52ee913984"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assemblee-virtuelle%2FSemantic-Bus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assemblee-virtuelle%2FSemantic-Bus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assemblee-virtuelle%2FSemantic-Bus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/assemblee-virtuelle%2FSemantic-Bus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/assemblee-virtuelle","download_url":"https://codeload.github.com/assemblee-virtuelle/Semantic-Bus/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246725395,"owners_count":20823628,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-mining","data-transformation","semantic-data-transformation","worflows","workflow-sharing"],"created_at":"2024-08-01T12:02:57.586Z","updated_at":"2025-04-03T07:33:19.782Z","avatar_url":"https://github.com/assemblee-virtuelle.png","language":"JavaScript","funding_links":[],"categories":["JavaScript","others"],"sub_categories":[],"readme":"# Semantic-Bus\n\nSemantic data transformation \u0026 semantic container crawling\n\n\n## Features\n\n- Ready for 100 MO / 100000 Item per process\n- Multiple source protocol\n- Multiple destination protocol\n- Rich API creation\n- Data transformation without coding\n- Value correspondence (translation between taxonomy)\n- Join data by field\n- Data aggregation from multiple source\n- Complex uniqueness\n- Geo data completion from address\n- Middle cache database for performance\n- Scrapping \u0026 crawling\n- Filter\n- Workflow sharing \u0026 multi User Edition\n- API parameters usable in workflow components\n\n## need tiers\n- Big Data Support (Infinit data Volume)\n  - we can do now it with external database as mongodb and advances workflows\n- Authentification to external services other than api tocken (headers or url)\n  - we use n8n or nocodeapi if we want to authentify to a data provider\n\n## Road map\n- Reusable and preconfigured pattern building from component\n  - pattern can be implement now whith workflow trigger by api and body parameters but this could improve.\n  - it is now possible using api and call it from an other workflow. Better usability and component have to improve experience and performance.\n- define how pay execution cause by internal API (http provider call by a http consumer of an other workflow)\n- HTTP provider external authentificaiton and restriciton\n  - authentified api calling\n    - secure api provided by http provider component when calling out of SemanticBus Instance\n  - api user restriction\n    - define who ca execute HTTP provider\n- better sharing and options of sharing\n  - group management\n  - change admin\n  - change role\n- new ergonomy\n- Automatic Ontology transformation (semantic web)\n  - manual transformation now\n- Multiple entry point and exit point for a components\n  - entry discrimantaion have to be specified in component now\n- Workflow sharing by Google Drive or other cloud or better wokflow management\n\n\n## Archi\n\n- Core (./core) ( shared low level services )\n- Main ( ./main ) ( main app : frontend and API)\n- Engine (./engine) ( engine for graph resolution )\n- Timer (./timer) ( service for schedule workflow )\n\n## Config\nYou have to duplicate config.js to local.config.js.\nSome config properties as googleAuth can be fullfill.\nYour local.config.js file will be ignore by git.\n\n## Launch with docker (recommended)\n### make (recommended)\n\n( DOCKER-COMPOSE REQUIRED )\n```bash\nmake start =\u003e start project\nmake log =\u003e log main and engine container\nmake restart =\u003e force recreate\nmake stop =\u003e kill all container\n\n```\ndocker container still alive sometime:\n```\ndocker kill semanticbus_mongo_1\ndocker kill rabbitmq\n```\n\n### docker-compose (Not recommended)\n\ndocker-compose up -d\n\n## Launch with nvm \u0026 npm (Not recommended)\n\n```bash\nsudo apt-get install g++ build-essential\ncurl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash\n```\n\nRestart console\n```bash\nnvm install 18\ncd core \u0026\u0026 npm install\ncd main \u0026\u0026 npm install\ncd engine \u0026\u0026 npm install\ncd main \u0026\u0026 node app.js\ncd engine \u0026\u0026 node app.js\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fassemblee-virtuelle%2FSemantic-Bus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fassemblee-virtuelle%2FSemantic-Bus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fassemblee-virtuelle%2FSemantic-Bus/lists"}