{"id":14070401,"url":"https://github.com/RemiRigal/DatasetExplorer","last_synced_at":"2025-07-30T07:33:12.710Z","repository":{"id":38460269,"uuid":"253818288","full_name":"RemiRigal/DatasetExplorer","owner":"RemiRigal","description":"A web tool for local dataset browsing and processing developped using the Flask + Angular stack.","archived":false,"fork":false,"pushed_at":"2022-09-08T14:02:50.000Z","size":5802,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-08-13T07:17:51.262Z","etag":null,"topics":["ai","angular","data-processing","data-science","data-visualization","dataset","dataset-analysis","docker","docker-compose","flask","web-application"],"latest_commit_sha":null,"homepage":"https://remirigal.github.io/DatasetExplorer","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RemiRigal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-07T14:29:16.000Z","updated_at":"2024-01-19T20:00:46.000Z","dependencies_parsed_at":"2022-08-24T02:50:20.735Z","dependency_job_id":null,"html_url":"https://github.com/RemiRigal/DatasetExplorer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RemiRigal%2FDatasetExplorer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RemiRigal%2FDatasetExplorer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RemiRigal%2FDatasetExplorer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RemiRigal%2FDatasetExplorer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RemiRigal","download_url":"https://codeload.github.com/RemiRigal/DatasetExplorer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228110232,"owners_count":17871150,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","angular","data-processing","data-science","data-visualization","dataset","dataset-analysis","docker","docker-compose","flask","web-application"],"created_at":"2024-08-13T07:07:43.638Z","updated_at":"2024-12-04T12:30:51.222Z","avatar_url":"https://github.com/RemiRigal.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# Dataset Explorer\n\nA web tool for local dataset browsing and processing developped using the Flask + Angular stack.\n\n\n## Features\n\nDataset Explorer provides the following features:\n   - Web-based (local server)\n   - Light weight and powerful (handle 100k+ files like a breeze)\n   - Easy visualization of data\n   - Plugin-based tool system for easy custom data processing\n\n![Browser](docs/assets/screenshots/Brower.png)\n\n## Documentation\n\nDocumentation is hosted on [GitHub Pages](https://remirigal.github.io/DatasetExplorer/).\n\n\n## Getting Started\n\n### Using Docker\n\nA docker compose file is provided, you can start the tool with:\n```shell script\ndocker-compose up -d\n```\n\nHowever, you must mount your dataset directory in the `/data` directory of the backend container. You can do so by updating the following line of the `docker-compose.yml` file:\n```yaml\nvolumes:\n  - ./data:/data # Replace '.data/' by the root path to your dataset\n```\n \n\n### From source\n\n#### Backend (Flask)\n\nThe backend requires `Python 3.8`.\n```shell script\ncd backend\npip install -r requirements.txt\nexport DATASET_EXPLORER_ROOT=/path/to/dataset/root\npython app.py\n```\n\n#### Frontend (Angular)\n\nThe frontend requires `Angular CLI v9.1.0` and `NodeJS v12.16.1`, make sure that they are installed first.\n```shell script\ncd frontend\nnpm install\nng serve --host 0.0.0.0\n```\n\n### Test tool\n\nThe app is available at http://127.0.0.1:4200.\n\n\n## Write custom plugins/tools\n\nDataset Explorer allows you to write custom Python tools so that you can instantly test your own processing pipeline on your data. To do so you will need to create a class that inherits the `BasePlugin` class, the simplest plugin looks like this:\n```python\n# my_custom_plugin.py\nimport cv2\nfrom dataset_explorer.io import FileType\nfrom dataset_explorer.plugins import BasePlugin\n\n\nclass MyCustomPlugin(BasePlugin):\n    \"\"\"\n    MyCustomPlugin inherits from the BasePlugin class\n    \"\"\"\n\n    def __init__(self):\n        \"\"\"\n        The child class must provide a name as well as the input/output types of the plugin\n        It's required that its constructor takes no argument\n        \"\"\"\n        super(MyCustomPlugin, self).__init__(\"MyCustomPlugin\", FileType.IMAGE, FileType.IMAGE)\n\n    def process(self, inFilename, outFilename):\n        \"\"\"\n        This method is called automatically, the inFilename argument is the path to the file to process\n        The outFilename is the path to the result file that must be created\n        For example, this function is converting the input image to black and white\n        \"\"\"\n        image = cv2.imread(inFilename)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n        cv2.imwrite(outFilename, image)\n```\n\nThis file must be placed either at `~/.DatasetExplorer/plugins`, or at any directory listed in the environment variable `DATASET_EXPLORER_PLUGINS` (separated by `:`).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRemiRigal%2FDatasetExplorer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FRemiRigal%2FDatasetExplorer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FRemiRigal%2FDatasetExplorer/lists"}