{"id":18964206,"url":"https://github.com/janeliascicomp/image_processing_pipeline","last_synced_at":"2026-04-02T20:30:14.349Z","repository":{"id":66366553,"uuid":"124900420","full_name":"JaneliaSciComp/image_processing_pipeline","owner":"JaneliaSciComp","description":"Interface to configure pipeline jobs for LightSheet","archived":false,"fork":false,"pushed_at":"2023-09-29T01:04:37.000Z","size":19662,"stargazers_count":2,"open_issues_count":15,"forks_count":0,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-01-31T22:29:10.857Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JaneliaSciComp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-12T14:19:33.000Z","updated_at":"2023-08-16T17:20:25.000Z","dependencies_parsed_at":"2023-02-20T16:01:22.689Z","dependency_job_id":"f8618400-0d2e-480f-a5c9-7fcf18ed24cb","html_url":"https://github.com/JaneliaSciComp/image_processing_pipeline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fimage_processing_pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fimage_processing_pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fimage_processing_pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Fimage_processing_pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JaneliaSciComp","download_url":"https://codeload.github.com/JaneliaSciComp/image_processing_pipeline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239959947,"owners_count":19725221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T14:23:17.193Z","updated_at":"2026-04-02T20:30:14.289Z","avatar_url":"https://github.com/JaneliaSciComp.png","language":"JavaScript","readme":"# Lightsheet Pipeline Flask App\n\nThis repo contains a python flask application to configure step parameters for pipeline jobs.\n\n### Development\n\nMake sure, your python is version 3.5 or higher and your pip is using this correct python:\n\n```bash\n$ python --version\n$ pip --version\n$ which python\n$ which pip\n```\n\nCreate a virtualenv environment, here the environment is called 'env':\n\n```bash\n$ virtualenv env --no-site-packages\n```\n\nor better\n\n```bash\n$ virtualenv --no-site-packages -p [path to your python] env\n```\n\nIf you are on a Mac, use venv instead of virtualenv, because mathplotlib needs a framework build of Python\n\n```bash\n$ python -m venv env\n```\n\nActivate the environment:\n\n```bash\n$ source env/bin/activate\n```\n\nInstall the requirements:\n\n```bash\n$ pip install -r requirements.txt\n```\n\nIn Ubuntu, I had trouble doing that, because of issues with permissions when accessing some python libs in my HOME directory. I resolved that by pointing the HOME environment variable to another folder:\n\n```bash\n$ export HOME=/opt/home\n$ sudo chown [your user name]:[your group] /opt/home\n```\n\nWith\n\n```bash\n$ python manage.py runserver [-b \u003cbinding ip\u003e] [-p \u003cport\u003e]\n```\n\nyou should be able to run the Flask application.\n\nIf you're done with coding, you can deactivate the environment with the command\n\n```bash\n$ deactivate\n```\n\n### Deployment\n\nIf you haven't done so, install the node.js flightplan library and tool globally:\n\n```bash\n$ npm install -g flightplan\n```\n\nIf there is no command `fly` available in your cmd, please [troubleshoot](https://stackoverflow.com/questions/14803978/npm-global-path-prefix) issues with your $PATH.\n\nIn the folder where package.json is located, install the npm packages to use flightplan in your project:\n\n```bash\n$ npm install\n```\n\nMake changes to target and config in flightplan.js as necessary. Then use flightplan to create a new version entry in package.json and as a git commit with\n\n```bash\n$ fly version:local [ patch | minor | major ]\n```\n\nFinally, deploy the application with\n\n```bash\n$ fly deploy:production\n```\nIf this does not work, make sure ssh-agent is running. If not, run ssh-agent and set the resulting environment variables. Then, run:\n```bash\neval `ssh-agent -s`\nssh-add -k ~/.ssh/id_rsa\n```\n\n--------\n\nLocate lightsheet service file:\n\n```bash\n$ locate systemd | grep lightsheet\n/etc/systemd/system/lightsheet.service\n```\n\nRestart lightsheet services:\n\n```bash\n$ sudo systemctl stop lightsheet\n$ sudo systemctl start lightsheet\n```\n\n```bash\n$ sudo systemctl restart nginx\n```\n\n## Production\nSupervisor config file is located in\n\n```bash\n/etc/supervisor/conf.d/pipeline.conf\n```\n\nAfter making changes to the supervisor config file, reload the configuration with\n\n```bash\nsudo supervisorctl reread; sudo supervisorctl update;\n```\n\nRestart the application with\n```bash\nsudo supervisorctl restart 'pipeline:'\n```\n\n## Access the database\n\nWe use MongoDB to store the data. To open a Mongo cmd, type\n\n```bash\nmongo --host [your mongodb host]:[mongodb port]\n```\n\nSome other helpful commands:\n```bash\n\u003e use lightsheet\n\u003e db.jobs.help()\n\u003e db.jobs.findOne()\n\u003e db.jobs.drop()\n\u003e db.jobs.remove({ state: { $eq: 'ERROR'} })\n\u003e db.step.remove({ \"text1\": { $exists: true } })\n\u003e db.parameter.find( { name: { $gt: 'test' } } )\n\u003e db.parameter.find(ObjectId(\"5b12047ea275276dec9a2eb9\"))\n```\n\nBackup and restore\n```bash\n mongodump --host 10.40.3.155 --port 27036 --db lightsheet --out /opt/tmp/dump/\n mongorestore --drop -d lightsheet /opt/tmp/dump/lightsheet/\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaneliascicomp%2Fimage_processing_pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaneliascicomp%2Fimage_processing_pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaneliascicomp%2Fimage_processing_pipeline/lists"}