https://github.com/janeliascicomp/neuronbridge-services
AWS Services for NeuronBridge
https://github.com/janeliascicomp/neuronbridge-services
Last synced: 9 months ago
JSON representation
AWS Services for NeuronBridge
- Host: GitHub
- URL: https://github.com/janeliascicomp/neuronbridge-services
- Owner: JaneliaSciComp
- License: bsd-3-clause
- Created: 2020-04-30T14:49:04.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2025-05-27T18:46:03.000Z (11 months ago)
- Last Synced: 2025-05-27T19:43:04.944Z (11 months ago)
- Language: JavaScript
- Size: 5.26 MB
- Stars: 2
- Watchers: 6
- Forks: 1
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# NeuronBridge backend services
[](https://zenodo.org/badge/latestdoi/260239328)
[](https://circleci.com/gh/JaneliaSciComp/neuronbridge-services)
## Deployment
As a prerequisite, you need to have the [AWS CLI](https://aws.amazon.com/cli/) installed and configured with proper AWS credentials.
### Deploy NeuronBridge shared VPC
The VPC deployment is a one time deployment for ALL deploy environments, no matter whether they are DEV, PROD, TEST, VAL and so on. This will create a VPC that will be shared by all alignment jobs so be careful not to remove it because if you do you will have to redeploy all align environments.
To deploy:
```
cd vpc
npm install
npm run sls -- deploy
```
Notice that stage is not needed because it will deploy to a 'shared' stage ('janelia-neuronbridge-vpc-shared')
### Deploy Burst Compute framework
First, follow the instructions at [JaneliaSciComp/burst-compute](https://github.com/JaneliaSciComp/burst-compute) to deploy the framework.
### Deploy Alignment Service
The NeuronBridge alignment service requires an AMI instance preconfigured with ECS and with all required volumes mounted as expected by the alignment batch job.
To create the AMI use these steps:
* start an [Amazon ECS-optimized Amazon Linux AMI](https://aws.amazon.com/marketplace/search/results?x=0&y=0&searchTerms=Amazon+ECS-Optimized+Amazon+Linux+AMI&page=1&ref_=nav_search_box).
* start the EC2 instance
* run the following commands that mount the expected volumes:
```
sudo yum -y update
sudo yum install -y fuse-devel
sudo mkfs -t ext4 /dev/xvdb
sudo mkdir /scratch_volume
sudo echo -e '/dev/xvdb\t/scratch_volume\text4\tdefaults\t0\t0' | sudo tee -a /etc/fstab
sudo mount –a
sudo stop ecs
sudo rm -rf /var/lib/ecs/data/ecs_agent_data.json
```
* save an image from the running EC2 instance
Once the AMI instance ID is available make sure you set the proper AMI instance in align/serverless.yml.
To deploy:
```
cd align
npm install
npm run sls -- deploy -s dev
```
The command above will create the compute environment, the job definition and the job queue.
### Deploy Color Depth Search Service
In order to create the color depth search lambdas run:
```
cd search
npm install
npm run sls -- deploy -s dev
```
To deploy with different search limits:
```
PER_DAY_SEARCH_LIMITS=2 CONCURRENT_SEARCH_LIMITS=2 npm run sls -- deploy -s cgdev
```
Note: a negative value for a limit means unlimited.
To update a single function, once you have a deployed stack
```
npm run sls -- deploy function -f -s dev
```
### Alignment Parameters
* force_voxel_size - if true it uses resolution parameters provided by the user
* xy_resolution
* z_resolution
* reference_channel
* number of slots (not exposed to the user)
### Color Depth Search Parameters:
* dataThreshold - default: 100
* maskThreshold - default: 100
* pixColorFluctuation (zSliceRange) - default: 2
* xyShift - default: 2
* mirrorMask - default: true
* minMatchingPixRatio - default: 2