Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mbta/otp-deploy

Deployment helpers for OpenTripPlanner @ MBTA
https://github.com/mbta/otp-deploy

Last synced: about 1 month ago
JSON representation

Deployment helpers for OpenTripPlanner @ MBTA

Awesome Lists containing this project

README 

        
# OTP Deploy



This repo contains all the deploy scripts, CI/CD, and config files for the MBTA's OpenTripPlanner instance.



* [OTP Repo](https://github.com/opentripplanner/OpenTripPlanner)

* [OTP Docs](https://docs.opentripplanner.org/en/dev-2.x/)

* [OTP Chatroom](https://gitter.im/opentripplanner/OpenTripPlanner)



## Setup



You'll need to clone this repo run the following from the project root:



1. `asdf install`

1. `direnv allow`

1. `./scripts/build.sh` - packages OTP into a jar, then runs the OTP build process



If you want to test with local GTFS changes, put a copy of your GTFS file (if you've built

one with [`gtfs_creator`](https://github.com/mbta/gtfs_creator), it will be written to

`/output/google/MBTA_GTFS.zip`) in `var/MBTA_GTFS.gtfs.zip` and re-run the

`build.sh` script. Note that it will be overwritten the next time you run `update_gtfs.sh`.



## Run Locally



All the build files, including configs, GTFS, OSM data, and the graph file that the build step creates, are placed in

the `./var` directory.



With all of that setup done, you should be able to run `./scripts/server.sh`. This will start your local

OTP instance and, when ready, print a message saying that the web server is ready and listening.



Open a browser pointing to [`localhost:8080`](http://localhost:8080), and you'll have a bare-bones web interface to OTP

where you can try out trip plans.



If you're not seeing any results from the current day, double check that you don't have an old GTFS file, if the

schedules don't include the requested date, OTP can't plan a trip for it. If OTP doesn't seem to include recent OSM

changes that you expect it have, double check that the `.pbf` file is up to date, and that the changeset is included

in the latest [Geofabrik update](http://download.geofabrik.de/north-america/us/massachusetts.html). These updates only

happen once per day, so you may need to wait up to 24 hours to pull any OSM changes you've made.



## Updating OTP from upstream



This repo uses env vars defined in `.envrc.global` to determine the OTP repo and commit to build with. You can test

locally with a different repo or commit by copying `.envrc.local` from `.envrc.local.template`, setting the values in

there will override the global ones.

The values from `.envrc.global` are read by the `set-otp-build-params` step in `.github/workflows/deploy.yml` and used

in the builds in AWS. You can add more cases there to override these for specific environments if you want to deploy a

different OTP branch for dev testing, though this should only need to be used temporarily for longer term testing.



The OTP_COMMIT var can be set either to a commit hash, or to a branch name. A branch isn't used in prod so that we're

always using a consistent version to build unless we specifically upgrade it. However, it will probably be more

convenient to use a branch locally and in dev, so feel free to set it to a branch in those cases.



To pull the latest OTP changes, do the following:



1. Check the [OpenTripPlanner](https://github.com/opentripplanner/OpenTripPlanner/commits/dev-2.x) repo for the latest

   commit on `dev-2.x` (this is their bleeding-edge release branch)

1. Copy the commit hash

1. Update it in `.envrc.global` and test the changes locally (making sure you're not overwriting them with

   `.envrc.local`)

1. Put up a PR and merge/deploy once approved



## Debugging



If you need to use a debugger, you can run OTP through IntelliJ. These steps must be run from a

cloned `OpenTripPlanner` repo, _not_ the `otp-deploy` deploy repo, though you can point the build

and run steps to use the config files in `otp-deploy`. The config here assumes that the two repos

exist alongside each other in the same directory.



Remember to double check the commit that the

`OpenTripPlanner` repo is on, depending on what you're testing, you might need it to match or be

different from the version set in `.envrc.global`.



1. Start by adding a new debug configuration,

go to `Edit Configurations` and create a new [Application template, following the IntelliJ

docs](https://www.jetbrains.com/help/idea/run-debug-configuration.html#createExplicitly) using the

values provided below.

   * Java version: Java 17 (browse to your asdf install, usually `~/.asdf/installs/java/`,

   for exact parity with the version used by the build scripts, though any Java 17 JDK is fine)

   * Main class: `org.opentripplanner.standalone.OTPMain`

   * Program arguments: `--load ../otp-deploy/var/`

   * Working directory: ``

1. You'll also need to [follow the IntelliJ docs to set your SDK in the project

setup](https://www.jetbrains.com/help/idea/sdk.html#change-project-sdk). Again, the most consistent

option is your asdf JDK, but any Java 17 JDK should work.

1. Make sure you run `mvn clean install` to fetch all the dependencies, you can do this with the

"Excute Maven Goal" button in the IntelliJ Maven submenu, or you can just run the command in the

terminal.

1. Once the config is created, you can run it normally or as a debugger. If you run into any issues

with the application running out of memory, you can go to `Modify options > Add VM options` in the

edit configuration UI then add the flag `-Xmx8G` to increase the JVM memory pool maximum.



If you need to run or debug the build process for some reason, just create another configuration

with the same values except with program arguments `--build --save ../otp-deploy/var/`.



## Docker



Building and running the docker image locally is usually not necessary, since it's faster to just

run it using the build scripts or IntelliJ.



From the OTP directory:



1. `docker-compose up`



The OTP web interface will then be running at [`localhost:5000`](http://localhost:5000), and you can update your

dotcom `.envrc` to point to this URL to test the docker image end to end locally.



## Deploy



Deploys to prod happen automatically when any changes are merged into `master`. You can manually

perform a dev deploy of any feature branch using the

[deploy workflow](https://github.com/mbta/otp-deploy/actions/workflows/deploy.yml). You can

select the branch you want to deploy and the environment you want to deploy to.



Additionally, deploys to the associated environment are triggered by the

[gtfs_creator](https://github.com/mbta/gtfs_creator) deploy workflows. Any GTFS changes require a

redeploy of OTP, if the triggered pipelines fail, the trip planner will not reflect the latest GTFS

changes until it has been successfully deployed.