{"id":25299223,"url":"https://github.com/frankfarrell/dublinbus","last_synced_at":"2026-05-02T11:35:25.954Z","repository":{"id":36419963,"uuid":"40724943","full_name":"frankfarrell/dublinbus","owner":"frankfarrell","description":"Realtime analysis of Dublin Bus Data (under major refactoring!)","archived":false,"fork":false,"pushed_at":"2018-01-25T17:18:32.000Z","size":6496,"stargazers_count":2,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-06T22:44:52.002Z","etag":null,"topics":["dublin-bus-data","gps-data","java","postgis","redis","spring-boot"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/frankfarrell.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-08-14T16:29:21.000Z","updated_at":"2023-04-20T07:55:40.000Z","dependencies_parsed_at":"2022-09-03T10:50:37.193Z","dependency_job_id":null,"html_url":"https://github.com/frankfarrell/dublinbus","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/frankfarrell/dublinbus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankfarrell%2Fdublinbus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankfarrell%2Fdublinbus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankfarrell%2Fdublinbus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankfarrell%2Fdublinbus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/frankfarrell","download_url":"https://codeload.github.com/frankfarrell/dublinbus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/frankfarrell%2Fdublinbus/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32533344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-02T11:28:32.350Z","status":"ssl_error","status_checked_at":"2026-05-02T11:27:30.140Z","response_time":132,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dublin-bus-data","gps-data","java","postgis","redis","spring-boot"],"created_at":"2025-02-13T04:54:02.879Z","updated_at":"2026-05-02T11:35:25.938Z","avatar_url":"https://github.com/frankfarrell.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# dublinbus\n\nRealtime analysis of Dublin Bus Data\n\nThis is an attempt to build a realtime bus tracking and analysis platform.\n\n## Background\n\n* Postgres/PostGIS Installed and Running \n* Redis Running on default port\n* Download the DublinBus data set and extract archives from http://dublinked.com/datastore/datasets/dataset-304.php (Not necessary for demo)\n* Java 8\n* Python\n\n## To Run\n\n1. Extract dublinbusdb.7z with 7zip, this contains a dump of db with tables, views, functions and some data. \nImport into postgres with psql: http://www.postgresql.org/docs/9.1/static/backup-dump.html\nEg \"psql dublinbus \u003c dublinbusdb.sql\"\n\n2. Build the server project\ncd server\ngradle build\n\n3. This gives a Spring Boot Jar with a simple main. \nTo run gradle bootRun\n\nThis should bind: \nReactor TcpServer on port 5001: \nRest interface to port 8080\nSocketIO interface on port 9002\n\n4. To Simulate a GPS Feed, we can run the Generator.py script. \nThis takes 2 args: datasource csv file location and line number to start from in this csv file. \nFor testing purposes (to save on the 1GB Full Dataset) run: \n\"python Generator.py Sample.csv 0\"\n\n5. To see if data is being processed, run redis-cli.exe\nCommand: \"SUBSCRIBE busdelayed.*\"\n\nThis should show a feed of calculations. Pick a line value after busdelayed.\u003cthisValue\u003e\n\n6. If you navigate your browser to localhost:8080?tripId=[thisValue] you will see a mpa with the route and the buses current position/delay. \n(The client is a work in progress!!!)\n\n## Architecture\n\n![Diagram](https://github.com/frankfarrell/dublinbus/blob/master/DublinBusPipeline.png)\n\n* The endpoint for these connections is a Spring Reactor TcpServer. \n\n* This persists the data in PostGIS using Hibernate Spatial and Spring Data Repositories\n\n* Postgres with the PostGIS extension is used as a datastore.\n\n* The Gps Data is sent to Realtime Processor. This is a simple Spring Service wrapped in a Hystrix Command. \n\n* REST interfaces are exposed for expected and actual routes including geojson. \n\n* Redis is used asa cache and also as a publish subscribe broker.\n \n* Browsers clients connect via SocketIO and subscribe for events for a given TripID\n\n## Background\n\nThe expected bus routes are loaded into PostGIS from GTFS files published by DublinBus. \nhttp://dublinked.com/datastore/datasets/dataset-254.php\nInstall gtfsdb: \nhttps://github.com/OpenTransitTools/gtfsdb\nLoad the files with: \n..\\gtfsdb\u003ebin\\gtfsdb-load --database_url postgresql://postgres@localhost:9995/dublinbus ..\\google_transit_dublinbus_P20130315-1546.zip\n\nAs gps data points arrive they are compared with the expected bus route. \nThere are two possible scenarios: \n1) Clean data: The gps data point vehicle journey id exists in expected routes and the timestamp is roughly sane. \n2) Dirty data: The expected journey is calculated as the closest in time to data point for the route number. \n\nIn both cases the data point is compared with expected and results are pushed via Redis pubsub and SocketIO to leaflet based clients. (Which need a bit of work)\n\n\n## Plan: \n\nBrowser to scrub data on date/time/coordinates and maybe area dimensions. \n 1) Map with realtime positions: User can draw a polygon to do a subselection. Maybe a heatmap here too?\n 2) Bar chart with historical lateness, cn scrub?\nUser can select a single bus/group of buses and view graph compaing expected and actual times and some history. \n\nAlerts based on bus bunching, etc","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankfarrell%2Fdublinbus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffrankfarrell%2Fdublinbus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffrankfarrell%2Fdublinbus/lists"}