{"id":27334922,"url":"https://github.com/dmuth/septastats","last_synced_at":"2025-04-12T14:46:39.045Z","repository":{"id":42952664,"uuid":"55027852","full_name":"dmuth/SeptaStats","owner":"dmuth","description":"Real-time dashboards, graphs, and analytics for Philadelphia's Regional Rail train system.","archived":false,"fork":false,"pushed_at":"2022-09-30T19:35:31.000Z","size":3801,"stargazers_count":15,"open_issues_count":9,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-05-02T06:07:38.604Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://www.septastats.com/","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dmuth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":"dmuth"}},"created_at":"2016-03-30T03:00:55.000Z","updated_at":"2022-04-17T01:17:09.000Z","dependencies_parsed_at":"2023-01-18T16:31:12.137Z","dependency_job_id":null,"html_url":"https://github.com/dmuth/SeptaStats","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmuth%2FSeptaStats","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmuth%2FSeptaStats/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmuth%2FSeptaStats/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dmuth%2FSeptaStats/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dmuth","download_url":"https://codeload.github.com/dmuth/SeptaStats/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248585249,"owners_count":21128974,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-12T14:46:38.336Z","updated_at":"2025-04-12T14:46:39.037Z","avatar_url":"https://github.com/dmuth.png","language":"PHP","funding_links":["https://github.com/sponsors/dmuth"],"categories":[],"sub_categories":[],"readme":"# Septa Stats\n\nSource code for http://www.SeptaStats.com/\n\nThis is a Splunk app which downloads Regional Rail train stats from SEPTA's public API\nonce per minute and contains dashboards and reports for visualizing that data.\n\n\u003cimg src=\"https://raw.githubusercontent.com/dmuth/SeptaStats/master/img/septa-stats-1.jpg\" width=\"260\" /\u003e \u003cimg src=\"https://raw.githubusercontent.com/dmuth/SeptaStats/master/img/septa-stats-2.jpg\" width=\"260\" /\u003e \u003cimg src=\"https://raw.githubusercontent.com/dmuth/SeptaStats/master/img/septa-stats-3.jpg\" width=\"260\" /\u003e\n\n\n## Installation Instructions\n\n- Clone this repo\n- Make sure Docker is installed\n- Run `docker-compose up -d` to start all the Docker containers\n- Splunk can be found at http://localhost:8001/, login is admin/password.  \n   - This instance should NOT be made available over the Internet.  Seriously. Don't do it.\n- The web app can be found at http://localhost:8002/ \n   - I recommend blocking this port off from the Internet and using your own instance of Nginx to proxy to it, with HTTPS.\n- AWS credentials should be in `aws-credentials.txt`\n   - Note that if you create a policy for your IAM credentials (as you should!), the ARN in the Resource array must end in `/*`. Exammple: `arn:aws:s3:::septa-stats/*`. There is a bug in the policy generator where this won't happen and your backups will fail. Be careful.\n\n\n\n## Architecture Overview\n\nThe Docker containers in this project are as follows:\n\n- `splunk`: Runs a PHP script to fetch from SEPTA's API every 55 seconds, and saves the data to a Splunk Index.\n  - Configuration of Splunk is saved in `splunk-files/`.\n- `web`: Nginx webserver\n- `php`: PHP running in FCGI mode to run PHP code\n- `redis`: Used to cache the results of Splunk queries\n- `backup`: Make regular backups of train data to AWS S3\n\n\n## Development\n\n\n### Working on PHP Code\n\n- `cp docker-compose.yml.example docker-compose.yml`\n- `docker-compose up`\n- PHP code lives under `htdocs/`\n- Go to https://localhost:8000/ in your web browser for Splunk. Password is in `docker-compose.yml`.\n- Go to http://localhost:8001/ in your web browser to view the website\n\n\n### Working on Splunk\n\nThis will stand up Splunk and spawn a shell in it:\n```\ndocker-compose kill splunk; \\\n\tdocker-compose rm -f splunk; \\\n\tdocker-compose build splunk; \\\n\tdocker-compose up -d splunk; \\\n\tdocker-compose exec splunk bash\n```\n\nTest the PHP script inside the container with:\n- `/opt/splunk/etc/apps/septa-stats/bin/get-regional-rail-trainview.php | jq .`\n\n\n### How to run the `backup` script in the foreground for development:\n\n```\ndocker-compose kill backup \\\n\t\u0026\u0026 docker-compose rm -f backup \\\n\t\u0026\u0026 docker-compose build backup \\\n\t\u0026\u0026 docker-compose up backup\n```\n\n\n## Deployment\n\n- SSH to the webserver\n- `cd /var/septa-stats/`\n- `git pull`\n- If you're doing an HTML or PHP update, you're done. Otherwise, restart the affected containers.\n\n\n## Exporting Data\n\nThe Spunk Index is written to `splunk-data/`, so that when the container is restarted, no data is lost.\n\nThe Redis data is written to `redis-data/`, so that no data is lost when that container is restarted.\n\n\n## Importing Data\n\nAssuming you exported the raw events from Splunk to a text file, you can import those events with these commands:\n\n- `docker-compose exec splunk bash`\n- `/opt/splunk/bin/splunk add oneshot ./septa-stats.txt -sourcetype oneshot -index septa_analytics`\n\n\n## FAQ: Why is Composer's vendor/ directory included?\n\nI'm a huge fan of reproducable builds, and I'm a little concerned about the possibility of \u003ca href=\"http://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/\"\u003ethings like this happening\u003c/a\u003e, \nwhich can cause both large amounts of breakage and application build failures.\n\n**That said**, I've been having some conversations with folks on r/PHP and other places, and while \nthey acknowledged that my concern is legitimate, the risk is very low, and including vendor/\njust adds so much more noise to code reviews.  That's a valid point, and I am reconsidering my \ndecision to include the vendor/ directory.  Feel free to reach out to me if you feel strongly\none way or the other on this issue.\n\n\n## Credits\n\nThis package uses the `timewrap` command, available at \u003ca href=\"https://splunkbase.splunk.com/app/1645/\"\u003ehttps://splunkbase.splunk.com/app/1645/\u003c/a\u003e.\n\n\n## Additional Questions?\n\nMore Questions?  Please email me at **dmuth@dmuth.org**, **doug.muth@gmail.com**, or open an issue here.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmuth%2Fseptastats","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdmuth%2Fseptastats","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdmuth%2Fseptastats/lists"}