{"id":17099347,"url":"https://github.com/mikeqfu/network-rail-track-fixity-layer","last_synced_at":"2025-09-02T22:32:59.532Z","repository":{"id":250357752,"uuid":"826282681","full_name":"mikeqfu/network-rail-track-fixity-layer","owner":"mikeqfu","description":"This project develops a data mining tool for analysing and predicting track movements using asset data, environmental factors and track design knowledge to model key parameters and generate fixity values for the GB rail network.","archived":false,"fork":false,"pushed_at":"2024-07-26T17:12:14.000Z","size":49892,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-29T00:31:19.030Z","etag":null,"topics":["data","data-integration","data-mining","data-science","information-management","knowledge-discovery","point-cloud","rail","rail-alignment","rail-track","track-fixity"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mikeqfu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-09T12:05:43.000Z","updated_at":"2024-07-26T17:20:54.000Z","dependencies_parsed_at":"2024-07-26T20:26:20.324Z","dependency_job_id":null,"html_url":"https://github.com/mikeqfu/network-rail-track-fixity-layer","commit_stats":null,"previous_names":["mikeqfu/network-rail-track-fixity-layer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeqfu%2Fnetwork-rail-track-fixity-layer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeqfu%2Fnetwork-rail-track-fixity-layer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeqfu%2Fnetwork-rail-track-fixity-layer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mikeqfu%2Fnetwork-rail-track-fixity-layer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mikeqfu","download_url":"https://codeload.github.com/mikeqfu/network-rail-track-fixity-layer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245145444,"owners_count":20568124,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","data-integration","data-mining","data-science","information-management","knowledge-discovery","point-cloud","rail","rail-alignment","rail-track","track-fixity"],"created_at":"2024-10-14T15:09:48.063Z","updated_at":"2025-03-23T18:20:50.924Z","avatar_url":"https://github.com/mikeqfu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Track Fixity Layer\n==================\n\nAuthors: [**Qian Fu**](https://research.birmingham.ac.uk/en/persons/qian-fu), [**John M. Easton**](https://research.birmingham.ac.uk/en/persons/john-easton) and [**Michael P. N. Burrow**](https://research.birmingham.ac.uk/en/persons/michael-burrow)\u003cbr\u003e\nProject support: **Sarah Jordan**\n\n\n## Introduction\n\nThis project aims to develop a data mining tool to assist engineers in analysing and predicting track movements over time. The analysis is based on recorded asset data, local operational and environmental factors and knowledge of track design in the area. In this prototype phase, the team focused on assembling a structured dataset for a representative track section to develop data-driven models of track movement, identify key track geometry parameters predicting future movements and generate fixity values for all reference curves used on the GB rail network.\n\n\n## Key achievements\n\nThe project has resulted in the following key achievements:\n- **Data integration and management**: A data cleaning and pre-processing workflow has been developed using the open-source PostgreSQL database. This workflow enables smooth integration and management of the data corpus. Initially, six industry data streams were used (CARRS, CNM, OPAS, ballast type, track quality, and point cloud surveys). The workflow is easily extensible, allowing for the inclusion of additional data sets in the future, provided they are referenced linearly (e.g., ELR + track mileage) or using standard geographic coordinate systems.\n- **Predictive machine learning model**: A machine learning model was trained on data from an 80-km section of the East Coast Main Line south of Edinburgh. This model predicts future track movements, with most predictions falling within the correct bin or within a single bin width of the true value. The model primarily used parameters such as curvature, cant, and maximum speed for its predictions.\n- **Python package development**: A three-module Python package has been developed to facilitate the rapid implementation of the described functionalities in software workflows.\n\nThese achievements lay the groundwork for further development and refinement in future phases of the project.\n\n\n## Technical Report\n\n- [Fu, Q.](https://research.birmingham.ac.uk/en/persons/qian-fu), [Easton, J. M.](https://research.birmingham.ac.uk/en/persons/john-easton) and [Burrow, M. P. N.](https://research.birmingham.ac.uk/en/persons/michael-burrow) (2021). Track Fixity Layer - Final Report. Approved by Network Rail Ltd. \\[[Appendix - Technical Documentation](docs/build/latex/track_fixity_layer.pdf)]\n\n\n## Conference/Workshop Posters\n\n- Clark Lecture 2024, University of Birmingham, Birmingham, 26 June 2024. \\[[Poster](docs/posters/3_Clark_Lecture_2024_Poster_Fu_et_al.pdf)]\n- Research Software Engineering in Data \u0026 AI Workshop, 15-17 February 2023, University of Warwick, Coventry. \\[[Poster](docs/posters/2_RSE_Data_AI_Workshop_2023_Fu_et_al.pdf)]\n- Transportation Research Board 102nd Annual Meeting, Washington, D.C., USA, 8–12 January 2023. \\[[Poster](docs/posters/1_TRBAM_2023_03682_Fu_et_al.pdf)]\n\n\n## Publication\n\n- [Fu, Q.](https://research.birmingham.ac.uk/en/persons/qian-fu), [Easton, J. M.](https://research.birmingham.ac.uk/en/persons/john-easton) and [Burrow, M. P. N.](https://research.birmingham.ac.uk/en/persons/michael-burrow) (2024). Development of an Integrated Computing Platform for Measuring, Predicting, and Analyzing the Profile-Specific Fixity of Railway Tracks. Transportation Research Record, 2678(6), 1-13. [doi:10.1177/03611981231191521](https://doi.org/10.1177/03611981231191521).\n\n\n## Demo\n\n\u003e **Note:** The following notebook code is for demonstration purposes only. Access to the project's database server will be necessary to successfully instantiate the prototype model. \n\n```python\n\u003e\u003e\u003e from src.modeller.prototype import TrackMovementEstimator\n\n\u003e\u003e\u003e # Specify element, direction and subsection length over which average movement is calculated\n\u003e\u003e\u003e element = 'Left Top'\n\u003e\u003e\u003e direction = 'Up'\n\u003e\u003e\u003e subsect_len = 10\n\n\u003e\u003e\u003e tme = TrackMovementEstimator(element=element, direction=direction, subsect_len=subsect_len)\n```\n    Initialising the feature collator ... Done.\n    Initialising the estimator ... Done.\n\n\n### 1. Create training and test data sets.\n\n```python\n\u003e\u003e\u003e tme.get_training_test_sets(test_size=0.2, random_state=1)\n```\n    Calculating track movement ... Done.\n    Collating features: \n      ballasts (src. Ballast summary) ... Done.\n      structures (src. CARRS and OPAS) ... Done.\n    Finalising the data integration ... Done.\n    Splitting into training and test sets ... Done.\n    The data is now ready for modelling.\n    \n```python\n\u003e\u003e\u003e tme.X_train\n```\n          Curvature    Cant  Max speed  ...  Retaining walls  Tunnels  Stations\n    3306   0.000000    0.00       90.0  ...                0        0         0\n    1910  -0.000858 -110.00       95.0  ...                0        0         0\n    1192  -0.000545  -89.01       80.0  ...                0        0         0\n    981   -0.000767 -130.00       95.0  ...                0        0         0\n    5139  -0.000256  -63.00      110.0  ...                0        0         0\n    ...         ...     ...        ...  ...              ...      ...       ...\n    1016   0.000000    0.00       95.0  ...                0        0         0\n    5907  -0.000504 -127.00      110.0  ...                0        0         0\n    4597  -0.001060 -125.00       85.0  ...                0        0         0\n    243   -0.000185  -21.42       95.0  ...                0        0         0\n    5869  -0.000394 -108.78      110.0  ...                0        0         0\n    \n    [5433 rows x 9 columns]\n\n```python\n\u003e\u003e\u003e tme.y_train\n```\n    2736    1\n    1578    3\n    1060    0\n    870     2\n    4512    3\n           ..\n    905     2\n    5192    2\n    3980    2\n    235     1\n    5157    3\n    Name: lateral_displacement_mean, Length: 5433, dtype: int64\n\n```python\n\u003e\u003e\u003e tme.X_test\n```\n          Curvature    Cant  Max speed  ...  Retaining walls  Tunnels  Stations\n    5628   0.000731  103.38      100.0  ...                0        0         0\n    5696   0.000955  135.00      100.0  ...                0        0         0\n    5431   0.000960  142.00      100.0  ...                0        0         0\n    3471  -0.000784 -108.50       90.0  ...                0        0         0\n    2908  -0.001264 -155.00       85.0  ...                0        0         0\n    ...         ...     ...        ...  ...              ...      ...       ...\n    247   -0.000337  -39.00       95.0  ...                0        0         0\n    3904   0.000000    0.00      125.0  ...                0        0         0\n    1054   0.000000    0.00       95.0  ...                0        0         0\n    3191  -0.000858 -156.00       90.0  ...                0        0         0\n    4026  -0.000271  -92.71      125.0  ...                0        0         0\n    \n    [1359 rows x 9 columns]\n\n```python\n\u003e\u003e\u003e tme.y_test\n```\n    4958    3\n    4999    0\n    4793    1\n    2901    2\n    2394    2\n           ..\n    239     2\n    3309    2\n    943     0\n    2621    3\n    3431    3\n    Name: lateral_displacement_mean, Length: 1359, dtype: int64\n\n### 2. Train an RF model\n\n```python\n\u003e\u003e\u003e tme.classifier(n_estimators=300, max_depth=15, oob_score=True, n_jobs=8)\n```\n    Model training in process ... \n    [Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n    [Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.1s\n    [Parallel(n_jobs=8)]: Done 300 out of 300 | elapsed:    0.3s finished\n    Done.\n    \n    Testing accuracy ... \n    [Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n    [Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 300 out of 300 | elapsed:    0.0s finished\n    Done.\n    \n    Mean accuracy: 49.96%\n\n                        Importance\n    Curvature            0.388747\n    Cant                 0.372477\n    Max speed            0.201602\n    Underline bridges    0.009513\n    Overline bridges     0.007261\n    Max axle load        0.006700\n    Retaining walls      0.006048\n    Tunnels              0.005787\n    Stations             0.001866\n\n### 3. Create a confusion matrix\n\n#### 3.1 View the confusion matrix for the trained RF model\n\n```python\n\u003e\u003e\u003e tme.view_confusion_matrix()\n```\n    [Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n    [Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 300 out of 300 | elapsed:    0.0s finished\n\n\u003cfigure\u003e\n  \u003cimg src=\"demos/images/tme_view_confusion_matrix.svg\" alt=\"Confusion matrix\" width=\"725\"/\u003e\n\u003c/figure\u003e\n\n#### 3.2 Normalise the confusion matrix over the predicted labels\n\n```python\n\u003e\u003e\u003e tme.view_confusion_matrix(normalise='pred')\n```\n    [Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n    [Parallel(n_jobs=8)]: Done  34 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 184 tasks      | elapsed:    0.0s\n    [Parallel(n_jobs=8)]: Done 300 out of 300 | elapsed:    0.0s finished\n    \n\u003cfigure\u003e\n  \u003cimg src=\"demos/images/tme_view_confusion_matrix_norm_pred.svg\" alt=\"Confusion matrix (normalise=pred)\" width=\"725\"/\u003e\n\u003c/figure\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikeqfu%2Fnetwork-rail-track-fixity-layer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmikeqfu%2Fnetwork-rail-track-fixity-layer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmikeqfu%2Fnetwork-rail-track-fixity-layer/lists"}