Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/druimalban/ukclimate
Parse raw historical Met Office weather station data
https://github.com/druimalban/ukclimate
haskell parsec recursion-schemes
Last synced: 5 days ago
JSON representation
Parse raw historical Met Office weather station data
- Host: GitHub
- URL: https://github.com/druimalban/ukclimate
- Owner: druimalban
- License: other
- Created: 2019-03-16T20:09:01.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2023-12-01T23:47:00.000Z (about 1 year ago)
- Last Synced: 2024-11-07T18:19:25.371Z (about 2 months ago)
- Topics: haskell, parsec, recursion-schemes
- Language: Haskell
- Homepage:
- Size: 19.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ukclimate - Parse raw historical Met Office weather station data
The Met Office provides a number of datasets for public benefit.
These largely relate to a number of historical measurements at various sites
around the UK.I wanted to use this data, but because it is largely not provided in a format
which I could easily process, like CSV, this became difficult, and I had to
manually parse it.The files are designed to be manually copied into a spreadsheet program.
These are separated by regular characters, but the format is not consistent
across years/locations, and there are additionally comments throughout.
I believe these are updated manually by Met Office staff.
The library and sample client is able to serialise most if not all examples of
historical station data in JSON and CSV available from the Met Office, at the
time of writing. Needless to say, this is a work in progress.I’m mostly writing this as a learning exercise for Haskell, particularly as
regards to parser combinators, as well as using the wonderful recursion-schemes
library, and lenses.Note: make sure `~/.ukclimate/data` is a real directory with sites.json in it.
You can make a symbolic link to the path of `contrib` in the source directory for
instance.-----
Implemented:
* Parse a given file into a general representation of historic data and regional time-ordered series
* Download and cache files from the Met Office
* Cache results from already-parsed files
* Debug annotations provided by attoparsec, making clear where failure occurs
To-do:
* Parsing regional sets that are rank-ordered rather than time-ordered (it's very similar data)
* Improve caching of results