{"id":35039507,"url":"https://github.com/xapple/forest_puller","last_synced_at":"2025-12-27T08:11:25.353Z","repository":{"id":57431894,"uuid":"234941204","full_name":"xapple/forest_puller","owner":"xapple","description":"  This python package lets you automatically download and parse european forest data from several different public sources.","archived":false,"fork":false,"pushed_at":"2022-05-19T14:13:43.000Z","size":6517,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-12-24T23:38:49.879Z","etag":null,"topics":["forest-inventory","open-data","python-pandas","visualization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xapple.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-01-19T17:55:39.000Z","updated_at":"2025-12-08T15:06:07.000Z","dependencies_parsed_at":"2022-09-02T11:01:40.207Z","dependency_job_id":null,"html_url":"https://github.com/xapple/forest_puller","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/xapple/forest_puller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xapple%2Fforest_puller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xapple%2Fforest_puller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xapple%2Fforest_puller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xapple%2Fforest_puller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xapple","download_url":"https://codeload.github.com/xapple/forest_puller/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xapple%2Fforest_puller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28075902,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-12-27T02:00:05.897Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["forest-inventory","open-data","python-pandas","visualization"],"created_at":"2025-12-27T08:11:22.322Z","updated_at":"2025-12-27T08:11:25.342Z","avatar_url":"https://github.com/xapple.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![PyPI version](https://badge.fury.io/py/forest-puller.svg)](https://badge.fury.io/py/forest-puller)\n[![GitHub last commit](https://img.shields.io/github/last-commit/xapple/forest_puller.svg)](https://github.com/xapple/forest_puller/commits/master)\n[![GitHub](https://img.shields.io/github/license/xapple/forest_puller.svg)](https://github.com/xapple/forest_puller/blob/master/LICENSE)\n![Continuous testing](https://github.com/xapple/forest_puller/workflows/Continuous%20testing/badge.svg)\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"docs/title_art/title.jpeg?raw=true\"\u003e\n\u003c/p\u003e\n\n`forest_puller` is a python package for retrieving data concerning forests of European countries. This includes the amount of forested areas, the forest inventory (standing stock), the forest growth rates as well as the forest loss dynamics (disturbances).\n\nThis software is accompanied by a scientific publication that is available here: https://www.mdpi.com/1999-4907/12/2/176\n\nThere are several forest public data sources accessible online that provide these types of information in various forms and granularity. This package automates the process of scrapping these websites and parsing the resulting CSV tables or excel files.\n\nOnce `forest_puller` is installed you can easily access forest data through standard python pandas data frames.\n\n\u003cp align=\"center\"\u003e\n\u003cimg height=\"300\" src=\"forest_puller/reports/template/logo.png?raw=true\"\u003e\n\u003c/p\u003e\n\n## Scope and sources\n\nCurrently `forest_puller` provides data for the following 27 EU member states (past and present):\n\n* Austria, Belgium, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, United Kingdom\n\nCurrently `forest_puller` caches and provides programmatic access to the forest-relevant data from these data sources:\n\n* IPCC (https://tinyurl.com/y474yu9e)\n* SOEF (https://dbsoef.foresteurope.org/)\n* FAOSTAT (http://www.fao.org/faostat/en/)\n* HPFFRE (https://doi.org/10.5061/dryad.4t880qh)\n* FRA (http://www.fao.org/forest-resources-assessment/en/)\n\nWhat other data source you would like to see added here? Contact the authors by opening an issue in the [the tracker](https://github.com/xapple/forest_puller/issues).\n\n## Prerequisites\n\nSince `forest_puller` is written in python, it is compatible with all operating systems: Linux, macOS and Windows. The only prerequisite is `python3` (which is often installed by default) along with the `pip3` package manager.\n\nTo check if you have `python3` installed, type the following on your terminal:\n\n    $ python3 -V\n\nIf you do not have `python3` installed, please refer to the section [obtaining python3](docs/markdown/installing_tips.md#obtaining-python3).\n\nTo check if you have `pip3` installed, type the following on your terminal:\n\n    $ pip3 -V\n\nIf you do not have `pip3` installed, please refer to the section [obtaining pip3](docs/markdown/installing_tips.md#obtaining-pip3).\n\n## Installing\n\nSimply type the following on your terminal:\n\n    $ pip3 install --user forest_puller\n\nOr if you want to install it for all users of the system:\n\n    $ sudo pip3 install forest_puller\n\n## Usage\n\nFor instance, to retrieve the net carbon dioxide emission of Austria in 2017 that were due to *coniferous* forest land from the IPCC official data source, you can use the following python code:\n\n```python\n# Import #\nfrom forest_puller.ipcc.country import countries\n\n# Get the country #\naustria = countries['AT']\n\n# Get the 2017 indexed dataframe #\nat_2017 = austria.years[2017].indexed\n\n# Print some data #\nprint(at_2017.loc['remaining_forest', 'Coniferous']['net_co2'])\n```\n\n     904282.4970403439\n\nTo see what other information is available, you can of course display the column titles and row indexes of the data frame at hand:\n\n```python\nprint(at_2017.columns)\n\n# Index(['area', 'area_mineral', 'area_organic', 'biomass_gains_ratio',\n#        'biomass_losses_ratio', 'biomass_net_change_ratio', 'net_dead_ratio',\n#        'net_litter_ratio', 'net_mineral_soil_ratio', 'net_organic_soil_ratio',\n#        'biomass_gains', 'biomass_losses', 'biomass_net_change', 'net_dead',\n#        'net_litter', 'net_mineral_soils', 'net_organic_soils', 'net_co2'],\n#        dtype='object', name='category')\n\nprint(at_2017.index)\n\n# MultiIndex(levels=[['cropland_to_forest', 'grassland_to_forest',\n# 'land_to_forest', 'other_land_to_forest', 'remaining_forest',\n# 'settlements_to_forest', 'total_forest', 'wetlands_to_forest'],\n# ['', 'Coniferous', 'Deciduous', 'Forest not in yield', 'Total']])\n```\n\nTo examine what countries and what years are available:\n\n```python\nprint(list(c.iso2_code for c in countries.values()))\n\n# ['AT', 'BE', 'BG', 'HR', 'CZ', 'DK', 'EE', 'FI', 'FR', 'DE', 'GR',\n# 'HU', 'IE', 'IT', 'LV', 'LT', 'LU', 'NL', 'PL', 'PT', 'RO', 'SK', 'SI',\n# 'ES', 'SE', 'GB', 'ZZ']\n\nprint(list(y for y in austria.years))\n# [1990, 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000,\n# 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012,\n# 2013, 2014, 2015, 2016, 2017]\n```\n\nTo get a large data frame with all years and all countries inside:\n\n```python\nfrom forest_puller.ipcc.concat import df\nprint(df)\n```\n\nTo get a data frame that compares forest area between several sources:\n\n```python\nfrom forest_puller.viz.area_comp import area_comp_data\nprint(area_comp_data.df)\n```\n\nTo get a data frame that compares gains (increments) and losses (harvest) between sources:\n\n```python\nfrom forest_puller.viz.converted_to_tons import converted_tons_data\nprint(converted_tons_data.df)\n```\n\n\n## Cache\n\nWhen you import `forest_puller`, we will check the `$FOREST_PULLER_CACHE` environment variable to see where to download and store the cached data. If this variable is not set, we will default to `~/.forest_puller` and clone a repository there.\n\n## Data sources\n\n### IPCC\n\nTo access the same forest data directly from the IPCC website without the use of `forest_puller`, you would have to first select your country of interest from the CRF country table in a browser at [this address](https://tinyurl.com/y474yu9e).\n\n![IPCC demo screenshot 1](docs/ipcc/ipcc_demo_1.png?raw=true \"IPCC demo screenshot 1\")\n\nThen you would have to manually download the `zip` file for that specific country through another page.\n\n![IPCC demo screenshot 2](docs/ipcc/ipcc_demo_2.png?raw=true \"IPCC demo screenshot 2\")\n\nNext, you would have to uncompress the `zip` file and locate the `xls` file that concerns the year you are interested in.\n\n![IPCC demo screenshot 4](docs/ipcc/ipcc_demo_4.png?raw=true \"IPCC demo screenshot 4\")\n\nFinally you would have to scroll to the right sheet in your spreadsheet software and find the pertinent cell.\n\n![IPCC demo screenshot 5](docs/ipcc/ipcc_demo_5.png?raw=true \"IPCC demo screenshot 5\")\n\nThis operation would have to be repeated for every country, and every year you are interested in.\n\nWith `forest_puller` you can easily display any information you want for all countries at the same time:\n\n```python\nfrom forest_puller.ipcc.country import countries\n\ncategory, key = ['total_forest', 'biomass_net_change']\nbiomass_net_change = {\n    k: c.last_year.indexed.loc[category, ''][key]\n    for k,c in countries.items()\n}\n\nimport pprint\npprint.pprint(biomass_net_change)\n```\n\n    {'AT': 1367857.0940855271,\n     'BE': 374245.08695361385,\n     'BG': 2192942.031982918,\n     'CZ': 387870.89395249996,\n     'DE': 12317598.87352293,\n     'DK': -216454.31026543948,\n     'EE': 320710.2459538891,\n     'ES': 8917649.261547482,\n     'FI': 6603815.0,\n     'FR': 15051831.9827214,\n     'GB': 2892518.0859005335,\n     'GR': 583205.0978272819,\n     'HR': 1477791.7578513895,\n     'HU': 1259385.5890665338,\n     'IE': 1069648.7636722159,\n     'IT': 5752883.095908434,\n     'LT': 2146933.309581986,\n     'LU': 101929.37461705346,\n     'LV': 1244965.2120000012,\n     'NL': 499021.93968,\n     'PL': 9353198.2907701,\n     'PT': 1536917.4736652463,\n     'RO': 5561343.4405591395,\n     'SE': 10185839.738999998,\n     'SI': 35391.09710503432,\n     'SK': 1184611.3471376207}\n\n### Forest Europe (SOEF)\n\nThis data is provided by the \"Ministerial Conference on the Protection of Forests in Europe\" and is accessible at: https://dbsoef.foresteurope.org/\n\nThree tables are provided for every country:\n\n* Table 1.1a: Forest area\n* Table 1.3a1: Age class distribution (area of even-aged stands)\n* Table 3.1: Increment and fellings\n\nIt is accessed in a similar way to other data sources:\n\n```python\nfrom forest_puller.soef.country import countries\n\ncountry = countries['AT']\nprint(country.forest_area.indexed)\nprint(country.age_dist.indexed)\nprint(country.fellings.indexed)\n```\n\nThere is also a large data frame containing all countries concatenated together:\n\n```python\nfrom forest_puller.soef.concat import tables\nprint(tables['forest_area'])\nprint(tables['age_dist'])\nprint(tables['fellings'])\n```\n\n### Faostat (forestry)\n\nThis data is acquired by picking the \"All Data Normalized\" option from the \"Bulk download\" sidebar at this address: http://www.fao.org/faostat/en/#data/FO\n\nIt is accessed in a similar way to other data sources:\n\n```python\nfrom forest_puller.faostat.forestry.country import countries\n\ncountry = countries['AT']\nprint(country.df)\n```\n\nThere is also a large data frame containing all countries concatenated together:\n\n```python\nfrom forest_puller.faostat.forestry.concat import df\nprint(df)\n```\n\n### Faostat (land)\n\nThis data is acquired by picking the \"All Data Normalized\" option from the \"Bulk download\" sidebar at this address: http://www.fao.org/faostat/en/#data/GF\n\nIt is accessed in a similar way to other data sources:\n\n```python\nfrom forest_puller.faostat.land.country import countries\n\ncountry = countries['AT']\nprint(country.df)\n```\n\nThere is also a large data frame containing all countries concatenated together:\n\n```python\nfrom forest_puller.faostat.land.concat import df\nprint(df)\n```\n\n### HPFFRE\n\nDiabolo was a project run by a consortium of 33 partners from 25 countries. Experts in the fields of policy analysis, forest inventory, forest modelling. 7 work packages.\n\n Link: http://diabolo-project.eu/\n\nOne of the outcomes of the Diabolo project is the following publication:\n\nVauhkonen et al. 2019 - [Harmonised projections of future forest resources in Europe](https://doi.org/10.1007/s13595-019-0863-6)\n\nAbbreviated \"hpffre\". The authors used EFDM (mainly) to project forest area, growing stock, fellings and above ground carbon for European countries. There are several scenario outcomes.\n\nThe dataset is available at: https://doi.org/10.5061/dryad.4t880qh\n\nIt is accessed in a similar way to other data sources:\n\n```python\nfrom forest_puller.hpffre.country import countries\n\ncountry = countries['AT']\nprint(country.df)\n```\n\nThere is also a large data frame containing all countries concatenated together:\n\n```python\nfrom forest_puller.hpffre.concat import df\nprint(df)\n```\n\n### FRA\n\nFRA stands for \"Forest Resource Assessment\".\n\nThe data is acquired by picking the \"CSV download\" option from the page located at: http://countrystat.org/home.aspx?c=FOR\u0026tr=3\n\nThe home page is at: http://www.fao.org/forest-resources-assessment/en/\n\nFive datasets are provided in the package:\n\n* \"Forest characteristics (1 000 ha) by FRA categories\"\n* \"Extent of forest and other wooded land (1 000 ha)\"\n* \"Forest establishment total (ha/yr) by FRA categories\"\n* \"Carbon stock (Million metric tonnes) by Forest/Other wooded land\"\n* \"Biomass stock (Million metric tonnes) by Forest/Other wooded land\"\n\nFRA is an effort of the Food and Agriculture Organization of the United Nations (FAO)\".\n\nIt is accessed in a similar way to other data sources:\n\n```python\nfrom forest_puller.fra.country import countries\n\ncountry = countries['AT']\nprint(country.df)\n```\n\nThere is also a large data frame containing all countries concatenated together:\n\n```python\nfrom forest_puller.fra.concat import df\nprint(df)\n```\n\n## Visualizations\n\nThe `forest_puller` package can also generate several plots that enable the user to compare and visualize the data.\n\nFor instance here is are a series of graphs comparing the total reported forest area between data sources as seen in the `forest_puller.viz.area` submodule:\n\n![Comparison of total forest area](docs/viz/area/area.svg?sanitize=true \"Comparison of total forest area\")\n\nAnother type of graph that can be produced is the comparison of gains and losses across several data-sources and across countries. This code is found in the `forest_puller.viz.increments` submodule and shows the five largest countries in terms of forest area.\n\n![Comparison of increments for SE](docs/viz/increments/SE.svg?sanitize=true \"Comparison of increments for SE\")\n![Comparison of increments for FR](docs/viz/increments/FR.svg?sanitize=true \"Comparison of increments for FR\")\n![Comparison of increments for FI](docs/viz/increments/FI.svg?sanitize=true \"Comparison of increments for FI\")\n![Comparison of increments for ES](docs/viz/increments/ES.svg?sanitize=true \"Comparison of increments for ES\")\n![Comparison of increments for DE](docs/viz/increments/DE.svg?sanitize=true \"Comparison of increments for DE\")\n\nWith data from the SOEF source, we can also plot a breakdown of the growing stock volume genus composition of many countries across time. This code is found in the `forest_puller.viz.genus_barstack` submodule.\n\n![Comparison of genus breakdown](docs/viz/genus/AT_BE_BG_HR_CY.svg?sanitize=true \"Comparison of genus breakdown\")\n![Comparison of genus breakdown](docs/viz/genus/CZ_DK_EE_FI_FR.svg?sanitize=true \"Comparison of genus breakdown\")\n![Comparison of genus breakdown](docs/viz/genus/DE_HU_IE_IT_LV.svg?sanitize=true \"Comparison of genus breakdown\")\n![Comparison of genus breakdown](docs/viz/genus/LT_NL_PL_PT_RO.svg?sanitize=true \"Comparison of genus breakdown\")\n![Comparison of genus breakdown](docs/viz/genus/SK_SI_ES_SE_GB.svg?sanitize=true \"Comparison of genus breakdown\")\n![Genera legend](docs/viz/genus/legend.svg?sanitize=true \"Genera legend\")\n\n## Auto-generated report\n\nAll the visualization produced by the package are assembled inside an auto-generated PDF report. You can download a cached version of this report here:\n\n\u003cp align=\"center\"\u003e\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n        \u003ca href=\"https://xapple.github.io/forest_puller/reports/comparison.pdf\"\u003e\n        \u003cimg src=\"docs/images/pdf_icon.png\" width=\"120em\"\u003e\n        \u003c/a\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n  \u003ctr\u003e\n     \u003ctd\u003e\n        \u003ca href=\"https://xapple.github.io/forest_puller/reports/comparison.pdf\"\u003e\n            \u003cp align=\"center\"\u003eComparison report\u003c/p\u003e\n        \u003c/a\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\u003c/p\u003e\n\n\n## Reporting issues\n\nIf you encounter an error when using the `forest_puller` package, please open an issue on [the tracker](https://github.com/xapple/forest_puller/issues) and we will get in contact with you.\n\n## Extra documentation\n\nMore documentation is available at:\n\n\u003chttp://xapple.github.io/forest_puller/forest_puller\u003e\n\nThis documentation is simply generated from the source code with:\n\n    $ pdoc --html --output-dir docs --force forest_puller","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxapple%2Fforest_puller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxapple%2Fforest_puller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxapple%2Fforest_puller/lists"}