{"id":42503792,"url":"https://github.com/dataforgoodfr/batch9_geowatchlabs-3-markets","last_synced_at":"2026-01-28T13:37:31.725Z","repository":{"id":52423425,"uuid":"359734917","full_name":"dataforgoodfr/batch9_geowatchlabs-3-markets","owner":"dataforgoodfr","description":null,"archived":false,"fork":false,"pushed_at":"2021-06-28T17:52:47.000Z","size":35337,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-24T10:06:57.405Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dataforgoodfr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-20T08:08:21.000Z","updated_at":"2021-06-28T17:52:50.000Z","dependencies_parsed_at":"2022-08-18T18:21:09.256Z","dependency_job_id":null,"html_url":"https://github.com/dataforgoodfr/batch9_geowatchlabs-3-markets","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dataforgoodfr/batch9_geowatchlabs-3-markets","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataforgoodfr%2Fbatch9_geowatchlabs-3-markets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataforgoodfr%2Fbatch9_geowatchlabs-3-markets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataforgoodfr%2Fbatch9_geowatchlabs-3-markets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataforgoodfr%2Fbatch9_geowatchlabs-3-markets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dataforgoodfr","download_url":"https://codeload.github.com/dataforgoodfr/batch9_geowatchlabs-3-markets/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dataforgoodfr%2Fbatch9_geowatchlabs-3-markets/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28846053,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T13:02:32.985Z","status":"ssl_error","status_checked_at":"2026-01-28T13:02:04.945Z","response_time":57,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-28T13:37:31.110Z","updated_at":"2026-01-28T13:37:31.715Z","avatar_url":"https://github.com/dataforgoodfr.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eGeoWatch Labs - Market\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n\n## Table of Contents\n\n- [Table of Contents](#table-of-contents)\n- [Getting started](#getting-started)\n  - [Repository structure](#repository-structure)\n    - [External data required](#external-data)\n    - [Config](#config)\n    - [Import functions](#import-functions)\n    - [Preprocessing functions](#preprocessing-functions)\n    - [Utils](#utils)\n- [About the project](#about-the-project)\n  - [Goal](#goal)\n  - [Data sources](#data-sources)\n  - [Scores](#scores)\n- [Data exploration](#data-exploration)\n  - [FSMS](#fsms)\n  - [Geospatial data](#geospatial-data)\n  - [Prices](#prices)\n  \n\n\n\n____________________________________________________________________\n\n## Getting started\n\n### Repository structure\n\nOur repository structure is the following :\n\n```\n├── config\n|    ├── aggregation.py\n|    └── preprocessing.py\n├── import_functions\n|    ├── automatic_analysis_and_aggregation.py\n|    ├── auxiliary_function_for_geo_files.py\n|    ├── auxiliary_function_for_importing_data.py\n|    └── manual_aggregation.py\n├── notebooks\n|    ├── faodata.ipynb\n|    ├── geoviz.ipynb\n|    ├── mean_of_communes_yield.ipynb\n|    └── study_data_types_preprocessing.ipynb\n├── preprocessing\n|    └── preprocessing.py\n└── utils\n    └── tiff_to_geojson.py\n```\n\n#### External data required\n\nPlease download \"Groupe 3 - Marchés Alimentaires\", \"Mauritania FSMS data\" and \"Communes.geojson\" in the ./src folder before running main.py\n  \n#### Config\n\nConfig folder is about all variables we setup for the project.\n\n\n#### Import functions\n\nThose are the functions extracting data from the zipfile and aggregating it into a csv file. \n\nThe most important code here is `manual_aggregation`, reading all files from the main zip, unzipping them and standardizing \ncolumn names according to manual inputs the team made in a csv, and aggregating it.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://github.com/dataforgoodfr/batch9_geowatchlabs-3-markets/blob/main/doc/manual_aggregation_readme.png\"/\u003e\n\u003c/p\u003e\n\nIt generates a standardized dataframe, and a metadata dataframe.\n\n⚠️ `manual_aggregation` may seem similar to `automatic_analysis_and_aggregation`, but it's not. `automatic_analysis_and_aggregation`\n aims at matching columns based on levenstein distance.\n\n#### Preprocessing functions\n\nThose are the functions reading the csv file and imputing missing values, standardizing data types and data before \nanalysis and clustering. \n\n#### Utils\n\nAdditional functions that can be useful for some operations.\n\n\n____________________________________________________________________\n\n\n## About the project\n\n### Goal\n\nStudy the influence of food price variations and other economic considerations (import, export, etc.) on food insecurity \nin the South agropastoral zone of Mauritania. \n\nThe goal is to find a relationship that enables us to accurately estimate food insecurity situation there.\n\nStudied moughataas, by region :\n\n| region  | moughataas |\n|---------|------------|\n|\u003cp align=\"center\"\u003e Trarza | \u003cp align=\"center\"\u003e  Ouad-Naga, Boutilimit |\n|\u003cp align=\"center\"\u003e Brakna | \u003cp align=\"center\"\u003e  Magta-Lahjar, Aleg |\n|\u003cp align=\"center\"\u003e Tagant | \u003cp align=\"center\"\u003e  Moudjeria |\n|\u003cp align=\"center\"\u003e Assaba | \u003cp align=\"center\"\u003e  Boumdeïd, Kankossa, Guerou, Barkeol, Kiffa |\n|\u003cp align=\"center\"\u003e Gorgol | \u003cp align=\"center\"\u003e  M'Bout, Monguel |\n|\u003cp align=\"center\"\u003e Guidimaka | \u003cp align=\"center\"\u003e  Ould Yenge, Selibaby |\n|\u003cp align=\"center\"\u003e Hodh el Gharbi | \u003cp align=\"center\"\u003e  Tamchakett, Aïoun, Kobenni, Tintane |\n|\u003cp align=\"center\"\u003e Hodh ech Chargui | \u003cp align=\"center\"\u003e  Timbédra, Néma, Bassikounou, Amourj, Djiguenni |\n\n### Data sources\n\nThree categories of data sources will be used in this project :\n - FSMS data \n - Geospatial data \n - Prices data \n\nSee [Data exploration](#data-exploration) for more information.\n\n### Scores\n\n____________________________________________________________________\n\n## Data exploration\n\n### FSMS\n\nMake sure to have `Mauritania FSMS data.zip` in your `$HOME`. The directory `Mauritania FSMS data` looks like this : \n\n```\nMauritania FSMS data\n├── year\n│    ├── monthYY\n│    │     ├── data.sav\n│    │     ├── pdf_1\n│    │     └── pdf_2\n[...]\n```\n\nWe will here use the `.sav` files for our analysis. The script `Mauritania_FSMS_aggregation.py` aims at normalizing \nthe data in this folder.\n\n### Geospatial data\n\n- GeoJSON of the studied zone\n- .tiff : births, population, roads, railways ... (uploaded by Simon on the Google Drive)\n\n### Prices\n\nHere are the different pricing sources that we've collected so far :\n     \n\n| link  | indice | type of data  | frequency  | geographic level  | who added this link |\n|-------|--------|---------------|------------|-------------------|---------------------|\n| [World Food Program](https://data.humdata.org/dataset/wfp-food-prices-for-mauritania?force_layout=desktop) |    | price by crop | yearly | market  | Lucas  |\n| [World Bank](https://data.worldbank.org/country/MR) |    |    |   | country  | Lucas  |\n| [FAO](https://fpma.apps.fao.org/giews/food-prices/tool/public/#/dataset/domestic) |    |    |   |   | Thibaut  |\n| [Food security portal](https://api.foodsecurityportal.org/organization/food-security-portal) |    |  |   |   | Simon  |\n| [FAO prices in Africa (.zip in Slack)](https://data-for-good.slack.com/archives/C01UPA0HKCY/p1618946330008000) |  |   | yearly  | country  | Hadrien  |\n                     \n\nThe column \"indice\" represent the pricing indice that can be computed with this data source (CPI, price of a typical food basket, price of basic food ...)\n\n\u003cu\u003e\u003cb\u003eNB :\u003c/b\u003e\u003c/u\u003e the column \"who added this link\" is here in case you want to ask questions about the source to the person who found it\n\nWe used the [World Food Program] source as it is the most granular and comprehensive for Mauritania.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataforgoodfr%2Fbatch9_geowatchlabs-3-markets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdataforgoodfr%2Fbatch9_geowatchlabs-3-markets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdataforgoodfr%2Fbatch9_geowatchlabs-3-markets/lists"}