{"id":19440477,"url":"https://github.com/devinit/fts_precodinglists","last_synced_at":"2026-06-06T23:32:31.700Z","repository":{"id":72842338,"uuid":"248226277","full_name":"devinit/fts_precodinglists","owner":"devinit","description":"Parsing and coding FTS data from API","archived":false,"fork":false,"pushed_at":"2020-06-22T11:31:54.000Z","size":685,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-07T21:46:51.363Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/devinit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-18T12:31:39.000Z","updated_at":"2020-06-22T11:31:56.000Z","dependencies_parsed_at":"2023-09-18T08:00:06.263Z","dependency_job_id":null,"html_url":"https://github.com/devinit/fts_precodinglists","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devinit%2Ffts_precodinglists","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devinit%2Ffts_precodinglists/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devinit%2Ffts_precodinglists/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/devinit%2Ffts_precodinglists/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/devinit","download_url":"https://codeload.github.com/devinit/fts_precodinglists/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240621280,"owners_count":19830509,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T15:29:26.611Z","updated_at":"2026-06-06T23:32:31.530Z","avatar_url":"https://github.com/devinit.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fts_precodinglists\n\nThe fts_finalcode.R is used to join the FTS database extracted from API with the CSV files that contain new variables to be added to the new modified FTS database. The fts_precodinglists.R was only used to select new flows in the Donors, Recipient.Organisations and Destination.Country that were not incorporated in the CVS coding lists.\n\n# Prepare and clean the the database before joining the csv files\n\nWe selected only the appropriate flows from status column ('paid' and 'commitment' contributions, excluding the pledges).\n\n```R\ndata \u003c- subset(data, data$status %in% c('paid','commitment'))\n```\n\nWe also selected the columns from the original database that are required in the final visualisation of the database\n\n```R\ndata \u003c- subset(data, select = c(\"id\",\"amountUSD\",\"budgetYear\",\"description\",\n                                \"flowType\",\"newMoney\",\"originalAmount\",\"originalCurrency\",\n                                \"method\",\"status\",\"boundary\",\"onBoundary\",\"source_Organization_name\",\n                                \"source_Location_name\",\"source_UsageYear_name\",\"destination_Organization_name\",\n                                \"destination_GlobalCluster_name\",\"destination_Location_name\",\"destination_UsageYear_name\",\n                                \"destination_Plan_name\",\"destination_Project_name\",\"parentFlowId\",\"grandBargainEarmarkingType\",\n                                \"source_Plan_id\",\"source_Plan_name\",\"destination_Cluster_name\",\"destination_Emergency_name\",\n                                \"exchangeRate\",\"source_Emergency_name\",\"source_GlobalCluster_name\"))\n```\n\nWe renamed the source_Organization_name, destination_Organization_name and destination_Location_name to Donor, Recipient.Organization and Destination.Country, respectively. The cvs files will be join to the FTS database by Donor, Recipient.Organization and Destination.Country columns.\n\n```R\nif(\"source_Organization_name\" %in% names(data)){\n  names(data)[which(names(data)==\"source_Organization_name\")] = \"Donor\"\n}\n```\n\nWe Removed \", Government of\" of Donor and Recipient.Organization columns to harmonise the name of the organisations with the cvs files.\n\n```R\ndata$Donor \u003c- gsub(\", Government of\",\"\",data$Donor)\nunique(data$Donor)\n```\nWe substituted the special characters to the foreign characters from Donor and Recipient.Organization columns\n\n```R\ncharacter_replacements = list(\n  c(\"á\",\"á\"), c(\"â\",\"â\"), c(\"ä\",\"ä\"),\n  c(\"é\",\"é\"), c(\"ë\",\"ë\"), c(\"ó\",\"ó\"),\n  c(\"ô\",\"ô\"), c(\"ö\",\"ö\"), c(\"ú\",\"ú\"),\n  c(\"ü\",\"ü\"), c(\"Äf\",\"ã\"), c(\"ÄT\",\"ê\"),\n  c('Å\"','ñ'), c(\"Å^\",\"ò\"), c(\"Å'\",\"õ\"),\n  c(\"Å.\",\"à\"), c(\"Å\u003e\",\"o\"), c(\"ů\",\"ù\"),\n  c(\"ű\",\"û\"), c(\"è\",\"è\"), c(\"ç\",\"ç\"),\n  c(\"ø\",\"ø\"), c('â???\"','-'), c(\"â???T\",\"'\"),\n  c(\"î\",\"î\"), c(\"ñ\",\"ñ\"), c(\"ê\",\"ê\"),\n  c(\"Ã???\",\"Ç\"), c(\"î\",\"î\"), c(\"ñ\",\"ñ\"),\n  c('â???\"','-'), c('Ã\"','Ä'),\n  c(\"Ã\",\"í\"), c(\"ï\",\"ï\"), c(\"í£\",\"ã\"),\n  c(\"í¯\",\"ï\"), c(\"í¦\",\"æ\"), c(\"í¥\",\"å\"),\n  c(\"â???T\",\"'\"),c('â???\"',\"-\"), c(\"í¦\",\"à\"),\n  c('Å\"',\"o\"), c(\"Å,\",\"à\"), c('Å\"',\"à\"),\n  c('â???\"','-'),c(\"â???T\",\"'\"),c('â???\"',\"-\"),\n  c('â???T',\"'\"), c('Å\"',\"à\"), c('â???\"',\"-\"),\n  c('â???T',\"'\")\n)\n\nfor(character_replacement in character_replacements){\n  from_character = character_replacement[1]\n  to_character = character_replacement[2]\n  data$Donor \u003c- gsub(\n    from_character,\n    to_character,\n    data$Donor,\n    ignore.case = FALSE, perl = FALSE,\n    fixed = TRUE, useBytes = FALSE\n  )\n  data$Recipient.Organization \u003c- gsub(\n    from_character,\n    to_character,\n    data$Recipient.Organization,\n    ignore.case = FALSE, perl = FALSE,\n    fixed = TRUE, useBytes = FALSE\n  )\n}\n```\n\n# Join the CVS to the FTS database\n\nThe following CVS files are linked to the FTS database by the Donor column: codenames2020, privatemoney2020, dacregion2020 and donorscountryid2020.\nThe following CVS files are linked to the FTS database by the Recipient.Organization column: recipientcodename2020, ngotype2020, deliverychannels2020 and recipientcountryid2020.\nThe following CVS files are linked to the FTS database by the Destination.Country column: odaeligible2020, destinationcountryid2020 and incomegroups2020.\n\n```R\nngotype \u003c- read.csv(\"Final lists coded/ngotype2020.csv\",na.strings=\"\",as.is=TRUE)\nngotype$lower.Recipient.Organization \u003c- lowerConv(ngotype$Recipient.Organization)\nngotype \u003c- ngotype[!duplicated(ngotype$Recipient.Organization),]\nngotype$Recipient.Organization \u003c- NULL\ndata \u003c- join(data, ngotype, by='lower.Recipient.Organization', type='left', match='first')\n\nwithoutngos \u003c- subset(data,is.na(ngotype))\nunique(withoutngos$Recipient.Organization)\n```\n\n# Create extra columns based on the existing final columns\n\nDomestic response is TRUE when donor country id is equal to the destination country id. When both columns are balnk, I consider it as non-domestic response (FALSE)\n\n```R\ndata$domesticresponse \u003c- ifelse(data$donorcountryid==data$destinationcountryid,TRUE,FALSE)\ndata$domesticresponse[is.na(data$domesticresponse)] \u003c- FALSE\n```\n\nWe created the column Deflator type merging the data from the columns Source_UsageYear_name and donorcountryid\n\n```R\ndata$deflatortype \u003c- paste(data$donorcountryid,data$source_UsageYear_name)\ndata$deflatortype[is.na(data$deflatortype)] \u003c- FALSE\n```\n\n# Merge deflator file and deflate amount USD values\n\nThe deflators2020 (csv file) is linked to the deflatortype column and the calculation are conduct according to the following code line:\n\n```R\ndeflators \u003c- read.csv(\"Final lists coded/deflators2020.csv\",na.strings=\"\",as.is=TRUE)\ndata \u003c- join(data, deflators, by='deflatortype', type='left', match='all')\ndata \u003c- transform(data,amountDeflated=as.numeric(amountUSD)/as.numeric(Deflators))\ndata \u003c- transform(data,amountDeflatedMillions=amountDeflated/1000000)\n\nwithoutdeflators \u003c- subset(data,is.na(deflators))\nunique(withoutdeflators$Donor)\n```\n\n# Exclude the columns that are not necessary in the final file\n\nWe excluded the following columns: deflatortype, lower.destinationcountrytype, destinationcountrytype, lower.deflatortype, lower.Donor and lower.Recipient.Organization.\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevinit%2Ffts_precodinglists","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevinit%2Ffts_precodinglists","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevinit%2Ffts_precodinglists/lists"}