{"id":29806062,"url":"https://github.com/maraujo/pySocialWatcher","last_synced_at":"2025-07-28T14:06:52.647Z","repository":{"id":21309758,"uuid":"79020302","full_name":"maraujo/pySocialWatcher","owner":"maraujo","description":" Social Watcher on Facebook Marketing API","archived":false,"fork":false,"pushed_at":"2022-12-27T14:57:40.000Z","size":491,"stargazers_count":104,"open_issues_count":8,"forks_count":52,"subscribers_count":14,"default_branch":"master","last_synced_at":"2024-04-16T12:19:07.781Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maraujo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-15T09:38:09.000Z","updated_at":"2024-03-12T15:24:11.000Z","dependencies_parsed_at":"2023-01-11T21:09:39.761Z","dependency_job_id":null,"html_url":"https://github.com/maraujo/pySocialWatcher","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/maraujo/pySocialWatcher","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraujo%2FpySocialWatcher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraujo%2FpySocialWatcher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraujo%2FpySocialWatcher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraujo%2FpySocialWatcher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maraujo","download_url":"https://codeload.github.com/maraujo/pySocialWatcher/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maraujo%2FpySocialWatcher/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267527835,"owners_count":24102019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-28T02:00:09.689Z","response_time":68,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-28T14:05:37.146Z","updated_at":"2025-07-28T14:06:52.633Z","avatar_url":"https://github.com/maraujo.png","language":"Python","readme":"# pySocialWatcher\n[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square)](http://makeapullrequest.com)\n### A Social Data Collector from Facebook Marketing API\n#### I'm more than pleased that many teams used this library. But I need your help to support it. Please feel free to pull requests that solve issues. Unfortunatelly, my time is very very limited to keep updating this repository.\n#### If this package helps your research somehow, reference this paper:\n\n```\n@inproceedings{araujo2017facebook,\n author = {Araujo, Matheus and Mejova, Yelena and Weber, Ingmar and Benevenuto, Fabricio},\n title = {Using Facebook Ads Audiences for Global Lifestyle Disease Surveillance: Promises and Limitations},\n series = {WebSci '17},\n year = {2017},\n location = {Troy, USA},\n publisher = {ACM},\n address = {New York, NY, USA},\n keywords = {Facebook, Advertising, Epidemihology, Social Media, Health},\n} \n```\n\n \n[![Build Status](https://travis-ci.org/maraujo/pySocialWatcher.svg?branch=master)](https://travis-ci.org/maraujo/pySocialWatcher)\n[![codecov](https://codecov.io/gh/maraujo/pySocialWatcher/branch/dev/graph/badge.svg)](https://codecov.io/gh/maraujo/pySocialWatcher)\n\n**Package Name:** pysocialwatcher\n\n**Facebook Ads API version supported:** 2.9\n\n**License:** MIT\n\n**Python Version:** 2.7\n\n\n### What is this for\nThis package tries to get the full potencial of the Facebook Marketing API for Social Analysis research.\nRecent works show that online social media has a huge potencial to provide interesting insights on trends of across demographic groups.\n\nExamples of research question that it can answer:\n* For each european country, get how many people are interested in Science?\n* Get how many people in each GCC country who is Graduated AND is interested in Football, and how many is not interested in Football breakdown by: gender, age range, scholarity, language and citizenship.\n\n##### Tutorial\nA guide for getting started with Facebook Ads API data collection can be found here: https://worldbank.github.io/connectivity_mapping/intro.html Please note that you will need to clone a fork (https://github.com/joaopalotti/pySocialWatcher), instead of the original repo. The fork includes support for Python 3 (Python 3.7 works for sure, very likely it should support Python 3.8 as well) and utilities to create JSON files and to post-process your collection.\n\n##### Facebook Marketing API References page:\nTargeting Specs: https://developers.facebook.com/docs/marketing-api/targeting-specs/v2.8\n\nAd Targeting Search API: https://developers.facebook.com/docs/marketing-api/targeting-search/v2.8\n\n### Limitations:\n* Current supported API fields are listed below:\n    ```\n    \"interests\",\n    \"behaviors\",\n    \"education_statuses\",\n    \"family_statuses\",\n    \"relationship_statuses\",\n    \"locales\",\n    \"genders\",\n    \"age_min\",\n    \"age_max\",\n    \"geo_locations\"\n    ```\n\n### Install\n    git clone https://github.com/maraujo/pySocialWatcher.git\n    cd pySocialWatcher\n    pip install -r requirements.txt\n    python setup.py install\n    \n### Quick Start\nYou should have a .csv file with your Facebook tokens and accountIDs.\nExample: pySocialWatcher/pysocialwatcher/facebook_tokens_example.csv\n  \n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcher.run_data_collection(\"pysocialwatcher/input_examples/quick_example.json\")\n\n### How it works (slides):\nCheck the slides: https://goo.gl/WzE9ic\n\n### Features\n1. Static input json format to make you experiments easily reproducible.\n2. Support multiple Facebook tokens.\n3. Multiple tokens are processed in parallel to speedup data collection.\n3. Complex logic queries in the Facebook Marketing API with 'or', 'and', 'not', for example:.\n```\n      \"interests\": [{\n            \"not\": [6003442346642],\n            \"and\": [6004115167424, 6003277229371],\n            \"name\": \"Not interested in Football, but interest in some physical activity\"\n        }\n```\n4. Automatically save the state every constants.SAVE_EVERY requests. If any problem happens you can load the incomplete file and continue the data collection (```load_data_and_continue_collection```)\n\n#### Input Json Format Example\n\nThe following input is an example of input format of the package. In this example it will perform several requests in the Facebook Marketing API in order to collect the audience realted to Soccer Interest in GCC countries.\n\n    {   \"name\": \"Soccer Interest\",\n         \"geo_locations\": [\n              { \"name\": \"countries\", \"values\": [\"BH\"] },\n              { \"name\": \"countries\", \"values\": [\"KW\"] },\n              { \"name\": \"countries\", \"values\": [\"OM\"] },\n              { \"name\": \"countries\", \"values\": [\"QA\"] },\n              { \"name\": \"countries\", \"values\": [\"SA\"] },\n              { \"name\": \"countries\", \"values\": [\"AE\"] }\n    ],\n    \"genders\": [1,2],\n    \"ages_ranges\": [\n        {\"min\":18, \"max\":24},\n        {\"min\":55}\n    ],\n    \"scholarities\":[{\n        \"name\" : \"Graduated\",\n        \"or\" : [3,7,8,9,11]\n      }\n    ],\n    \"languages\":[{\n        \"name\" : \"Arabic\",\n        \"values\" : [28]\n        },\n        null\n    ],\n    \"behavior\": [\n        {\n          \"or\": [6015559470583],\n          \"name\": \"Expats\"\n        },\n        {\n          \"not\": [6015559470583],\n          \"name\": \"Not Expats\"\n        }\n    ],\n    \"interests\": [{\n            \"not\": [6003442346642],\n            \"and\": [6004115167424, 6003277229371],\n            \"name\": \"Not interested in Football, but interest in physical activity\"\n        },{\n            \"or\": [6003442346642],\n            \"name\" : \"Football\"\n        }\n    ]\n    }\nIn total it will perform 192 requests which are created in the following way:\n```\nFor each GCC country in the geo_locations field:\n    For each gender in [1,2]:\n        For each age range in [18-24,55+]:\n            For each scholarity group:\n                For each language group:\n                    For each behavior group:\n                        For each interest group:\n                            doFacebookAPIRequest()\n            \n```\nSo it will collect the audience for all of the combinations specified in the input file. If you don't want to specify a specific field you can ommit in the input or put a null value in the list like:\n```\n\"languages\":[{\n    \"name\" : \"Arabic\",\n    \"values\" : [28]\n    },\n    null\n],\n\n```\n#### Find Targeting IDs given Query\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcher.print_search_targeting_from_query_dataframe(\"Parents\")\n    \n    +----+-----------------+----------------------------------------+------------------+------------------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------+\n    |    |   audience_size | description                            |               id | name                                                 | path                                                                                                    | type              |\n    |----+-----------------+----------------------------------------+------------------+------------------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------|\n    |  0 |       286670228 | People who are parents.                |    6002714398372 | Parents (All)                                        | [u'Demographics', u'Parents', u'All Parents', u'Parents (All)']                                         | family_statuses   |\n    |  1 |        92661179 | Parents with children 18-26 years old. |    6023005718983 | (18-26 Years) Parents with Adult Children            | [u'Demographics', u'Parents', u'All Parents', u'(18-26 Years) Parents with Adult Children ']            | family_statuses   |\n    |  2 |        45169601 | Parents with children 13-18 years old. |    6023005681983 | (13-18 Years) Parents with Teenagers                 | [u'Demographics', u'Parents', u'All Parents', u'(13-18 Years) Parents with Teenagers ']                 | family_statuses   |\n    |  3 |        13123437 | Parents with Children ages 8-12        |    6023080302983 | (08-12 Years) Parents with Preteens                  | [u'Demographics', u'Parents', u'All Parents', u'(08-12 Years) Parents with Preteens']\n    ...\n    \n#### Find Interest IDs given name Name\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcher.print_interests_given_query(\"Family\")\n    \n    +----+------------+---------------+--------------------------+--------------------------------------------------------------------+\n    |    |   audience |   interest_id | name                     | path                                                               |\n    |----+------------+---------------+--------------------------+--------------------------------------------------------------------|\n    |  0 | 1019524230 | 6012684376438 | Family and relationships | [u'Interests', u'Family and relationships']                        |\n    |  1 |  788072890 | 6003476182657 | Family                   | [u'Interests', u'Family and relationships', u'Family']             |\n    |  2 |  178716020 | 6003190413105 | Family (biology)         | [u'Interests', u'Additional Interests', u'Family (biology)']       |\n    |  3 |   58127760 | 6003206382686 | family  planning         | [u'Interests', u'Additional Interests', u'family  planning']       |\n    |  4 |   25284610 | 6002966041646 | family films             | [u'Interests', u'Additional Interests', u'family films']           |\n    |  5 |   18734150 | 6003305411169 | Family Guy               | [u'Interests', u'Additional Interests', u'Family Guy']             |\n    |  6 |   12447740 | 6003455599405 | Royal family             | [u'Interests', u'Additional Interests', u'Royal family']           |\n    |  7 |   10252970 | 6003143952966 | Family of Barack Obama   | [u'Interests', u'Additional Interests', u'Family of Barack Obama'] |\n    +----+------------+---------------+--------------------------+--------------------------------------------------------------------+\n\n#### Find Behavior IDs Lists\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcher.print_behaviors_list()\n    +-----+------------+---------------+-------------------------------------------------------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------+\n    |     |   audience |   behavior_id | description                                                                                                       | name                                               | path                                                                                       |\n    |-----+------------+---------------+-------------------------------------------------------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------|\n    |   0 |  249325232 | 6002714895372 | People whose activities on Facebook suggest they are Frequent travelers.                                          | All frequent travelers                             | [u'Travel', u'All frequent travelers']                                                     |\n    |   1 |   34932108 | 6002714898572 | People who list themselves as small business owners or own small business pages on Facebook                       | Small business owners                              | [u'Digital activities', u'Small business owners']                                          |\n    |   2 |    6444115 | 6002764392172 | People who have used Facebook Payments platform in the past 90 days                                               | FB Payments (All)                                  | [u'Digital activities', u'FB Payments (All)']                                              |\n    |   3 |   91484312 | 6003050295572 | Users who uploaded \u003e50 photos on Facebook in last month.                                                          | Photo uploaders\n    ...\n\n#### Find Geo Location Key Given Query and Location Type\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcherAPI.print_geo_locations_given_query_and_location_type(\"new\", [\"city\"])\n    +----+---------+-------------------------+-----------------+-------------------+--------+\n    |    |     key | name                    |   supports_city |   supports_region | type   |\n    |----+---------+-------------------------+-----------------+-------------------+--------|\n    |  0 | 2490299 | New York                |               1 |                 1 | city   |\n    |  1 | 2490287 | New Rochelle            |               1 |                 1 | city   |\n    |  2 | 2528778 | New Braunfels           |               1 |                 1 | city   |\n    |  3 | 2511352 | New Castle              |               1 |                 1 | city   |\n    ....\n\n#### Find all Geo Locations Given Location Type and Country\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcherAPI.print_geo_locations_given_query_and_location_type(None, [\"region\"], country_code='ID')\n\n    +----+----------------+----------------+-------+------------------------------+-----------------+-------------------+--------+\n    |    | country_code   | country_name   |   key | name                         | supports_city   | supports_region   | type   |\n    |----+----------------+----------------+-------+------------------------------+-----------------+-------------------+--------|\n    |  0 | ID             | Indonesia      |  1662 | Bali                         | True            | True              | region |\n    |  1 | ID             | Indonesia      |  1676 | East Nusa Tenggara           | True            | True              | region |\n    |  2 | ID             | Indonesia      |  1685 | West Java                    | True            | True              | region |\n    |  3 | ID             | Indonesia      |  1675 | West Nusa Tenggara           | True            | True              | region |\n    ...\n    \n#### Find KML Given Location Key and Location Type\n    \u003e\u003e\u003e from pysocialwatcher import watcherAPI \n    \u003e\u003e\u003e watcher = watcherAPI() \n    \u003e\u003e\u003e watcher.load_credentials_file(\"pysocialwatcher/credentials.csv\")\n    \u003e\u003e\u003e watcherAPI.get_kml_given_geolocation(\"countries\", [\"BR\",\"CL\",\"AT\",\"US\",\"QA\"])\n    +----+---------------------------------------------------+-------------------+\n    |    |     kml                                           | name              |\n    |----+---------------------------------------------------+-------------------+\n    |  0 | \u003cPolygon\u003e\u003couterBoundaryIs\u003e\u003cLinearRing\u003e\u003ccoordin... | Brazil            |\n    |  1 | \u003cPolygon\u003e\u003couterBoundaryIs\u003e\u003cLinearRing\u003e\u003ccoordin... | Chile             |\n    |  2 | \u003cPolygon\u003e\u003couterBoundaryIs\u003e\u003cLinearRing\u003e\u003ccoordin... | Austria           |\n    |  3 | \u003cPolygon\u003e\u003couterBoundaryIs\u003e\u003cLinearRing\u003e\u003ccoordin... | United States     |\n    ....\n\n\n#### Advaced Configurations\n##### Change Sleep time between requests to 10s:\n\n    from pysocialwatcher import constants\n    constants.SLEEP_TIME = 10\n\n##### Change save temporary file every 1000 requests:\n\n    from pysocialwatcher import constants\n    constants.SAVE_EVERY = 1000\n  \n##### Specify that results should be saved to the directory `data/pySocialWatcher/test_query_results/`\n\n    watcher.run_data_collection(\"pySocialWatcher/pysocialwatcher/quick_example.json\", \n                                \"data/pySocialWatcher/test_query_results/\")\n\n_Note: Assumes that the directory already exists. Filepath is added to the beginning of the output `.csv` files generated._ \n\n\n### Potential Issues:\n1. If you received the error: *You are calling a deprecated version of the Ads API*, means that Facebook updated the API. One way to fix is changing the first 3 variables of the constants.py file to the current Facebook API. This does not guarantee that everything will work.\n2. Various users are getting issues with this package. Since I personally do not use it more, I recommend the fork from Joao Palotti.\n   - I suggest you have a look at this resource here: https://worldbank.github.io/connectivity_mapping/intro.html\n   - It is a small tutorial describing how to use pySocialWatcher. Note that you will need to clone my fork (https://github.com/joaopalotti/pySocialWatcher), instead of the original repo. The fork includes support for Python 3 (Python 3.7 works for sure, very likely it should support Python 3.8 as well) and utilities to create JSON files and to post-process your collection.\n\n\n\n### Change Log\n* 2.0a - Fix bug from @joaopalotti and thank @KangboLu, @kpolimis, @khof312, @mfatehkia for previous commits.\n* 0.1j - Get more informative dataframes from: get_behavior_dataframe, get_interests_given_query, get_search_targeting_from_query_dataframe\n* 0.1i - Errors more understable and some small fixes.\n* 0.1h - AND working for behavior. Thanks @ianbstewart.\n* 0.1g - Besides MAU, now we also captures DAU. Thanks @VatsalaSingh.\n* 0.1f - Call version v2.11 of Facebook Marketing API\n* 0.1e - Add support to *not* for demographics that uses specific integer numbers such as scholarity\n* 0.1d - Add support to household_composition\n* 0.1b2 - Add support to relationship_statuses and check input keys\n* 0.1b0 - Search any query with get_search_targeting_from_query_dataframe()\n* 0.1a3 - Resilient error handling when error code = 2 (try again more constants.MAX_NUMBER_TRY times)\n* 0.1a2 - Fix bug when have multiple operators like (and, not) in the same query\n* 0.1a1 - Add MIT License and 'Geo Location' field support in facebook API\n* 0.1a0 - First Alpha Release\n","funding_links":[],"categories":["شبکه های اجتماعی","APIs \u0026 SDKs"],"sub_categories":["کار با زمان و تقویم"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaraujo%2FpySocialWatcher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaraujo%2FpySocialWatcher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaraujo%2FpySocialWatcher/lists"}