{"id":22480507,"url":"https://github.com/brizandrew/fb-page-scraper","last_synced_at":"2025-03-27T18:42:17.157Z","repository":{"id":97029762,"uuid":"104763920","full_name":"brizandrew/fb-page-scraper","owner":"brizandrew","description":"A scraper to collect the content, metadata, and analytics of the posts of public Facebook pages.","archived":false,"fork":false,"pushed_at":"2018-02-06T18:27:49.000Z","size":32,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-02-01T21:41:35.105Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brizandrew.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-25T14:54:00.000Z","updated_at":"2023-07-26T13:20:34.000Z","dependencies_parsed_at":"2024-03-09T01:00:35.384Z","dependency_job_id":null,"html_url":"https://github.com/brizandrew/fb-page-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brizandrew%2Ffb-page-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brizandrew%2Ffb-page-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brizandrew%2Ffb-page-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brizandrew%2Ffb-page-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brizandrew","download_url":"https://codeload.github.com/brizandrew/fb-page-scraper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245904931,"owners_count":20691695,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-06T15:24:57.881Z","updated_at":"2025-03-27T18:42:17.152Z","avatar_url":"https://github.com/brizandrew.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Facebook Page Scraper\n\nA scraper to collect the content, metadata, and analytics of the posts of public Facebook pages.\n\n## Installation\nMake sure you have an sql database with a valid superuser.\n\nClone the git repository from GitHub.\n```bash\n$ git clone git@github.com:brizandrew/fb-page-scraper.git\n```\n\nCd into the `fb-page-scraper` directory and install all the dependencies.\n```bash\n$ cd fb-page-scraper\n$ npm install --production\n```\n\nCopy and paste the environment file and fill out the database information using a text editor.\n```bash\n$ cp .template.env .env\n$ nano .env\n```\n\nCreate your database tables using the `createTables` command.\n```bash\n$ npm run createTables\n```\n\nAdd pages to your `pages.json` file by running the `addPage` command, and following the prompts. You'll need the page's Facebook Id which you can find using [this](https://findmyfbid.com/).\n```bash\n$ npm run addPage\n```\n\nRun the `updatePages` command to sync your `pages.json` file with your database.\n```bash\n$ npm run updatePages\n```\n\n## Usage\n\n### Scraping\n\nRun the scraper by using the `scrape` command.\n```bash\n$ npm run scrape\n```\n\n### Crontab\n\nThe scraper will only collect data on pages at least an hour old. It is designed to be run once an hour for consistent data collection. To do this set up a cron job.\n```bash\n$ crontab -e\n```\n\nThis will bring up your cron jobs. Add the following as a line to run the scrape command every hour.:\n```\n0 * * * * cd /absolute/path/to/app/directory \u0026\u0026 npm run scrape;\n```\n\nSave and quit using `:x`.\n\n### Getting the Results\nPrint the results of the database into `data/output.csv` by using the `print` command.\n```bash\n$ npm run print\n```\n\n### Reseting The Content\n\nTo delete all of the post data in the database use the `resetdb` command.\n```bash\n$ npm run resetdb\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrizandrew%2Ffb-page-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrizandrew%2Ffb-page-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrizandrew%2Ffb-page-scraper/lists"}