{"id":25528963,"url":"https://github.com/meltanolabs/tap-stackexchange","last_synced_at":"2025-08-04T06:09:46.626Z","repository":{"id":37956211,"uuid":"417668017","full_name":"MeltanoLabs/tap-stackexchange","owner":"MeltanoLabs","description":"Singer tap for the StackExchange API","archived":false,"fork":false,"pushed_at":"2025-07-28T16:55:10.000Z","size":1296,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-28T18:41:34.943Z","etag":null,"topics":["elt","extract-data","meltano-sdk","singer-io","singer-tap","stackexchange"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MeltanoLabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-10-15T23:54:43.000Z","updated_at":"2025-07-28T16:55:13.000Z","dependencies_parsed_at":"2023-02-19T11:31:24.779Z","dependency_job_id":"78253a55-0502-4e3a-8163-69dd60a689fa","html_url":"https://github.com/MeltanoLabs/tap-stackexchange","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/MeltanoLabs/tap-stackexchange","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MeltanoLabs%2Ftap-stackexchange","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MeltanoLabs%2Ftap-stackexchange/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MeltanoLabs%2Ftap-stackexchange/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MeltanoLabs%2Ftap-stackexchange/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MeltanoLabs","download_url":"https://codeload.github.com/MeltanoLabs/tap-stackexchange/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MeltanoLabs%2Ftap-stackexchange/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268656814,"owners_count":24285392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-04T02:00:09.867Z","response_time":79,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elt","extract-data","meltano-sdk","singer-io","singer-tap","stackexchange"],"created_at":"2025-02-19T23:29:41.733Z","updated_at":"2025-08-04T06:09:46.573Z","avatar_url":"https://github.com/MeltanoLabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `tap-stackexchange`\n\n[![Test](https://github.com/MeltanoLabs/tap-stackexchange/actions/workflows/test.yml/badge.svg)](https://github.com/MeltanoLabs/tap-stackexchange/actions/workflows/test.yml)\n[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=MeltanoLabs_tap-stackexchange\u0026metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=MeltanoLabs_tap-stackexchange)\n[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=MeltanoLabs_tap-stackexchange\u0026metric=security_rating)](https://sonarcloud.io/summary/new_code?id=MeltanoLabs_tap-stackexchange)\n[![Duplicated Lines (%)](https://sonarcloud.io/api/project_badges/measure?project=MeltanoLabs_tap-stackexchange\u0026metric=duplicated_lines_density)](https://sonarcloud.io/summary/new_code?id=MeltanoLabs_tap-stackexchange)\n\nSinger tap for the StackExchange API.\n\nBuilt with the [Meltano SDK](https://sdk.meltano.com) for Singer Taps and Targets.\n\n## Capabilities\n\n* `sync`\n* `catalog`\n* `state`\n* `discover`\n\n## Settings\n\n| Setting   | Required | Default | Description |\n|:----------|:--------:|:-------:|:------------|\n| key       | False    | None    | Pass this to receive a higher request quota |\n| site      | False    | stackoverflow.com | StackExchange site |\n| tags      | False    | None    | Question tags |\n| start_date| False    | None    | The earliest record date to sync |\n\nA full list of supported settings and capabilities is available by running: `tap-stackexchange --about`\n\n## Custom filter\n\nThe StackExchange API supports a number of [custom filters](https://api.stackexchange.com/docs/filters) that can be used to\ninclude or exclude certain fields from the response objects. This application has a baked-in filter with the following\nparameters:\n\n- `include`\n\n  | Parameter                | Description                              |\n  |:-------------------------|:-----------------------------------------|\n  | `question.comment_count` | The number of comments on the question   |\n  | `tag.last_activity_date` | The date of the last activity on the tag |\n\n- `unsafe=false`\n\n### Update the baked-in filter\n\nTo update the baked-in filter, edit the `FILTER_ID` constant in `tap_stackexchange/tap.py`.\n\nTo generate a new filter, use the _Try It_ button on the [StackExchange API documentation](https://api.stackexchange.com/docs/create-filter), using the baked-in filter as the `base` parameter.\n\n### Use a custom filter\n\nTo use a custom filter, set the `filter` setting to the filter ID. Note that if you use a custom filter, you will need to\nuse a custom catalog that includes the fields you want to sync. That is you will need to\n\n1. Write the default catalog to a file: `tap-stackexchange --discover \u003e catalog.json`\n2. Edit the catalog file to include the fields not included by the default API filter\n3. Run the tap with the custom catalog: `tap-stackexchange --config config.json --catalog catalog.json`\n\n## Installation\n\n```bash\npipx install git+https://github.com/edgarrmondragon/tap-stackexchange.git\n```\n\n### Source Authentication and Authorization\n\nRegister a [new application on Stack Apps](https://stackapps.com/apps/oauth/register) and copy the generated `key`.\n\n## Usage\n\nYou can easily run `tap-stackexchange` by itself or in a pipeline using [Meltano](https://meltano.com/).\n\n### Executing the Tap Directly\n\n```bash\ntap-stackexchange --version\ntap-stackexchange --help\ntap-stackexchange --config CONFIG --discover \u003e ./catalog.json\n```\n\n### Initialize your Development Environment\n\n```bash\npipx install poetry\npoetry install\n```\n\n\u003c!--\n### Create and Run Tests\n\nCreate tests within the `tap_stackexchange/tests` subfolder and\n  then run:\n\n```bash\npoetry run pytest\n```\n\nYou can also test the `tap-stackexchange` CLI interface directly using `poetry run`:\n\n```bash\npoetry run tap-stackexchange --help\n```\n--\u003e\n\n### Testing with [Meltano](https://www.meltano.com)\n\n_**Note:** This tap will work in any Singer environment and does not require Meltano.\nExamples here are for convenience and to streamline end-to-end orchestration scenarios._\n\nYour project comes with a custom `meltano.yml` project file already created. Open the `meltano.yml` and follow any _\"TODO\"_ items listed in\nthe file.\n\nNext, install Meltano (if you haven't already) and any needed plugins:\n\n```bash\n# Install meltano\npipx install meltano\n# Initialize meltano within this directory\ncd tap-stackexchange\nmeltano install\n```\n\nNow you can test and orchestrate using Meltano:\n\n```bash\n# Test invocation:\nmeltano invoke tap-stackexchange --version\n\n# OR run a test `elt` pipeline:\nmeltano elt tap-stackexchange target-sqlite --job_id=stackexchange-sqlite\n\n# Runtime configuration\nTAP_STACKEXCHANGE__LOAD_SCHEMA=dragon_ball_gt \\\nTAP_STACKEXCHANGE_SITE=anime \\\nTAP_STACKEXCHANGE_TAGS='[\"dragon-ball-gt\"]' \\\nmeltano elt tap-stackexchange target-sqlite\n```\n\n### SDK Dev Guide\n\nSee the [dev guide](https://sdk.meltano.com/en/latest/dev_guide.html) for more instructions on how to use the SDK to\ndevelop your own taps and targets.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeltanolabs%2Ftap-stackexchange","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmeltanolabs%2Ftap-stackexchange","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmeltanolabs%2Ftap-stackexchange/lists"}