{"id":19655953,"url":"https://github.com/dislick/beobachter","last_synced_at":"2026-05-15T06:10:41.924Z","repository":{"id":36565300,"uuid":"224467867","full_name":"dislick/beobachter","owner":"dislick","description":"Beobachter (german: Observer) periodically scrapes web pages and extracts values.","archived":false,"fork":false,"pushed_at":"2022-12-05T02:22:08.000Z","size":884,"stargazers_count":2,"open_issues_count":10,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-23T21:35:34.898Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dislick.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-27T16:04:10.000Z","updated_at":"2019-12-19T17:01:35.000Z","dependencies_parsed_at":"2023-01-17T02:45:39.107Z","dependency_job_id":null,"html_url":"https://github.com/dislick/beobachter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dislick%2Fbeobachter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dislick%2Fbeobachter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dislick%2Fbeobachter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dislick%2Fbeobachter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dislick","download_url":"https://codeload.github.com/dislick/beobachter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240964382,"owners_count":19885742,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:25:18.973Z","updated_at":"2026-05-15T06:10:41.897Z","avatar_url":"https://github.com/dislick.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Beobachter 👀\n\n\u003e Early alpha! Everything is subject to change and certain features are not implemented yet.\n\n## About\n\nThis Node.js service performs highly configurable, periodic web scraping. Results can be stored in time-series databases for further analysis using visualization tools like [Grafana](https://grafana.com).\n\n## Short Example\n\nA configuration like this lets you download a certain web page every 60 seconds and run a JavaScript function on it to extract the price of an item. It uses the simple `adapter-console` to record values to standard output.\n\n```jsonc\n{\n  \"adapters\": [{ \"name\": \"adapter-console\", \"options\": { \"colors\": true } }],\n  \"tasks\": [\n    {\n      \"type\": \"browser\",\n      \"name\": \"digitec-amd-ryzen-9-3950x\",\n      \"description\": \"Watches the prize of the new 16-core Ryzen 3950X\",\n      \"url\": \"https://www.digitec.ch/de/s1/product/amd-ryzen-9-3950x-am4-360ghz-16-core-prozessor-11239808\",\n      \"fn\": \"return parseInt(document.querySelector('meta[property=\\\"product:price:amount\\\"]').content, 10)\",\n      \"interval\": 60\n    }\n  ]\n}\n```\n\n```\n[2019-12-13T21:27:54] [digitec-amd-ryzen-9-3950x] [60s] 854\n[2019-12-13T21:28:55] [digitec-amd-ryzen-9-3950x] [60s] 854\n[2019-12-13T21:29:56] [digitec-amd-ryzen-9-3950x] [60s] 854\n```\n\n## Documentation\n\n### Adapters\n\nAdapters are required to record values. The following adapters are available/planned:\n\n- `adapter-console` Logs to standard output. `[available]`\n- `adapter-influxdb` Stores values in InfluxDB. `[planned]`\n- `adapter-csv` Writes values to a CSV file. `[planned]`\n\n### Configuration\n\nThere are multiple ways of configuring Beobachter.\n\n1. `config.json` in the same directory.\n2. Environment variable `BEOBACHTER_CONFIG` set to the absolute path of your `config.json`.\n\n#### Adapters\n\nBeobachter needs you to configure at least one adapter. Specify its `name` and `config` which you will find at `packages/adapter-*` in this repository.\n\n```json\n{\n  \"adapters\": [\n    {\n      \"name\": \"adapter-console\",\n      \"config\": {\n        \"colors\": true\n      }\n    }\n  ]\n}\n```\n\n#### Tasks\n\nTasks tell Beobachter what it needs to do. You can specify any number of tasks. There are different types of tasks:\n\n- `browser`\n- `http-json`\n- `http-text`\n\n\u003e You must not use spaces in `name`. This constraint allows adapters to record data more predictably.\n\n```jsonc\n{\n  \"tasks\": [\n    {\n      \"type\": \"browser\",\n      \"name\": \"github-vscode-stargazers-browser\",\n      \"description\": \"\",\n      \"url\": \"https://github.com/microsoft/vscode/stargazers\",\n      \"fn\": \"return parseInt(document.querySelector('#repos .Counter').innerText.replace(',', ''))\",\n      \"interval\": 60 // seconds\n    },\n    {\n      \"type\": \"http-json\",\n      \"name\": \"github-vscode-stargazers-json\",\n      \"description\": \"\",\n      \"url\": \"https://api.github.com/repos/microsoft/vscode\",\n      \"path\": \"stargazers_count\",\n      \"interval\": 60 // seconds\n    }\n  ]\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdislick%2Fbeobachter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdislick%2Fbeobachter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdislick%2Fbeobachter/lists"}