{"id":30327310,"url":"https://github.com/oz/aguaxaca","last_synced_at":"2025-08-18T00:10:31.646Z","repository":{"id":309987508,"uuid":"1036273311","full_name":"oz/aguaxaca","owner":"oz","description":"Aguaxaca is a project to make accessible water distribution schedules in the city of Oaxaca, Mexico.","archived":false,"fork":false,"pushed_at":"2025-08-11T20:38:03.000Z","size":70,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-15T01:38:28.377Z","etag":null,"topics":["oaxaca","water","website"],"latest_commit_sha":null,"homepage":"https://agua.cypr.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-11T20:30:36.000Z","updated_at":"2025-08-12T00:58:35.000Z","dependencies_parsed_at":"2025-08-15T01:38:32.060Z","dependency_job_id":"ba8f26f3-5635-41d3-b6b6-7d654c35a2c9","html_url":"https://github.com/oz/aguaxaca","commit_stats":null,"previous_names":["oz/aguaxaca"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/oz/aguaxaca","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oz%2Faguaxaca","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oz%2Faguaxaca/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oz%2Faguaxaca/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oz%2Faguaxaca/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oz","download_url":"https://codeload.github.com/oz/aguaxaca/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oz%2Faguaxaca/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270923824,"owners_count":24668625,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-17T02:00:09.016Z","response_time":129,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["oaxaca","water","website"],"created_at":"2025-08-18T00:10:29.936Z","updated_at":"2025-08-18T00:10:31.636Z","avatar_url":"https://github.com/oz.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Aguaxaca 🚰\n\nAs of 2025, Oaxaca's water distribution schedules are shared daily through\nimages, on two social networks: Facebook and X (formerly Twitter). I imagine\nthat it's a workaround to message-length and formatting limitations of these\nplatforms. This is still annoying. It excludes people with visual disabilities,\nand also makes it hard to use, or search, the data. This also forces the\ncitizens to create accounts on privately owned social networks.\n\nThis program won't solve these issues because they are not entirely technical,\nbut it's a small workaround to:\n\n1. extract data from public notices as they are published,\n2. share the same data in text form (accessible!), and\n3. provide access to historical records.\n\n\u003e [!CAUTION]\n\u003e 🚧 The project is a work-in-progress, evolving when free time allows it.\n\u003e Don't rely on it too much.\n\nA public instance is running at https://agua.cypr.io\n\n# Self hosting\n\n## Using docker\n\nA docker image is published on Docker Hub.\n\nTo run the web server locally:\n\n```\ndocker run ozzz/aguaxaca:latest -p 8080:8080 -v ./data:/data\n```\n\nThe `./data` directory is where the program will store collected image\nfiles, and its SQLite database.\n\nPeriodically, you can fetch fresh data by running:\n\n```\ndocker run ozzz/aguaxaca:latest -v ./data:/data aguaxaca collect\n```\n\nTo parse the data, usually right after a successfull \"collect\" run, use:\n\n```\ndocker run ozzz/aguaxaca:latest -v ./data:/data aguaxaca analyze\n```\n\nUse `-e` or `--env-file` to pass relevant env. variables to the container.\nSee below for a list of known variables.\n\n## Environment variables\n\nRequired, for the *analyze* sub-command:\n\n- `ANTHROPIC_API_KEY`: Anthropic private API key, to extract text from images.\n\nOptional, for the *collect* sub-command:\n\n- `NITTER_HOST`: where we fetch tweets, defaults to `http://nitter`.\n- `NITTER_ACCOUNT`: Twitter/X handle, defaults to `SOAPA_Oax`.\n\n# Technical information\n\nThe repository is organized as follow:\n\n- `app/` —  the core application types.\n- `collector/` —  collect images from social networks.\n- `parser/` —  parse collected images into structured data structures.\n- `web/` —  web server.\n\n## Data collection\n\nTo download public schedules shared recently, run:\n\n```\naguaxaca collect\n```\n\nData collection currently relies on [Nitter](https://nitter.net), a pure HTML\nfront-end for X, to fetch the images from the @SOAPA_Oax account. Based on the\ncurrent publication schedule, running colleciton about 3-4 times per day is\nenough.\n\nRunning your own private Nitter instance is not much work. To use a public\ninstance change the `NITTER_HOST` environment variable, and ask for permission maybe.\n\nOther improvements to explore:\n\n- Scrape X directly (probably stupidly expensive these days), or\n- scrape Facebook (really?), or\n- use Nitter's RSS feed instead of scraping (though RSS is often disabled on\n  public Nitter instances).\n\n## Image analysis\n\nTo extract text, and store data from downloaded images, run:\n\n```\naguaxaca analyze\n```\n\nTesseract is an okay open-source OCR program, but because of the layout of\nSOAPA's images, it won't work well here. Instead, we rely on genAI for OCR-ing\n*and* formatting the output.\n\nWe use Anthropic's APIs to extract information from the images (target is Sonnet\n4 model for now). This means that you will need an Anthropic API key with some\ncredits to run the parser. Currently, this costs a few cents per image.\n\nWith the correct hardware, using a local model would also work, but that's way\nmore expensive than Anthropic for now. 💸\n\nLook into `parser/parser.go` for a prompt that will extract information from\nSOAPA_Oax's publications. Here's a sample response from Sonnet 4.0:\n\n```\ndate,schedule,location_type,location_name\n2025-07-21,matutino-vespertino,colonia,Libertad\n2025-07-21,matutino-vespertino,colonia,10 de Abril\n2025-07-21,matutino-vespertino,colonia,Monte Albán\n2025-07-21,matutino-vespertino,colonia,Adolfo López Mateos\n2025-07-21,matutino-vespertino,colonia,Presidente Juárez\n2025-07-21,matutino-vespertino,colonia,Margarita Maza de Juárez\n2025-07-21,matutino-vespertino,colonia,Bosque Sur\n2025-07-21,matutino-vespertino,colonia,\"Jardín (sector Bugambilias)\"\n2025-07-21,matutino-vespertino,fraccionamiento,Jardines de Las Lomas\n2025-07-21,matutino-vespertino,ejido,\"Guadalupe Victoria (sector 1, 2ª sección Oeste)\"\n2025-07-21,matutino-vespertino,unidad,Ferrocarrilera\n```\n\n## Data store\n\n### Dev notes\n\nSqlite should be fine for a long while, with FTS5 providing full-text search on locations.\n\nAs we get more data, we could provide more services:\n\n1. figure out unique IDs for each zone —  the original data, with district names, is often incoherent and not precise.\n2. compute some stats like: delivery interval in days, number of deliveries tracked per year, etc.\n3. provide an export function for people interested in the raw data.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foz%2Faguaxaca","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foz%2Faguaxaca","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foz%2Faguaxaca/lists"}