{"id":31845066,"url":"https://github.com/horizonsrc/whurl","last_synced_at":"2026-01-20T17:01:19.424Z","repository":{"id":247483762,"uuid":"765384822","full_name":"HorizonsRC/whurl","owner":"HorizonsRC","description":"Hilltop Server URL Generator","archived":false,"fork":false,"pushed_at":"2025-10-05T22:28:45.000Z","size":1433,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-10-05T22:47:18.552Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HorizonsRC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-29T20:18:31.000Z","updated_at":"2025-10-05T22:28:49.000Z","dependencies_parsed_at":"2025-08-26T01:15:57.457Z","dependency_job_id":"b3d874f0-8c5a-4946-a778-acc38fed9535","html_url":"https://github.com/HorizonsRC/whurl","commit_stats":null,"previous_names":["horizonsrc/hurl","horizonsrc/whurl"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/HorizonsRC/whurl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fwhurl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fwhurl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fwhurl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fwhurl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HorizonsRC","download_url":"https://codeload.github.com/HorizonsRC/whurl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HorizonsRC%2Fwhurl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279010671,"owners_count":26084785,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-12T02:00:06.719Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-12T07:56:55.316Z","updated_at":"2025-10-12T07:57:02.256Z","avatar_url":"https://github.com/HorizonsRC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# WHURL - Which Hydro URL\n\nA Python client library for interacting with Hilltop Server APIs, developed by Horizons Regional Council as a dependency of [Hydrobot](https://github.com/HorizonsRC/hydrobot). WHURL provides a clean, Pythonic interface for fetching environmental and scientific data from Hilltop servers.\n\n[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)\n[![Python Version](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org/downloads/)\n[![Poetry](https://img.shields.io/badge/dependency%20management-Poetry-blue)](https://python-poetry.org/)\n[![Version](https://img.shields.io/badge/version-0.1.1-green)](https://github.com/HorizonsRC/whurl)\n\n## Overview\n\nWHURL (Which Hydro URL) is designed to simplify interactions with Hilltop Server, a platform commonly used for storing and managing environmental data such as water levels, flow rates, rainfall measurements, and other scientific observations. The library handles URL generation, request validation, XML parsing, and provides structured Python objects for easy data manipulation.\n\n\u003e **⚠️ Work in Progress**: This library is currently under active development. While core functionality is stable and tested, not all Hilltop API endpoints are supported yet. See the [Planned Features](#planned-features) section for upcoming enhancements.\n\n### Key Features\n\n- **Simple Client Interface**: Easy-to-use `HilltopClient` with methods for all major Hilltop operations\n- **Request Validation**: Pydantic-based validation ensures proper request formatting\n- **Type Safety**: Full type hints for better IDE support and code reliability\n- **XML Response Parsing**: Automatic parsing of Hilltop XML responses into Python objects\n- **Error Handling**: Comprehensive exception handling with detailed error messages\n- **Configuration Management**: Support for environment variables and direct configuration\n- **Context Manager Support**: Proper resource cleanup with `with` statement support\n\n## Installation\n\n### Prerequisites\n\n- Python 3.11 or higher\n- [Poetry](https://python-poetry.org/) for dependency management\n- Internet connectivity to reach your Hilltop server\n\n### Install from PyPI (Recommended)\n\n```bash\npip install whurl\n```\n\n## Quick Start\n\n### Basic Setup\n\n```python\nfrom whurl.client import HilltopClient\n\n# Option 1: Direct configuration\nclient = HilltopClient(\n    base_url=\"https://your-hilltop-server.com\",\n    hts_endpoint=\"data.hts\",\n    timeout=60  # Optional, defaults to 60 seconds\n)\n\n# Option 2: Using environment variables\n# Set HILLTOP_BASE_URL and HILLTOP_HTS_ENDPOINT\nclient = HilltopClient()\n\n# Always use as a context manager for proper cleanup\nwith client:\n    # Your code here\n    status = client.get_status()\n    print(f\"Server status: {status}\")\n```\n\n### Environment Variables\n\nWHURL supports configuration via environment variables:\n\n```bash\nexport HILLTOP_BASE_URL=\"https://your-hilltop-server.com\"\nexport HILLTOP_HTS_ENDPOINT=\"data.hts\"\n```\n\nOr create a `.env` file:\n\n```env\nHILLTOP_BASE_URL=https://your-hilltop-server.com\nHILLTOP_HTS_ENDPOINT=data.hts\n```\n\n### Simple Example\n\n```python\nfrom whurl.client import HilltopClient\n\nwith HilltopClient() as client:\n    # Get server status\n    status = client.get_status()\n    print(f\"Server is running: {status.server}\")\n\n    # List all sites\n    sites = client.get_site_list()\n    print(f\"Found {len(sites.sites)} sites\")\n\n    # Get measurements for a specific site\n    measurements = client.get_measurement_list(site=\"YourSiteName\")\n    for measurement in measurements.measurements:\n        print(f\"Measurement: {measurement.name}, Units: {measurement.units}\")\n```\n\n## API Reference\n\n### HilltopClient\n\nThe main client class for interacting with Hilltop servers.\n\n#### Constructor\n\n```python\nHilltopClient(\n    base_url: str | None = None,\n    hts_endpoint: str | None = None,\n    timeout: int = 60\n)\n```\n\n**Parameters:**\n\n- `base_url`: The base URL of your Hilltop server (e.g., \"https://data.council.govt.nz\")\n- `hts_endpoint`: The HTS endpoint path (e.g., \"foo.hts\")\n- `timeout`: Request timeout in seconds (default: 60)\n\n### Client Methods\n\n#### get_status()\n\nGet the status of the Hilltop server.\n\n```python\nstatus = client.get_status()\nprint(status.server)  # Server information\nprint(status.version)  # Hilltop version\n```\n\n**Returns:** `StatusResponse` object\n\n#### get_site_list(\\*\\*kwargs)\n\nRetrieve a list of sites from the Hilltop server.\n\n```python\n# Get all sites\nsites = client.get_site_list()\n\n# Get sites with location information\nsites = client.get_site_list(location=\"Yes\")\n\n# Filter by measurement type\nsites = client.get_site_list(measurement=\"Flow\")\n\n# Filter by collection\nsites = client.get_site_list(collection=\"River\")\n```\n\n**Parameters:**\n\n- `location`: \"Yes\", \"LatLong\", or None - Include location data\n- `measurement`: Filter sites by measurement type\n- `collection`: Filter sites by collection name\n- `site_parameters`: Include site parameters\n- `bounding_box`: Spatial filter (format: \"x1,y1,x2,y2\")\n\n**Returns:** `SiteListResponse` object\n\n#### get_measurement_list(\\*\\*kwargs)\n\nGet available measurements for a site or collection.\n\n```python\n# Get measurements for a specific site\nmeasurements = client.get_measurement_list(site=\"YourSiteName\")\n\n# Get measurements with units information\nmeasurements = client.get_measurement_list(site=\"YourSiteName\", units=\"Yes\")\n\n# Get measurements for a collection\nmeasurements = client.get_measurement_list(collection=\"River\")\n```\n\n**Parameters:**\n\n- `site`: Site name (required if collection not specified)\n- `collection`: Collection name\n- `units`: \"Yes\" to include units information\n\n**Returns:** `MeasurementListResponse` object\n\n#### get_site_info(\\*\\*kwargs)\n\nGet detailed information about a specific site.\n\n```python\nsite_info = client.get_site_info(site=\"YourSiteName\")\nprint(site_info.site_name)\nprint(site_info.location)\n```\n\n**Parameters:**\n\n- `site`: Site name (required)\n\n**Returns:** `SiteInfoResponse` object\n\n#### get_data(\\*\\*kwargs)\n\nRetrieve measurement data from the Hilltop server.\n\n```python\n# Get recent data\ndata = client.get_data(\n    site=\"YourSiteName\",\n    measurement=\"Flow\",\n    from_datetime=\"2024-01-01T00:00:00\",\n    to_datetime=\"2024-01-31T23:59:59\"\n)\n\n# Get data with statistics\ndata = client.get_data(\n    site=\"YourSiteName\",\n    measurement=\"Flow\",\n    method=\"Average\",\n    interval=\"1 day\"\n)\n\n# Get data with custom time intervals\ndata = client.get_data(\n    site=\"YourSiteName\",\n    measurement=\"Rainfall\",\n    time_interval=\"1 hour\",\n    alignment=\"00:00\"\n)\n```\n\n**Parameters:**\n\n- `site`: Site name\n- `measurement`: Measurement name\n- `from_datetime`: Start datetime (ISO format)\n- `to_datetime`: End datetime (ISO format)\n- `method`: Statistical method (\"Interpolate\", \"Average\", \"Total\", \"Moving Average\", \"EP\", \"Extrema\")\n- `interval`: Time interval for statistics (e.g., \"1 day\", \"4 hours\")\n- `time_interval`: Regular time interval for data\n- `alignment`: Time alignment (e.g., \"00:00\")\n- `collection`: Collection name\n- `gap_tolerance`: Maximum gap between data points\n- `format`: Output format (\"Native\" or other formats)\n\n**Returns:** `GetDataResponse` object\n\n#### get_time_range(\\*\\*kwargs)\n\nGet the available time range for a measurement at a site.\n\n```python\ntime_range = client.get_time_range(\n    site=\"YourSiteName\",\n    measurement=\"Flow\"\n)\nprint(f\"Data available from {time_range.from_time} to {time_range.to_time}\")\n```\n\n**Parameters:**\n\n- `site`: Site name (required)\n- `measurement`: Measurement name (required)\n- `format`: \"json\" for JSON response, omit for XML\n\n**Returns:** `TimeRangeResponse` object\n\n#### get_collection_list(\\*\\*kwargs)\n\nGet a list of available collections.\n\n```python\ncollections = client.get_collection_list()\nfor collection in collections.collections:\n    print(f\"Collection: {collection.name}\")\n```\n\n**Returns:** `CollectionListResponse` object\n\n## Usage Examples\n\n### Environmental Data Monitoring\n\n```python\nfrom whurl.client import HilltopClient\nimport pandas as pd\n\nwith HilltopClient() as client:\n    # Monitor water levels at multiple sites\n    sites = [\"Site1\", \"Site2\", \"Site3\"]\n    water_levels = {}\n\n    for site in sites:\n        data = client.get_data(\n            site=site,\n            measurement=\"Water Level\",\n            from_datetime=\"2024-01-01T00:00:00\",\n            to_datetime=\"2024-01-31T23:59:59\"\n        )\n        water_levels[site] = data.to_dataframe()  # Convert to pandas DataFrame\n\n    # Analyze the data\n    for site, df in water_levels.items():\n        print(f\"{site}: Max level = {df['Value'].max():.2f} m\")\n```\n\n### Rainfall Analysis\n\n```python\nwith HilltopClient() as client:\n    # Get hourly rainfall data\n    rainfall = client.get_data(\n        site=\"RainfallSite\",\n        measurement=\"Rainfall\",\n        method=\"Total\",\n        interval=\"1 hour\",\n        from_datetime=\"2024-01-01T00:00:00\",\n        to_datetime=\"2024-01-07T23:59:59\"\n    )\n\n    # Calculate daily totals\n    daily_rain = client.get_data(\n        site=\"RainfallSite\",\n        measurement=\"Rainfall\",\n        method=\"Total\",\n        interval=\"1 day\",\n        from_datetime=\"2024-01-01T00:00:00\",\n        to_datetime=\"2024-01-31T23:59:59\"\n    )\n```\n\n### Site Discovery and Exploration\n\n```python\nwith HilltopClient() as client:\n    # Discover sites with flow measurements\n    sites = client.get_site_list(measurement=\"Flow\", location=\"LatLong\")\n\n    for site in sites.sites:\n        print(f\"Site: {site.name}\")\n        print(f\"Location: {site.latitude}, {site.longitude}\")\n\n        # Get available measurements\n        measurements = client.get_measurement_list(site=site.name, units=\"Yes\")\n        for measurement in measurements.measurements:\n            print(f\"  - {measurement.name} ({measurement.units})\")\n\n        # Get data time range\n        time_range = client.get_time_range(site=site.name, measurement=\"Flow\")\n        print(f\"  Data: {time_range.from_time} to {time_range.to_time}\")\n        print()\n```\n\n## Error Handling\n\nWHURL provides comprehensive error handling through custom exceptions:\n\n```python\nfrom whurl.client import HilltopClient\nfrom whurl.exceptions import (\n    HilltopError,\n    HilltopConfigError,\n    HilltopRequestError,\n    HilltopResponseError,\n    HilltopParseError\n)\n\ntry:\n    with HilltopClient() as client:\n        data = client.get_data(site=\"InvalidSite\", measurement=\"Flow\")\n\nexcept HilltopConfigError as e:\n    print(f\"Configuration error: {e}\")\n    # Handle missing base_url or hts_endpoint\n\nexcept HilltopRequestError as e:\n    print(f\"Request error: {e}\")\n    # Handle invalid parameters\n\nexcept HilltopResponseError as e:\n    print(f\"Server error: {e}\")\n    # Handle HTTP errors or server-side issues\n\nexcept HilltopParseError as e:\n    print(f\"Parse error: {e}\")\n    # Handle XML parsing errors\n\nexcept HilltopError as e:\n    print(f\"General Hilltop error: {e}\")\n    # Handle any other Hilltop-related errors\n```\n\n### Exception Hierarchy\n\n- `HilltopError` - Base exception for all WHURL errors\n  - `HilltopConfigError` - Configuration issues (missing URLs, credentials)\n  - `HilltopRequestError` - Request validation errors (invalid parameters)\n  - `HilltopResponseError` - HTTP and server response errors\n  - `HilltopParseError` - XML parsing and data conversion errors\n\n## Validation and Data Types\n\nWHURL uses Pydantic for request validation and type safety. Common validation rules:\n\n### Time Intervals\n\n```python\n# Valid time interval formats\n\"10 seconds\"    # or \"10 second\", \"10s\"\n\"5 minutes\"     # or \"5 minute\", \"5m\"\n\"1 hour\"        # or \"1h\"\n\"1 day\"         # or \"1d\"\n\"1 week\"        # or \"1w\"\n\"1 month\"       # or \"1mo\"\n\"1 year\"        # or \"1y\"\n```\n\n### Date Formats\n\n```python\n# ISO 8601 format required\n\"2024-01-01T00:00:00\"\n\"2024-12-31T23:59:59\"\n```\n\n### Statistical Methods\n\n- `\"Interpolate\"` - Interpolated values\n- `\"Average\"` - Average over interval\n- `\"Total\"` - Sum over interval\n- `\"Moving Average\"` - Moving average\n- `\"EP\"` - End period\n- `\"Extrema\"` - Min/max values\n\n## Development and Contributing\n\nWHURL uses [Poetry](https://python-poetry.org/) for dependency management and packaging. This ensures reproducible builds and simplifies development workflows.\n\n### Development Setup\n\n```bash\n# Clone the repository\ngit clone https://github.com/HorizonsRC/whurl.git\ncd whurl\n\n# Install Poetry if you haven't already\npip install poetry\n\n# Install all dependencies (runtime + development)\npoetry install\n\n# Activate the virtual environment\npoetry shell\n\n# Run tests to verify setup\npoetry run pytest\n```\n\n### Legacy pip Development Setup\n\nIf you prefer using pip directly:\n\n```bash\n# Clone the repository\ngit clone https://github.com/HorizonsRC/whurl.git\ncd whurl\n\n# Install in development mode\npip install -e .\n\n# Install development dependencies\npip install pytest pytest-mock pytest-httpx\n\n# Run tests\npython -m pytest tests/\n```\n\n### Running Tests\n\nWHURL uses a comprehensive testing strategy that includes both local mocked tests and remote API validation using a fixture cache system. For detailed information about testing categories, modes, and troubleshooting, see the [Testing Strategy Documentation](docs/TESTING_STRATEGY.md).\n\n### Code Style\n\nThis project uses:\n\n- Type hints for better code documentation\n- Pydantic for data validation\n- Black for code formatting (if applicable)\n- PEP 257 compliant docstrings\n\n## Configuration Reference\n\n### Environment Variables\n\n| Variable               | Description                | Example                        |\n| ---------------------- | -------------------------- | ------------------------------ |\n| `HILLTOP_BASE_URL`     | Base URL of Hilltop server | `https://data.council.govt.nz` |\n| `HILLTOP_HTS_ENDPOINT` | HTS endpoint file          | `foo.hts`                      |\n\n### Client Configuration\n\n```python\nclient = HilltopClient(\n    base_url=\"https://your-server.com\",  # Taken from .env file if not set.\n    hts_endpoint=\"data.hts\",             # Taken from .env file if not set.\n    timeout=60                           # Optional, default 60 seconds\n)\n```\n\n## License\n\nThis project is licensed under the GNU General Public License v3.0 - see the [LICENSE](LICENSE) file for details.\n\n## Contact and Support\n\n- **Author**: Nic Mostert\n- **Email**: nicolas.mostert@horizons.govt.nz\n- **Organization**: Horizons Regional Council\n- **Repository**: https://github.com/HorizonsRC/whurl\n\n## Acknowledgments\n\nDeveloped by Horizons Regional Council for environmental data management and analysis. Special thanks to the Hilltop development team for their comprehensive API documentation.\n\n---\n\n**Version**: 0.1.1  \n**Python Compatibility**: 3.11+  \n**License**: GPL-3.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorizonsrc%2Fwhurl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhorizonsrc%2Fwhurl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorizonsrc%2Fwhurl/lists"}