{"id":17568732,"url":"https://github.com/pmhalvor/whale-speech","last_synced_at":"2026-02-24T07:40:16.203Z","repository":{"id":256842373,"uuid":"856550501","full_name":"pmhalvor/whale-speech","owner":"pmhalvor","description":"A pipeline to map whale sightings to hydrophone audio","archived":false,"fork":false,"pushed_at":"2025-04-22T11:18:04.000Z","size":17141,"stargazers_count":1,"open_issues_count":9,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-09T00:34:38.293Z","etag":null,"topics":["beam","bigquery","gcs","mle","model-as-a-service","python","tensorflow2"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pmhalvor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-09-12T19:10:54.000Z","updated_at":"2025-04-22T11:18:08.000Z","dependencies_parsed_at":"2025-08-22T08:35:54.401Z","dependency_job_id":"4f6c8df3-2a9e-45d3-b612-8a997fb67bc6","html_url":"https://github.com/pmhalvor/whale-speech","commit_stats":null,"previous_names":["pmhalvor/whale-speech"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pmhalvor/whale-speech","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmhalvor%2Fwhale-speech","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmhalvor%2Fwhale-speech/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmhalvor%2Fwhale-speech/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmhalvor%2Fwhale-speech/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pmhalvor","download_url":"https://codeload.github.com/pmhalvor/whale-speech/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pmhalvor%2Fwhale-speech/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29775194,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-24T04:54:30.205Z","status":"ssl_error","status_checked_at":"2026-02-24T04:53:58.628Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beam","bigquery","gcs","mle","model-as-a-service","python","tensorflow2"],"created_at":"2024-10-21T17:05:36.839Z","updated_at":"2026-02-24T07:40:16.161Z","avatar_url":"https://github.com/pmhalvor.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 📣🐋 Whale Speech \nA pipeline to map whale encounters to hydrophone audio.\n\n\u003csub\u003e\nDerived from \u003ca href=\"https://docs.mbari.org/pacific-sound/notebooks/humpbackwhales/detect/PacificSoundDetectHumpbackSong/\"\u003e PacificSoundDetectHumpbackSong\u003c/a\u003e, though not directly affiliated with MBARI, NOAA, or HappyWhale.\n\u003c/sub\u003e\n\n\n## Pipeline description\n\nStages:\n1. **Input**: When (and where*) to look for whale encounters on [HappyWhale](https://happywhale.com/).\n2. **Geometry Search**: Query [open-oceans/happywhale](https://github.com/open-oceans/happywhale) to find potential whale encounters. \n\n   \u0026rarr; Expected outputs: encounter ids, start and end times, and longitude and latitude.\n\n3. **Retrive Audio**: Download audio from MBARI's [Pacific Ocean Sound Recordings](https://registry.opendata.aws/pacific-sound/) around the time of the encounter. \n    \n    \u0026rarr; Expected outputs: audio array, start and end times, and encounter ids.\n    \n4. **Parse Audio**: Break audio into non-overlaping segments with flagged frequency detections. \n        \n    \u0026rarr; Expected outputs: cut audio array, detection intervals, and encounter ids.\n\n5. **Classify Audio**: Use a NOAA and Google's [humpback_whale model](https://tfhub.dev/google/humpback_whale/1) to classify the flagged segments.\n\n    \u0026rarr; Expected outputs: resampled audio, classification score array, and encounter ids.\n\n6. **Postprocess Labels**: Build clip-intervals for each encounter for playback snippets.\n\n    \u0026rarr; Expected outputs: encounter ids, cut/resampled audio array, and aggregated classification score.\n\n7. **Output**: Map the whale encounter ids to the playback snippets.\n\n\u003c!-- Light mode --\u003e\n[![](https://mermaid.ink/img/pako:eNpVkttOwkAQhl9lMleaFIJFTo0x4SBIItGoV1ouhnZKm2y7ZA9oJby7S1uJzNX-s98cMweMZMwYYCLkV5SSMvA-CwtwNv5MKEioFZHgIiYFy2JnjV5Dq3UPk6v6cyvkhmHBMmejSnhjUlF63SSoyGlD5lZnEbw6LNszjG2cyYab1FwtppV4aIKSTBhW8EJKX8Y8VNi8wTZWCM0lw1SQ1llSXrBzuGu3Hb1saCU30sDKzS1cw2rP6gyeki4azNAWXqQ2OyUj1hqeaMNCN-iiQh8__9rUKTxb4_az_j_TAk4KPcxZ5ZTFbs-HkydEk3LOIQbuGXNCVpgQw-LoULJGvpVFhIFRlj1U0m5TdFXchB7aXUyGZxltFeVn746KDykvNAYH_Mag2_HbN0P_ZtjvjUa3frff9bDEoHP08KeK6LRHtQ38nt8b3A4HHnKcGalW9WFU93H8BWH3qDQ?type=png)](https://mermaid.live/edit#pako:eNpVkttOwkAQhl9lMleaFIJFTo0x4SBIItGoV1ouhnZKm2y7ZA9oJby7S1uJzNX-s98cMweMZMwYYCLkV5SSMvA-CwtwNv5MKEioFZHgIiYFy2JnjV5Dq3UPk6v6cyvkhmHBMmejSnhjUlF63SSoyGlD5lZnEbw6LNszjG2cyYab1FwtppV4aIKSTBhW8EJKX8Y8VNi8wTZWCM0lw1SQ1llSXrBzuGu3Hb1saCU30sDKzS1cw2rP6gyeki4azNAWXqQ2OyUj1hqeaMNCN-iiQh8__9rUKTxb4_az_j_TAk4KPcxZ5ZTFbs-HkydEk3LOIQbuGXNCVpgQw-LoULJGvpVFhIFRlj1U0m5TdFXchB7aXUyGZxltFeVn746KDykvNAYH_Mag2_HbN0P_ZtjvjUa3frff9bDEoHP08KeK6LRHtQ38nt8b3A4HHnKcGalW9WFU93H8BWH3qDQ)\n\n\u003c!-- Dark mode --\u003e\n\u003c!-- [![](https://mermaid.ink/img/pako:eNpVkttOwkAQhl9lMleaFALl3BgTzpJIJOKVlIttO6WN2y7ZA1oJ7-7SViNztf_sN8fMGUMREXoYc_EZJkxqeJv5OVgb72LmxawRMk55xCSs8qPRag-NxiNM7qrPAxcBwZJERloWsCUmw-S-TlCS05rMjEpDeLVYeiIYmygVNTepuEpMSzGvg-KUa5KwYVLdxsxLbFFjgeFcUUEw5UypNC5u2AU8NJuWXtW0FIHQsLZzc9uwPJH8A69JlzWm2QE2QumjFCEpBc8sIK5qdFmiT7vfNlUCL0bb_ez_z7SEq0IHM5IZSyO75_PV46NOKCMfPfu0q_3w0c8vlmNGi22Rh-hpachBKcwhQVvCjuegOUZM0yxlB8myP--R5e9C3Gj0zviFXqflNttDtz3s90ajrtvpdxws0GtdHPwuI1rNUWUDt-f2Bt3hwEGKUi3kurqK8jguPzO5pvE?type=png)](https://mermaid.live/edit#pako:eNpVkttOwkAQhl9lMleaFALl3BgTzpJIJOKVlIttO6WN2y7ZA1oJ7-7SViNztf_sN8fMGUMREXoYc_EZJkxqeJv5OVgb72LmxawRMk55xCSs8qPRag-NxiNM7qrPAxcBwZJERloWsCUmw-S-TlCS05rMjEpDeLVYeiIYmygVNTepuEpMSzGvg-KUa5KwYVLdxsxLbFFjgeFcUUEw5UypNC5u2AU8NJuWXtW0FIHQsLZzc9uwPJH8A69JlzWm2QE2QumjFCEpBc8sIK5qdFmiT7vfNlUCL0bb_ez_z7SEq0IHM5IZSyO75_PV46NOKCMfPfu0q_3w0c8vlmNGi22Rh-hpachBKcwhQVvCjuegOUZM0yxlB8myP--R5e9C3Gj0zviFXqflNttDtz3s90ajrtvpdxws0GtdHPwuI1rNUWUDt-f2Bt3hwEGKUi3kurqK8jguPzO5pvE) --\u003e\n\n\n\n\n\u003csub\u003e\n*Currently only support encounters around the Monterey Bay Hydrophone (\u003ca href=\"https://www.mbari.org/technology/monterey-accelerated-research-system-mars/\"\u003eMARS\u003c/a\u003e).\n\u003c/sub\u003e\n\n\u003cbr\u003e\n\n## Getting started\n\n### Install\n\nCreate a virtual environment and install the required packages.\nWe'll use conda for this, but you can use any package manager you prefer.\n\nSince we're developing on an M1 machine, we'll need to specify the `CONDA_SUBDIR` to `osx-arm64`.\nThis step should be adapted based on the virtual environment you're using.\n\n#### M1:\n```bash\nCONDA_SUBDIR=osx-arm64 conda create -n whale-speech python=3.11\nconda activate whale-speech\npip install -r requirements.txt\n```\n\n\n#### Other:\n```bash\nconda create -n whale-speech python=3.11\nconda activate whale-speech\npip install -r requirements.txt\n```\n\n### Google Cloud SDK\nTo run the pipeline on Google Cloud Dataflow, you'll need to install the Google Cloud SDK.\nYou can find the installation instructions [here](https://cloud.google.com/sdk/docs/install).\n\nMake sure you authentication your using and initialize the project you are using.\n```bash\ngcloud auth login\ngcloud init\n```\n\nFor newly created projects, each of the services used will need to be enabled. \nThis can be easily done in the console, or via the command line. \nFor example:\n```bash\ngcloud services enable bigquery.googleapis.com\ngcloud services enable dataflow.googleapis.com\ngcloud services enable storage-api.googleapis.com\ngcloud services enable run.googleapis.com\n```\n\n### Run locally \nTo run the pipeline and model server locally, you can use the `make` target `local-run`.\n\n```bash\nmake local-run\n```\n\nThis target starts by killing any previous model servers that might be running (needed for when a pipeline fails, without tearing down the server, causing the previous call to hang). \nThen it starts the model server in the background and runs the pipeline.\n\n\n### Build and push the model server\nTo build and push the model server to your model registry (stored as an environment variable), you can use the following `make` target.\n\n```bash\nmake build-push-model-server\n```\nThis target builds the model server image and pushes it to the registry specified in the `env.sh` file.\nThe tag is a combination of the version set in the makefile and the last git commit hash. \nThis helps keep track of what is included in the image, and allows for easy rollback if needed.\nThe target fails if there are any uncommited changes in the git repository.\n\nThe `latest` tag is only added to images deployed via GHA.\n\n### Run pipeline with Dataflow\nTo run the pipeline on Google Cloud Dataflow, you can use the following `make` target.\n\n```bash\nmake run-dataflow\n```\nLogging in the terminal will tell you the status of the pipeline, and you can follow the progress in the [Dataflow console](https://console.cloud.google.com/dataflow/jobs).\n\nIn addition to providing the inference url and filesystem to store outputs on, the definition of the above target also provides an example on how a user can pass additional arguments to and request different resources for the pipeline run. \n\n**Pipeline specific parameters**\nYou can configure all the paramters set in the config files directly when running the pipeline.\nThe most important here is probably the start and end time for the initial search. \n\n```bash\n\t\t--start \"2024-07-11\" \\\n\t\t--end \"2024-07-11\" \\\n\t\t--offset 0 \\\n\t\t--margin 1800 \\\n\t\t--batch_duration 60 \n```\n\nNote that any parameters with the same name under different sections will only be updated if its the last section in the list. \nAlso, since these argparse-parameters are added automatically, behavior of boolean flags might be unexpected (always true is added). \n\u003c!-- TODO fix behavior of boolean in-line parameters --\u003e\n\n**Compute resources**\nThe default compute resources are quite small and slow. To speed things up, you can request more workers and a larger machine type. For more on Dataflow resources, check out [the docs](https://cloud.google.com/dataflow/docs/reference/pipeline-options#worker-level_options).\n```\n\t\t--worker_machine_type=n1-highmem-8 \\\n\t\t--disk_size_gb=100 \\\n\t\t--num_workers=8 \\\n\t\t--max_num_workers=8 \\\n```\n\n\nNote, you may need to configure IAM permissions to allow Dataflow Runners to access images in your Artifact Registry. Read more about that [here](https://cloud.google.com/dataflow/docs/concepts/security-and-permissions).\n\n\n## References \n- [HappyWhale](https://happywhale.com/)\n- [open-oceans/happywhale](https://github.com/open-oceans/happywhale)\n- [NOAA and Google's humpback_whale model](https://tfhub.dev/google/humpback_whale/1)\n- [Google Cloud Console](https://console.cloud.google.com/)\n- [Monterey Bay Hydrophone MARS](https://www.mbari.org/technology/monterey-accelerated-research-system-mars/)\n- [MBARI's Pacific Ocean Sound Recordings](https://registry.opendata.aws/pacific-sound/)\n- J. Ryan et al., \"New Passive Acoustic Monitoring in Monterey Bay National Marine Sanctuary,\" OCEANS 2016 MTS/IEEE Monterey, Monterey, CA, USA, 2016, pp. 1-8, doi: [10.1109/OCEANS.2016.7761363](https://ieeexplore.ieee.org/document/7761363).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmhalvor%2Fwhale-speech","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpmhalvor%2Fwhale-speech","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpmhalvor%2Fwhale-speech/lists"}