{"id":15222249,"url":"https://github.com/googlecloudplatform/dataflow-contact-center-speech-analysis","last_synced_at":"2025-10-20T01:30:51.446Z","repository":{"id":66041187,"uuid":"264249159","full_name":"GoogleCloudPlatform/dataflow-contact-center-speech-analysis","owner":"GoogleCloudPlatform","description":"Speech Analysis Framework, a collection of components and code from Google Cloud that you can use to transcribe audio files to create analytics.","archived":false,"fork":false,"pushed_at":"2024-05-03T08:49:46.000Z","size":181,"stargazers_count":71,"open_issues_count":2,"forks_count":40,"subscribers_count":21,"default_branch":"main","last_synced_at":"2024-12-18T08:40:17.709Z","etag":null,"topics":["cloud-functions","data-loss-prevention","dataflow","google-cloud","natural-language-processing","speech-to-text"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GoogleCloudPlatform.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-05-15T16:52:26.000Z","updated_at":"2024-11-21T13:26:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"50e852e5-f999-4367-873c-229e9828d7ad","html_url":"https://github.com/GoogleCloudPlatform/dataflow-contact-center-speech-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fdataflow-contact-center-speech-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fdataflow-contact-center-speech-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fdataflow-contact-center-speech-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fdataflow-contact-center-speech-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GoogleCloudPlatform","download_url":"https://codeload.github.com/GoogleCloudPlatform/dataflow-contact-center-speech-analysis/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237243005,"owners_count":19278060,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-functions","data-loss-prevention","dataflow","google-cloud","natural-language-processing","speech-to-text"],"created_at":"2024-09-28T15:11:20.021Z","updated_at":"2025-10-20T01:30:46.093Z","avatar_url":"https://github.com/GoogleCloudPlatform.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Speech Analysis Framework\n\nThis repository contains the Speech Analysis Framework, a collection of components and code from Google Cloud that you can use to transcribe audio, and create a data pipeline workflow to display analytics of the transcribed audio files.\n\nIt can and:\n* Process uploaded audio files to Cloud Storage.\n* Enrich the processed audio files with Cloud Speech-to-Text and Cloud Natural Language APIs.\n* Write the enriched data to BigQuery.\n* Redact sensitive information with Cloud Data Loss Prevention.\n\nYou can:\n* Gain insights into quality metrics to track such as call silence, call duration, agent speaking time, user speaking time, and sentence heat maps.\n* Build visualizations reports\n* Examples of what the visualizations you can build:\n\n![Speech Analysis Framework Sentence Heat Map](images/visualize-speech-data-with-framework-5heatmap.png \"Speech Analysis Framework Sentence Heat Map\")\n\nSpeech Analysis Framework Limitations:\n* The framework can only identify two callers on a stereo or mono audio file. This is a limitation within the Framework code not Cloud Speech-to-Text API.\n* The framework can only process .wav or .flac files. This is a limitation within the Framework code not Cloud Speech-to-Text API.\n\nThe process follows:\n1. An audio file is uploaded to Cloud Storage\n2. The Cloud Function is triggered on object.create\n3. The Cloud Function sends a long running job request to Cloud Speech-to-Text\n4. The Cloud Function then sends the job ID from Cloud Speech-to-Text with additional metadata to Cloud Pub/Sub\n5. The Cloud Dataflow job enriches the data, optionally redacts sensitive information and writes to BigQuery\n\nTo Learn More visit [Visualize speech data with Speech Analysis Framework](hhttps://cloud.google.com/solutions/visualize-speech-data-with-framework)\n\n\n## If you want to do more with DLP audio redaction, check out the [Speech Redaction Framework](https://github.com/GoogleCloudPlatform/dataflow-speech-redaction)\n![Speech Redaction Framework](images/srf-diagram-1.png \"Speech Redaction Framework\")\n\n## Installing the Speech Analysis Framework\n\n1. [Install the Google Cloud SDK](https://cloud.google.com/sdk/install)\n\n2. Create a storage bucket for **Dataflow Staging Files**\n\n``` shell\ngsutil mb gs://[BUCKET_NAME]/\n```\n\n3. Through the [Google Cloud Console](https://console.cloud.google.com) create a folder named **tmp** in the newly created bucket for the DataFlow staging files\n\n4. Create a storage bucket for **Uploaded Audio Files**\n\n``` shell\ngsutil mb gs://[BUCKET_NAME]/\n```\n\n5. Create a BigQuery Dataset\n``` shell\nbq mk [YOUR_BIG_QUERY_DATABASE_NAME]\n```\n\n6. Create Cloud Pub/Sub Topic\n``` shell\ngcloud pubsub topics create [YOUR_TOPIC_NAME]\n```\n\n7. Enable Cloud Dataflow API\n``` shell\ngcloud services enable dataflow\n```\n\n8. Enable Cloud Speech-to-Text API\n``` shell\ngcloud services enable speech\n```\n\n9. Enable Cloud Natural Language API\n``` shell\ngcloud services enable language.googleapis.com\n```\n\n10. Enable DLP __Optional__\n``` shell\ngcloud services enable dlp.googleapis.com\n```\n\n11. Deploy the Google Cloud Function\n* In the cloned repo, go to the “saf-longrun-job-func” directory and deploy the following Cloud Function.\n``` shell\ngcloud functions deploy safLongRunJobFunc --region=us-central1 --stage-bucket=[YOUR_UPLOADED_AUDIO_FILES_BUCKET_NAME] --runtime=nodejs8 --trigger-event=google.storage.object.finalize --trigger-resource=[YOUR_UPLOADED_AUDIO_FILES_BUCKET_NAME]\n```\n\n12. Deploy the Cloud Dataflow Pipeline\n* python3 --version Python 3.7.8\n* In the cloned repo, go to “saf-longrun-job-dataflow” directory and deploy the Cloud Dataflow Pipeline. Run the commands below to deploy the dataflow job.\n``` shell\n# Apple/Linux\npython3 -m venv env\nsource env/bin/activate\npip3 install apache-beam[gcp]\npip3 install dateparser\n```\nor\n``` shell\n# Windows\npython3 -m venv env\nenv\\Scripts\\activate\npip3 install apache-beam[gcp]\npip3 install dateparser\n```\n* The Dataflow job will create the **BigQuery Table** you listed in the parameters.\n* Please wait as it might take a few minutes to complete.\n``` shell\npython3 saflongrunjobdataflow.py --project=[YOUR_PROJECT_ID] --input_topic=projects/[YOUR_PROJECT_ID]/topics/[YOUR_TOPIC_NAME] --runner=DataflowRunner --temp_location=gs://[YOUR_DATAFLOW_STAGING_BUCKET]/tmp --output_bigquery=[DATASET NAME].[TABLE] --requirements_file=\"requirements.txt\"\n```\n\n13. In the cloned repo, go to “sample-audio-files” to locate sample audio files to process by Speech Analysis Framework\n* For the [TOPIC_NAME], do not include the full path, just the name of the TOPIC\n* Choose true or false to run DLP. DLP will use all info types to scan the data.\n\n``` shell\n# stereo wav audio sample\ngsutil -h x-goog-meta-dlp:[true or false] -h x-goog-meta-callid:1234567 -h x-goog-meta-stereo:true -h x-goog-meta-pubsubtopicname:[TOPIC_NAME] -h x-goog-meta-year:2019 -h x-goog-meta-month:11 -h x-goog-meta-day:06 -h x-goog-meta-starttime:1116 cp [YOUR_FILE_NAME.wav] gs://[YOUR_UPLOADED_AUDIO_FILES_BUCKET_NAME]\n```\n\n``` shell\n# mono flac audio sample\ngsutil -h x-goog-meta-dlp:[true or false] -h x-goog-meta-callid:1234567 -h x-goog-meta-stereo:false -h x-goog-meta-pubsubtopicname:[TOPIC_NAME] -h x-goog-meta-year:2019 -h x-goog-meta-month:11 -h x-goog-meta-day:06 -h x-goog-meta-starttime:1116 cp [YOUR_FILE_NAME.flac] gs://[YOUR_UPLOADED_AUDIO_FILES_BUCKET_NAME]\n```\n\n14. After a few minutes you will be able to see the data in BigQuery.\n* Sample select statements that can be executed in the BigQuery console.\n``` sql\n-- Order Natural Language Entities for all records\nSELECT\n  *\nFROM (\n  SELECT\n    entities.name,\n    entities.type,\n    COUNT(entities.name) AS count\n  FROM\n    `[YOUR_PROJECT_ID].[YOUR_DATASET].[YOUR_TABLE]`,\n    UNNEST(entities) entities\n  GROUP BY\n    entities.name,\n    entities.type\n  ORDER BY\n    count DESC )\n```\n\n``` sql\n-- List word, start time, end time, speaker tag and confidence for all records\nSELECT\n  ARRAY(\n  SELECT\n    AS STRUCT word,\n    startSecs,\n    endSecs,\n    speakertag,\n    confidence\n  FROM\n    UNNEST(words)) transcript\nFROM\n  `[YOUR_PROJECT_ID].[YOUR_DATASET].[YOUR_TABLE]`\n```\n\n``` sql\n-- Search Transcript with a regular expression\nSELECT\n  transcript,\n  fileid,\n  callid,\n  year,\n  month,\n  day,\n  sentimentscore,\n  magnitude,\n  date,\n  silencesecs\nFROM\n  `[YOUR_PROJECT_ID].[YOUR_DATASET].[YOUR_TABLE]`\nWHERE\n  (REGEXP_CONTAINS(transcript, '(?i) [YOUR_WORD]' ))\n```\n\n**This is not an officially supported Google product**","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fdataflow-contact-center-speech-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgooglecloudplatform%2Fdataflow-contact-center-speech-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fdataflow-contact-center-speech-analysis/lists"}