{"id":22382055,"url":"https://github.com/googleinterns/deepspeech-reconstruction","last_synced_at":"2025-07-31T03:30:47.924Z","repository":{"id":94623306,"uuid":"358393446","full_name":"googleinterns/deepspeech-reconstruction","owner":"googleinterns","description":null,"archived":true,"fork":false,"pushed_at":"2022-03-22T04:25:23.000Z","size":932,"stargazers_count":3,"open_issues_count":0,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-05T12:41:56.404Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/googleinterns.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-04-15T21:10:17.000Z","updated_at":"2023-05-18T06:08:13.000Z","dependencies_parsed_at":"2023-04-14T07:32:18.249Z","dependency_job_id":null,"html_url":"https://github.com/googleinterns/deepspeech-reconstruction","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/googleinterns/deepspeech-reconstruction","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleinterns%2Fdeepspeech-reconstruction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleinterns%2Fdeepspeech-reconstruction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleinterns%2Fdeepspeech-reconstruction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleinterns%2Fdeepspeech-reconstruction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/googleinterns","download_url":"https://codeload.github.com/googleinterns/deepspeech-reconstruction/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/googleinterns%2Fdeepspeech-reconstruction/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267983362,"owners_count":24176056,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-31T02:00:08.723Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-05T00:11:40.214Z","updated_at":"2025-07-31T03:30:47.912Z","avatar_url":"https://github.com/googleinterns.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Speech Reconstruction\n\nDemonstrate privacy leakage from gradients on DeepSpeech model.\n\n## Reconstruct MFCCs given transcript\n\nGeneral use:\n\n```\n% (optional) Save checkpoint of an untrained model\n./bin/train_cv\n\n% Export one sample and its gradients (files saved to outputs/{}_grads.pkl and outputs/{}_samples.pkl)\n./bin/export_gradients \u003cpath-to-csv\u003e\n\n% Reconstruct audio signal\npython reconstruct.py \\\n    --train_files samples/1-batch-2.csv \\\n    --alphabet_config_path data/alphabet.txt \\\n    --scorer '' \\\n    --load_checkpoint_dir checkpoints --load_train last \\\n    --train_batch_size 1 \\\n    --dropout_rate 0 \\\n    --num_reconstructed_frames 2 --reconstructed_pos random \\\n    --learning_rate 0.1 --force_initialize_learning_rate true \\\n    --reduce_lr_on_plateau --plateau_epochs 2000 --plateau_reduction 0.5 --es_min_delta 0 \\\n    --ema 0 \\\n    --num_iterations 50000 \\\n    --init zero \\\n    --summary_dir logs \\\n    --gradient_distance l2 \\\n    --regularization variation --alpha 0.5\n```\n\n## Experiments\n\n### Preparation\n\n```\nconda create --name speech-reconstruction\nconda activate speech-reconstruction\nconda install -y python=3.7\nconda install -y -c conda-forge ffmpeg\npip install -r requirements.txt\n```\n\nCreate a DeepSpeech checkpoint\n\n```\nCUDA_VISIBLE_DEVICES=0 ./bin/create_checkpoint\n```\n\nRun `export_dataset.ipnyb` in `deep_speaker` to input for reconstruction (change the path to `$HOME/.deep-speaker-wd/...`). This python code should create a bunch of csv files in `samples/librispeech/single/`, each containing a path to the utterance (.wav) and the transcript.\n\n### Reconstruct utterances\n\nReconstruction scripts perform the following steps:\n\n- Export original MFCCs, transcript and gradients (client update) to `outputs/librispeech/\u003cutt_id\u003e` (or some directory in `outputs` depending on the type of reconstruction)\n- Run reconstruction by matching a dummy gradient with a client update loaded from file\n- Store checkpoints (every 1000 iterations) to `outputs/librispeech/\u003cutt_id\u003e/checkpoint-{}.pkl`\n\n**Examples:**\n\nMinimal example:\n\n```\nCUDA_VISIBLE_DEVICES=0 ./examples/reconstruct-single\n```\n\nFor experiments with LibriSpeech's utterances, refer to `./bin/prepare` for preparation of the dataset.\n\nReconstruct single utterance\n\n```\n% Short utterance\n./bin/reconstruct-librispeech/reconstruct-sample-frame-cosine \u003cutt_id\u003e\n\n% Reconstruct utterances with ids stored in a file (chunk_size=8 and chunk_pos=1 mean it only reconstructs utt at 8k+1, used for reconstructing on multiple GPUs)\npython ./bin/reconstruct-librispeech/reconstruct.py -p \u003cpath-to-utt-ids.txt\u003e --bash \u003cpath-to-bash\u003e --output_path \u003cpath-to-output-dir\u003e --chunk_size 8 --chunk_pos 1\n```\n\nReconstruct multiple utterances\n\n```\n% Generate batch by sampling from utterances in test set (options specified in the .py file)\npython src/scripts/librispeech/create_batch.py\n% Reconstruct\npython ./bin/reconstruct-librispeech/reconstruct-batch.py -p \u003cpath-to-batch-ids.txt\u003e --bash \u003cpath-to-bash\u003e --bs 2 --bt 1s-2s --output_path \u003cpath-ot-output-dir\u003e --chunk_size 8 --chunk_pos 1\n```\n\nReconstruct single utterance, multiple step\n\n```\n./bin/reconstruct-librispeech/multi-step/reconstruct-multi-step-{}-lr-{} \u003cutt-id\u003e\n```\n\nReconstruct single utterance with DP-SGD\n\n```\n./bin/reconstruct-librispeech/dpsgd/reconstruct-sample-frame-cosine-noise-{} \u003cutt-id\u003e\n```\n\n### Attack speaker identity\n\nEdit paths in `src/deep_speaker/test_reconstruction.py` to specify a Deep Speaker's pretrained model and location to reconstructed utterances.\n\nSpeaker id results are generated in the same folder for each utterance.\n\n```\n# Generate files for prediction results\nCUDA_VISIBLE_DEVICES=0 python src/deep_speaker/test_reconstruction.py --all_speakers\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleinterns%2Fdeepspeech-reconstruction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogleinterns%2Fdeepspeech-reconstruction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogleinterns%2Fdeepspeech-reconstruction/lists"}