{"id":15679364,"url":"https://github.com/ccoreilly/wav2vec2-catala","last_synced_at":"2025-05-07T09:32:14.974Z","repository":{"id":107871685,"uuid":"347734808","full_name":"ccoreilly/wav2vec2-catala","owner":"ccoreilly","description":"Wav2Vec 2.0 catalan training scripts and models","archived":false,"fork":false,"pushed_at":"2021-06-18T20:59:30.000Z","size":26519,"stargazers_count":12,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-31T08:43:08.899Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ccoreilly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-14T19:34:02.000Z","updated_at":"2023-05-24T15:52:21.000Z","dependencies_parsed_at":null,"dependency_job_id":"554156b0-3678-4fcd-9897-8ade31952e45","html_url":"https://github.com/ccoreilly/wav2vec2-catala","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccoreilly%2Fwav2vec2-catala","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccoreilly%2Fwav2vec2-catala/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccoreilly%2Fwav2vec2-catala/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ccoreilly%2Fwav2vec2-catala/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ccoreilly","download_url":"https://codeload.github.com/ccoreilly/wav2vec2-catala/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252851141,"owners_count":21814082,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T16:29:41.949Z","updated_at":"2025-05-07T09:32:14.960Z","avatar_url":"https://github.com/ccoreilly.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Wav2Vec2 Català\n\nModels de reconeixement automàtic de la parla Wav2Vec2 pel Català.\n\nS'ha fet fine-tuning a partir de dos models base, el [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) i el [facebook/wav2vec2-large-100k-voxpopuli](https://huggingface.co/facebook/wav2vec2-large-100k-voxpopuli). Els podeu trobar a:\n- [ccoreilly/wav2vec2-large-xlsr-catala](https://huggingface.co/ccoreilly/wav2vec2-large-xlsr-catala).\n- [ccoreilly/wav2vec2-large-100k-voxpopuli-catala](https://huggingface.co/ccoreilly/wav2vec2-large-100k-voxpopuli-catala).\n\nFine-tuned Wav2Vec2 models for the Catalan language based on [facebook/wav2vec2-large-xlsr-53](https://huggingface.co/facebook/wav2vec2-large-xlsr-53) and [facebook/wav2vec2-large-100k-voxpopuli](https://huggingface.co/facebook/wav2vec2-large-100k-voxpopuli)\n\nYou can find the models in the huggingface repository: \n- [ccoreilly/wav2vec2-large-xlsr-catala](https://huggingface.co/ccoreilly/wav2vec2-large-xlsr-catala).\n- [ccoreilly/wav2vec2-large-100k-voxpopuli-catala](https://huggingface.co/ccoreilly/wav2vec2-large-100k-voxpopuli-catala).\n\n## Datasets\n\n- [Common Voice](https://huggingface.co/datasets/common_voice)\n- [ParlamentParla](https://www.openslr.org/59/)\n\n## WER\n\nAvaluada en els següents datasets no vistos durant l'entrenament:\n\nWord error rate was evaluated on the following datasets unseen by the model:\n\n| Dataset | XLSR-53 | VoxPopuli |\n| ------- | --- | --- |\n| [Test split CV+ParlamentParla]((https://github.com/ccoreilly/wav2vec2-catala/blob/master/test-filtered.csv)) | 6,92% | 5.98% |\n| [Google Crowsourced Corpus](https://www.openslr.org/69/) | 12,99% | 12,14% |\n| Audiobook “La llegenda de Sant Jordi” | 13,23% | 12,02% |\n\n\nCom que les dades de CommonVoice contenen metadades sobre l'edat, el gènere i la variant dialectal del parlant, podem avaluar el model segons aquests paràmetres. Desafortunadament, per alguna de les categories no hi ha prou dades com per considerar la mostra significativa, és per això que s'acompanya la taxa d'error amb la mida de la mostra.\n\n| Edat | Mostra | XLSR-53 | VoxPopuli |\n| ------- | --- | --- | --- |\n| 10-19 | 64 | 7,96% | 8,54% |\n| 20-29 | 330 | 7,52% | 6,10% |\n| 30-39 | 377 | 5,65% | 4,55% |\n| 40-49 | 611 | 6,37% | 6,17% |\n| 50-59 | 438 | 5,75% | 5,30% |\n| 60-69 | 166 | 4,82% | 4,20% |\n| 70-79 | 37 | 5,81% | 5,33% |\n\n| Accent | Mostra | XLSR-53 | VoxPopuli |\n| ------- | --- | --- | --- |\n| Balear | 64 | 5,84% | 5,11% |\n| Central | 1202 | 5,98% | 5,37% |\n| Nord-occidental | 140 | 6,60% | 5,77% |\n| Septentrional | 75 | 5,11% | 5,58% |\n| Valencià | 290 | 5,69% | 5,30% |\n\n| Sexe | Mostra | XLSR-53 | VoxPopuli |\n| ------- | --- | --- | --- |\n| Femení | 749 | 5,57% | 4,95% |\n| Masculí | 1280 | 6,65% | 5,98% |\n\n## Com fer-lo servir / Usage\n\n```python\nimport torch\nimport torchaudio\nfrom datasets import load_dataset\nfrom transformers import Wav2Vec2ForCTC, Wav2Vec2Processor\n\ntest_dataset = load_dataset(\"common_voice\", \"ca\", split=\"test[:2%]\")\n\nprocessor = Wav2Vec2Processor.from_pretrained(\"ccoreilly/wav2vec2-large-100k-voxpopuli-catala\") \nmodel = Wav2Vec2ForCTC.from_pretrained(\"ccoreilly/wav2vec2-large-100k-voxpopuli-catala\")\n\nresampler = torchaudio.transforms.Resample(48_000, 16_000)\n\n# Preprocessing the datasets.\n# We need to read the audio files as arrays\ndef speech_file_to_array_fn(batch):\n\tspeech_array, sampling_rate = torchaudio.load(batch[\"path\"])\n\tbatch[\"speech\"] = resampler(speech_array).squeeze().numpy()\n\treturn batch\n\ntest_dataset = test_dataset.map(speech_file_to_array_fn)\ninputs = processor(test_dataset[\"speech\"][:2], sampling_rate=16_000, return_tensors=\"pt\", padding=True)\n\nwith torch.no_grad():\n\tlogits = model(inputs.input_values, attention_mask=inputs.attention_mask).logits\n\npredicted_ids = torch.argmax(logits, dim=-1)\n\nprint(\"Prediction:\", processor.batch_decode(predicted_ids))\nprint(\"Reference:\", test_dataset[\"sentence\"][:2])\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccoreilly%2Fwav2vec2-catala","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fccoreilly%2Fwav2vec2-catala","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fccoreilly%2Fwav2vec2-catala/lists"}