{"id":37708839,"url":"https://github.com/margaritageleta/pixinwav","last_synced_at":"2026-01-16T13:13:12.521Z","repository":{"id":38352274,"uuid":"316819596","full_name":"margaritageleta/PixInWav","owner":"margaritageleta","description":"Hide image in audio. Image in audio steganography with deep learning. Deep learning-based psychoacoustic steganography within earshot.","archived":false,"fork":false,"pushed_at":"2022-09-28T02:38:34.000Z","size":14433,"stargazers_count":26,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-09-05T13:55:40.664Z","etag":null,"topics":["deep-learning","digital-image-processing","digital-signal-processing","pytorch","steganography"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2106.09814","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/margaritageleta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.bib","codeowners":null,"security":null,"support":null}},"created_at":"2020-11-28T20:50:39.000Z","updated_at":"2025-09-04T03:57:21.000Z","dependencies_parsed_at":"2022-08-03T08:31:05.816Z","dependency_job_id":null,"html_url":"https://github.com/margaritageleta/PixInWav","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/margaritageleta/PixInWav","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/margaritageleta%2FPixInWav","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/margaritageleta%2FPixInWav/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/margaritageleta%2FPixInWav/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/margaritageleta%2FPixInWav/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/margaritageleta","download_url":"https://codeload.github.com/margaritageleta/PixInWav/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/margaritageleta%2FPixInWav/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478965,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T11:59:17.896Z","status":"ssl_error","status_checked_at":"2026-01-16T11:55:55.838Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","digital-image-processing","digital-signal-processing","pytorch","steganography"],"created_at":"2026-01-16T13:13:11.593Z","updated_at":"2026-01-16T13:13:12.512Z","avatar_url":"https://github.com/margaritageleta.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PixInWav: Residual Steganography for Hiding Pixels in Audio\n\nThis repository includes a python implemenation of `StegoUNet`, a deep neural network modelling an audio steganographic function. \n\n\u003e Steganography comprises the mechanics of hiding secret data within a cover media which may be publicly available with the main premise that the fact that the communication is  taking  place  is  hidden  as  well. \n\n![alt text](front/img/example.png \"Example\")\n\nIf you find this paper or implementation useful, please consider citing our [ICASSP paper](https://ieeexplore.ieee.org/document/9746191):\n```{tex}\n@INPROCEEDINGS{geleta2021pixinwav,  \n      author={Geleta, Margarita and Puntí, Cristina and McGuinness, Kevin and Pons, Jordi and Canton, Cristian and Giro-i-Nieto, Xavier},  \n      booktitle={ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},   \n      title={Pixinwav: Residual Steganography for Hiding Pixels in Audio},   \n      year={2022},  volume={},  number={},  \n      pages={2485-2489},  \n      doi={10.1109/ICASSP43922.2022.9746191}\n}\n```\nAnd/or the [ArXiv preprint](https://arxiv.org/abs/2106.09814):\n```{tex}\n@misc{geleta2021pixinwav,\n      title={PixInWav: Residual Steganography for Hiding Pixels in Audio}, \n      author={Margarita Geleta and Cristina Punti and Kevin McGuinness and Jordi Pons and Cristian Canton and Xavier Giro-i-Nieto},\n      year={2021},\n      eprint={2106.09814},\n      archivePrefix={arXiv},\n      primaryClass={cs.MM}\n}\n```\n\n## Repository outline\n\nIn the `src` folder we find:\n\n- `umodel.py`: the complete audio steganography model with RGB or B\u0026W  images as input.\n- `loader.py`: the loader script to create the customized dataset from RGB or B\u0026W image (ImageNet) + audio.\n- `trainer_rgb.py`: a script to either train a model from scratch using provided training data or loading a pre-trained `StegoUNet` model for RGB or B\u0026W images.\n- `losses.py`: a script with all the losses and metrics defined for training. Uses a [courtesy script](https://github.com/Po-Hsun-Su/pytorch-ssim) to compute the SSIM metric.\n- `pystct.py`: [courtesy script](https://github.com/jonashaag/pydct) to perform Short-Time Cosine Transform on raw audio waveforms.\n- `pydtw.py`: [courtesy script](https://github.com/Sleepwalking/pytorch-softdtw) to compute SoftDTW as an additional term in the loss function.\n\nIn the `scripts` folder we find:\n\n- `train.sh`: a sample `sbatch` script for Slurm used for sending training jobs.\n\n## Dependencies\n\nFirst, create a virtual environment on your local repository and activate it:\n```\n$ python3 -m venv env\n$ source env/bin/activate\n```\nThe dependencies are listed in `requirements.txt`. Note that you need [PyTorch](https://pytorch.org) v1.7.1 and [TorchAudio](torchaudio) v0.7.2. With `pip` installed, just run:\n```\n$ (env) pip3 install -r requirements.txt\n```\n\n## Data\nWe use [ImageNet](http://image-net.org) (ILSVRC2012) 10,000 images for training and 900 images for validation. Regarding audio, we use [FSDNoisy18K](http://www.eduardofonseca.net/FSDnoisy18k/) which has 17584 audios for training and 946 audios for validation. Each audio has a different duration, in our case we sample randomly different sections of audios that correspond to 1.5 seconds approximately (67522 samples). \n\n## Usage\nAfter the installation of the requirements, to execute the `trainer_rgb.py` script, do:\n```\n$ (env) srun -u --gres=gpu:2,gpumem:12G \n        -p gpi.compute \n        --time 23:59:59 \n        --mem 50G python3 trainer_rgb.py \n        --beta [beta_value] \n        --lr [learning_rate_value] \n        --summary \"[description_of_the_run]\" \n        --experiment [experiment_number]\n        --add_noise [True/False]\n        --noise_kind [gaussian/speckle/salt/pepper/salt\u0026pepper]\n        --noise_amplitude [float]\n        --add_dtw_term [True/False]\n        --rgb [use_rgb_or_b\u0026w_images]\n        --transform [cosine/fourier]\n        --on_phase [if_fourier_hide_on_magnitude_or_phase]\n        --architecture [resindep/resdep/resscale/plaindep]\n```\nReserve as minimum 12G of GPU memory per GPU, otherwise you may be CUDA OOM. Or, run the `sbatch` script as follows:\n```\n$ (env) ./train.sh [experiment_number]\n```\nDefining all the arguments and hyperparameters in the script beforehand.\n\n### Loss function and optimization\n+ `--lr` defined the learning rate of the Adam optimizer.\n+ `--beta` determines the beta parameter of the loss function, refer to the paper for details. \n+ `--add_dtw_term` allows adding an additional term to the loss function. Adding it has shown improvements, refer to the paper for details.\n### Model architecture and constraints\n+ With `--rgb` you can choose to train on RGB or B\u0026W images.\n+ `--architecture` allows to change the underlying architecture. It lists the 4 types of model explained in the paper, refer to it for more details.\n+ With `--transform` you can change the transform to obtain the audio spectrogram. Available transforms include STDCT (Short-Time Discrete Cosine Transform Type II) and STFT (Short-Time Fourier). \n+ If you use STFT, you can choose to hide the image in the magnitude or in the phase. You can control thos behaviour with `--on_phase`.\n### Noise addition\n+ For increasing the robustness of the steganographic function, you can add noise into the audio during training time with `--add_noise`.\n+ If you `--add_noise` then you should choose the `--noise_kind` and `--noise_amplitude`.\n\n### Monitor the training process\nBy default, `wandb` checkpoints are created when you execute the `trainer_rgb.py` script (you should login into your [wandb](https://wandb.ai) account first). This allows tracking the learning curves in the web application.\n\nIf you prefer using `tensorboard` checkpoints, you will need to install `tensorboardX` and add the needed lines of code to save the values. Once it is done, just run in another shell window:\n```\n$ (env) tensorboard dev upload --logdir 'logs/[timestamp]'\n```\nWhere `logs` is the directory you choose to store your logs.\n\n### Training from a checkpoint\nTo train a model from a checkpoint, follow these steps in the `main` function in `trainer_rgb.py`:\n```\n## Load the checkpoint\nchk = torch.load('[checkpoint_path]/[checkpoint_name].pt', map_location='cpu')\nmodel = StegoUNet()\nmodel = nn.DataParallel(model)\n## Load the weights into the model\nmodel.load_state_dict(chk['state_dict'])\n\n[...]\n\ntrain(\n    model=model, \n    tr_loader=train_loader, \n    vd_loader=test_loader, \n    beta=float(args.beta), \n    lr=float(args.lr), \n    epochs=15, \n    slide=15,\n    prev_epoch=chk['epoch'], ## Specify this!\n    prev_i=chk['i'], ## Specify this!\n    summary=args.summary,\n    experiment=int(args.experiment)\n)\n```\n\n## License\n\n**NOTICE**: This software is available for use free of charge for academic research use only. Commercial users, for profit companies or consultants, and non-profit institutions not qualifying as *academic research* must contact `geleta@berkeley.edu` for a separate license. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmargaritageleta%2Fpixinwav","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmargaritageleta%2Fpixinwav","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmargaritageleta%2Fpixinwav/lists"}