{"id":17662506,"url":"https://github.com/tommoral/dicodile","last_synced_at":"2025-08-19T22:32:18.468Z","repository":{"id":43690065,"uuid":"167703861","full_name":"tomMoral/dicodile","owner":"tomMoral","description":"Experiments for \"Distributed Convolutional Dictionary Learning (DiCoDiLe): Pattern Discovery in Large Images and Signals\"","archived":false,"fork":false,"pushed_at":"2024-12-09T16:52:44.000Z","size":34301,"stargazers_count":19,"open_issues_count":14,"forks_count":10,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-12-09T17:46:42.290Z","etag":null,"topics":["convolutional-dictionary-learning","distributed-computing","pattern-discovery","signals"],"latest_commit_sha":null,"homepage":"https://tommoral.github.io/dicodile/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tomMoral.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-26T15:26:24.000Z","updated_at":"2024-12-09T16:52:49.000Z","dependencies_parsed_at":"2024-10-23T20:37:18.383Z","dependency_job_id":"24729bf3-d5e1-4fbf-89d5-a02d4be9eb3d","html_url":"https://github.com/tomMoral/dicodile","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomMoral%2Fdicodile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomMoral%2Fdicodile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomMoral%2Fdicodile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomMoral%2Fdicodile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tomMoral","download_url":"https://codeload.github.com/tomMoral/dicodile/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230374118,"owners_count":18216041,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["convolutional-dictionary-learning","distributed-computing","pattern-discovery","signals"],"created_at":"2024-10-23T18:22:44.009Z","updated_at":"2024-12-19T04:07:03.201Z","avatar_url":"https://github.com/tomMoral.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"|Build Status| |codecov|\n\nThis package is still under development. If you have any trouble running this code,\nplease `open an issue on GitHub \u003chttps://github.com/tomMoral/dicodile/issues\u003e`_.\n\nDiCoDiLe\n--------\n\nPackage to run the experiments for the preprint paper `DiCoDiLe: Distributed Convolutional Dictionary Learning \u003chttps://hal.science/hal-02371715v2/file/PAMI_dicodile.pdf\u003e`__.\n\nInstallation\n^^^^^^^^^^^^\n\nAll the tests should work with python \u003e=3.6. This package depends on the python\nlibrary ``numpy``, ``matplotlib``, ``scipy``, ``mpi4py``, ``joblib``. The\npackage can be installed with the following command run from the root of the\npackage.\n\n.. code:: bash\n\n    pip install  -e .\n\nOr using the conda environment:\n\n.. code:: bash\n\n    conda env create -f dicodile_env.yml\n\nTo build the doc use:\n\n.. code:: bash\n\n    pip install  -e .[doc]\n    cd docs\n    make html\n\nTo run the tests:\n\n.. code:: bash\n\n    pip install  -e .[test]\n    pytest .\n\nUsage\n^^^^^\n\nAll experiments are with ``mpi4py`` and will try to spawned workers depending on the parameters set in the experiments. If you need to use an ``hostfile`` to configure indicate to MPI where to spawn the new workers, you can set the environment variable ``MPI_HOSTFILE=/path/to/the/hostfile`` and it will be automatically detected in all the experiments. Note that for each experiments you should provide enough workers to allow the script to run.\n\nAll figures can be generated using scripts in ``benchmarks``. Each script will generate and save the data to reproduce the figure. The figure can then be plotted by re-running the same script with the argument ``--plot``. The figures are saved in pdf in the ``benchmarks_results`` folder. The computation are cached with ``joblib`` to be robust to failures.\n\n.. note::\n\n   Open MPI tries to use all **up** network interfaces. This might cause the program to hang due to virtual network interfaces which could not actually be used to communicate with MPI processes. For more info `Open MPI FAQ \u003chttps://www.open-mpi.org/faq/?category=tcp#tcp-selection\u003e`_.\n\n   In case your program hangs, you can launch computation with the ``mpirun`` command:\n\n   - either spefifying usable interfaces using ``--mca btl_tcp_if_include`` parameter:\n\n   .. code-block:: bash\n\n\t $ mpirun -np 1 \\\n\t\t  --mca btl_tcp_if_include wlp2s0 \\\n\t\t  --hostfile hostfile \\\n\t\t  python -m mpi4py examples/plot_mandrill.py\n\n   - or by excluding the virtual interfaces using ``--mca btl_tcp_if_exclude`` parameter:\n\n   .. code-block:: bash\n\n\t $ mpirun -np 1 \\\n\t\t  --mca btl_tcp_if_exclude docker0 \\\n\t\t  --hostfile hostfile \\\n\t\t  python -m mpi4py examples/plot_mandrill.py\n\nAlternatively, you can also restrict the used interface by setting environment variables ``OMPI_MCA_btl_tcp_if_include`` or ``OMPI_MCA_btl_tcp_if_exclude``\n\n   .. code-block:: bash\n\n\t $ export OMPI_MCA_btl_tcp_if_include=\"wlp2s0\"\n\n\t $ export OMPI_MCA_btl_tcp_if_exclude=\"docker0\"``\n\n\n.. |Build Status| image:: https://github.com/tomMoral/dicodile/workflows/unittests/badge.svg\n.. |codecov| image:: https://codecov.io/gh/tomMoral/dicodile/branch/main/graph/badge.svg\n   :target: https://codecov.io/gh/tomMoral/dicodile\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftommoral%2Fdicodile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftommoral%2Fdicodile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftommoral%2Fdicodile/lists"}