{"id":15668091,"url":"https://github.com/evadne/gen_magic","last_synced_at":"2025-04-16T03:44:19.740Z","repository":{"id":45145230,"uuid":"195120838","full_name":"evadne/gen_magic","owner":"evadne","description":"Fantasia in Elixir — Binary-level file content identification, powered by libmagic","archived":false,"fork":false,"pushed_at":"2023-07-13T14:55:09.000Z","size":68,"stargazers_count":27,"open_issues_count":2,"forks_count":6,"subscribers_count":4,"default_branch":"develop","last_synced_at":"2025-03-29T04:43:01.086Z","etag":null,"topics":["elixir","magic"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/evadne.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-07-03T20:15:51.000Z","updated_at":"2025-02-27T05:21:40.000Z","dependencies_parsed_at":"2024-06-13T07:33:12.507Z","dependency_job_id":null,"html_url":"https://github.com/evadne/gen_magic","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evadne%2Fgen_magic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evadne%2Fgen_magic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evadne%2Fgen_magic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/evadne%2Fgen_magic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/evadne","download_url":"https://codeload.github.com/evadne/gen_magic/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249191936,"owners_count":21227683,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","magic"],"created_at":"2024-10-03T14:06:27.839Z","updated_at":"2025-04-16T03:44:19.721Z","avatar_url":"https://github.com/evadne.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GenMagic\n\n[![Build Status](https://travis-ci.org/evadne/gen_magic.svg?branch=develop)](https://travis-ci.org/evadne/gen_magic)\n\n**GenMagic** provides supervised and customisable access to [libmagic](http://man7.org/linux/man-pages/man3/libmagic.3.html) using a supervised external process.\n\nWith this library, you can start an one-off process to run a single check, or run the process as a daemon if you expect to run many checks.\n\n## Installation\n\nThe package can be installed by adding `gen_magic` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:gen_magic, \"~\u003e 1.1.1\"}\n  ]\nend\n```\n\nYou must also have [libmagic](http://man7.org/linux/man-pages/man3/libmagic.3.html) installed locally with headers, alongside common compilation tools (i.e. build-essential). These can be acquired by apt-get, yum, brew, etc.\n\n-  On Debian Linux, install [libmagic-dev](https://packages.debian.org/sid/libmagic-dev) to get the headers.\n\n-  On Alpine Linux, install [file-dev](https://pkgs.alpinelinux.org/package/edge/main/x86_64/file-dev) to get the headers.\n\n-  On macOS, install [libmagic](https://formulae.brew.sh/formula/libmagic) via Homebrew to get everything.\n\nAdditionally, [pkg-config](https://www.freedesktop.org/wiki/Software/pkg-config/) is required as it is used to locate the correct version of libmagic during compilation.\n\nCompilation of the underlying C program is automatic and handled by [elixir_make](https://github.com/elixir-lang/elixir_make).\n\n## Usage\n\nDepending on the use case, you may utilise a single (one-off) GenMagic process without reusing it as a daemon, or utilise a connection pool (such as Poolboy) in your application to run multiple persistent GenMagic processes.\n\n### Direct Usage\n\nTo use GenMagic directly, you can use `GenMagic.Helpers.perform_once/1`:\n\n```elixir\niex(1)\u003e GenMagic.Helpers.perform_once \".\"\n{:ok,\n %GenMagic.Result{\n   content: \"directory\",\n   encoding: \"binary\",\n   mime_type: \"inode/directory\"\n }}\n```\n\nNotes:\n\n1.  See `GenMagic.Server.start_link/1` and `t:GenMagic.Server.option/0` for more information on startup parameters.\n\n2.  See `GenMagic.Result` for details on the result provided.\n\n### Pooled Usage\n\nTo use the GenMagic server as a daemon, you can start it first, keep a reference, then feed messages to it as you require:\n\n```elixir\n{:ok, pid} = GenMagic.Server.start_link([])\n{:ok, result} = GenMagic.Server.perform(pid, path)\n```\n\nIf you wish to use a pool, the following pool implementations are bundled:\n\n- `GenMagic.Pool.Poolboy`\n- `GenMagic.Pool.NimblePool`\n\n## Configuration\n\nWhen using `GenMagic.Server.start_link/1` to start a persistent server, or `GenMagic.Helpers.perform_once/2` to run an ad-hoc request, you can override specific options to suit your use case.\n\n| Name | Default | Description |\n| - | - | - |\n| `:startup_timeout` | 1000 | Number of milliseconds to wait for client startup |\n| `:process_timeout` | 30000 | Number of milliseconds to process each request |\n| `:recycle_threshold` | 10 | Number of cycles before the C process is replaced |\n| `:database_patterns` | `[:default]` | Databases to load |\n\nSee `t:GenMagic.Server.option/0` for details.\n\n### Use Cases\n\n### Ad-Hoc Requests\n\nFor ad-hoc requests, you can use the helper method `GenMagic.Helpers.perform_once/2`:\n\n```elixir\niex(1)\u003e GenMagic.Helpers.perform_once(Path.join(File.cwd!(), \"Makefile\"))\n{:ok,\n %GenMagic.Result{\n   content: \"makefile script, ASCII text\",\n   encoding: \"us-ascii\",\n   mime_type: \"text/x-makefile\"\n}}\n```\n\n### Supervised Requests\n\nThe Server should be run under a supervisor which provides resiliency.\n\nHere we run it under a supervisor:\n\n```elixir\niex(1)\u003e {:ok, pid} = Supervisor.start_link([{GenMagic.Server, name: :gen_magic}], strategy: :one_for_one)\n{:ok, #PID\u003c0.199.0\u003e}\n```\n\nNow we can ask it to inspect a file:\n\n```elixir\niex(2)\u003e GenMagic.Server.perform(:gen_magic, Path.expand(\"~/.bash_history\"))\n%GenMagic.Result{\n content: \"ASCII text\",\n encoding: \"us-ascii\",\n mime_type: \"text/plain\"\n}}\n```\n\nNote that in this case we have opted to use a named process.\n\n### Pooled Requests\n\nFor concurrency *and* resiliency, you can use GenMagic in a pool.\n\nYou can add a pool in your application supervisor by adding it as a child:\n\n\n```elixir\nchildren = [\n  {GenMagic.Pool.NimblePool, pool_name: MyApp.GenMagicPool, pool_size: 2}\n]\n\nopts = [strategy: :one_for_one, name: MyApp.Supervisor]\nSupervisor.start_link(children, opts)\n```\n\nAnd then you can use it with `c:GenMagic.Pool.perform/3`:\n\n\n```elixir\niex(1)\u003e GenMagic.Pool.NimblePool.perform(MyApp.GenMagicPool, Path.expand(\"~/.bash_history\"), [])\n{:ok, …}\n```\n\n### Check Uploaded Files\n\nIf you use Phoenix, you can inspect the file from your controller:\n\n```elixir\ndef upload(conn, %{\"upload\" =\u003e %{path: path}}) do,\n  {:ok, result} = GenMagic.Helpers.perform_once(:gen_magic, path)\n  text(conn, \"Received your file containing #{result.content}\")\nend\n```\n\nObviously, it will be more ideal if you have wrapped `GenMagic.Server` in a pool, to avoid constantly starting and stopping the underlying C program.\n\n## Notes\n\n### Soak Test\n\nRun an endless cycle to prove that the program is resilient:\n\n```bash\nfind /usr/share/ -name *png | xargs mix run test/soak.exs\nfind . -name *ex | xargs mix run test/soak.exs\n```\n\n### Debian Linux Test\n\nThe Debian Linux image is based on the [official Elixir image](https://github.com/c0b/docker-elixir).\n\n```bash\ndocker run --rm -it $(docker build -q -f ./infra/docker-app-test/Dockerfile .) mix test\n```\n\n### Alpine Linux Test\n\nThe Alpine Linux image is based on [Bitwalker’s Elixir on Alpine Linux image](https://github.com/bitwalker/alpine-elixir).\n\n```bash\ndocker run --rm -it $(docker build -q -f ./infra/docker-app-test-alpine/Dockerfile .) mix test\n```\n\n## Acknowledgements\n\nDuring design and prototype development of this library, the Author has drawn inspiration from the following individuals, and therefore thanks all contributors for their generosity:\n\n- [devstopfix](https://github.com/devstopfix)\n  - Enhanced Elixir Wrapper (based on GenServer)\n  - Initial Hex packaging (v.0.22)\n  - Soak testing\n\n- [hrefhref](https://github.com/hrefhref)\n  - Valgrind rework\n  - Alpine Linux testing\n\n- [Kleidukos](https://github.com/Kleidukos)\n  - Makefile rework\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevadne%2Fgen_magic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fevadne%2Fgen_magic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fevadne%2Fgen_magic/lists"}