{"id":32173534,"url":"https://github.com/digital-identity-labs/smee","last_synced_at":"2026-02-19T07:02:57.739Z","repository":{"id":126850406,"uuid":"583413349","full_name":"Digital-Identity-Labs/smee","owner":"Digital-Identity-Labs","description":"SAML Metadata Entity Extractor (etc)","archived":false,"fork":false,"pushed_at":"2026-01-26T19:05:16.000Z","size":729,"stargazers_count":1,"open_issues_count":41,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-01-27T05:47:49.679Z","etag":null,"topics":["elixir","metadata","metadata-extraction","metadata-management","saml"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Digital-Identity-Labs.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-12-29T17:50:03.000Z","updated_at":"2026-01-26T19:05:20.000Z","dependencies_parsed_at":"2024-04-06T20:27:01.216Z","dependency_job_id":"fa7cff8e-0f4d-4fec-9846-3536ab43c06a","html_url":"https://github.com/Digital-Identity-Labs/smee","commit_stats":null,"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Digital-Identity-Labs/smee","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Digital-Identity-Labs%2Fsmee","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Digital-Identity-Labs%2Fsmee/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Digital-Identity-Labs%2Fsmee/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Digital-Identity-Labs%2Fsmee/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Digital-Identity-Labs","download_url":"https://codeload.github.com/Digital-Identity-Labs/smee/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Digital-Identity-Labs%2Fsmee/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29605806,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T06:47:36.664Z","status":"ssl_error","status_checked_at":"2026-02-19T06:45:47.551Z","response_time":117,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","metadata","metadata-extraction","metadata-management","saml"],"created_at":"2025-10-21T18:57:35.225Z","updated_at":"2026-02-19T07:02:57.726Z","avatar_url":"https://github.com/Digital-Identity-Labs.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Smee\n\u003cimg src=\"https://raw.githubusercontent.com/Digital-Identity-Labs/smee/a897646d54d5c6c7ef852b11a0e5d64611147580/logo.png\" width=\"128px\" alt=\"Smee Logo\" style=\"float: right; margin: 6px;\"\u003e\n\n`Smee` is a pragmatic library for handling SAML metadata with Elixir, Erlang or any other BEAM language.\n\nSmee started life as an sprawling mess of code used over five years in an internal application to download and process\nSAML metadata. It's now been extracted, restructured and cleaned up, with the aim of reusing it in other \nprojects. Smee can be used within applications or in simpler scripts for processing metadata.\n\n[![Hex pm](http://img.shields.io/hexpm/v/smee.svg?style=flat)](https://hex.pm/packages/smee)\n[![API Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](http://hexdocs.pm/smee/)\n![Github Elixir CI](https://github.com/Digital-Identity-Labs/smee/workflows/Elixir%20CI/badge.svg)\n\n[![Run in Livebook](https://livebook.dev/badge/v1/blue.svg)](https://livebook.dev/run?url=https%3A%2F%2Fraw.githubusercontent.com%2FDigital-Identity-Labs%2Fsmee%2Fmain%2Fsmee_notebook.livemd)\n\n## Features\n\n* Download remote SAML metadata or load local files, with effective caching\n* Manage and compare metadata files and individual entity metadata \n* MDQ API (which can also emulate MDQ style lookups with aggregate files)\n* A focus on streaming with reliable and surprisingly low memory usage\n* Filter entity streams by various criteria\n* Validate XML signatures, automatically download and confirm signing certificates\n* Transform metadata using XSLT, or extract data\n* Access XML using Erlang's Xmerl library (sweetened by SweetXML)\n* Recombine entity streams into aggregates or other data formats\n* Can be used with applications or in simple .exs scripts\n\nThe top level `Smee` module contains simplified, top level functions better suited to simpler scripts. Other modules in\nSmee contain more tools for handling SAML metadata, such as:\n\n* `Smee.Source` - define sources of metadata\n* `Smee.Metadata` - functions for handling metadata aggregates\n* `Smee.Entity` - individual SAML entity definitions\n* `Smee.Extract` - processing metadata to extract information\n* `Smee.Fetch` - downloading metadata sources\n* `Smee.MDQ` - functions for MDQ clients (and emulating MDQ clients)\n* `Smee.Filter` - filtering streams of entity records\n* `Smee.Transform` - processing and editing entity XML\n* `Smee.Publish` - Formatting and outputting metadata in various formats\n* `Smee.Stats` - Simple stats for entity streams\n* `Smee.Sigils` - An ~x sigil for XPaths optimized for use with SAML\n\n## Extensions and Extras\n\n* [SmeeFeds](https://github.com/Digital-Identity-Labs/smee_feds) - a federation management extension for use in\n  research, testing and development. SmeeFeds has a built-in list of many education and research federations.\n* [SmeeView](https://github.com/Digital-Identity-Labs/smee_view) - an experimental package for extracting information\n  from metadata XML - if it's in there, SmeeView should make it easy to extract and process.\n* SmeeOrgs (Coming Soon) - Extract, merge and enhance Organization information from metadata \n* SmeeLogos (Coming Soon) - Fetch, process and audit logo data in metadata \n\n## Examples\n\n### Getting entity details from an MDQ service\n\n```elixir\nalias Smee.MDQ\n\ncern_idp = \n  MDQ.source(\"http://mdq.ukfederation.org.uk/\")\n  |\u003e MDQ.lookup!(\"https://cern.ch/login\")\n\n```\n\n### Downloading and streaming a metadata aggregate to output a list of all IdP entityIDs\n\n```elixir\nalias Smee.{Source, Fetch, Filter, Metadata}\n\n\"http://metadata.ukfederation.org.uk/ukfederation-metadata.xml\"\n|\u003e Source.new()\n|\u003e Fetch.remote!()\n|\u003e Metadata.stream_entities()\n|\u003e Filter.idp()\n|\u003e Stream.map(fn e -\u003e e.uri end)\n|\u003e Enum.to_list()\n\n```\n\n### Create an aggregate that only contains SPs with SIRTFI, that were registered over the previous 12 months \n\n```elixir\nalias Smee.{Source, Fetch, Filter, Metadata, Publish}\n\n\"http://metadata.ukfederation.org.uk/ukfederation-metadata.xml\"\n|\u003e Source.new()\n|\u003e Fetch.remote!()\n|\u003e Metadata.stream_entities()\n|\u003e Filter.days(365)\n|\u003e Filter.sp()\n|\u003e Filter.assurance(\"https://refeds.org/sirtfi\")\n|\u003e Publish.write_aggregate()\n\n```\n### Munge XML with XSLT\n\n```elixir\nalias Smee.{Source, Fetch, Transform, Metadata}\n\n## Using a builtin function\n\nnew_xml = Source.new(\"adfs.xml\")\n|\u003e Fetch.local!()\n|\u003e Transform.decruft_sp!()\n|\u003e Metadata.xml()\n\nFile.save!(\"adfs_new.xml\", new_xml)\n\n## Or any XSLT stylesheet\n\n{:ok, updated_metadata} = Transform.transform(metadata, stylesheet, [exampleParam: \"example value\"])\n\n```\n\n## Installation\n\nIf [available in Hex](https://hex.pm/docs/publish), the package can be installed\nby adding `Smee` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:smee, \"~\u003e 0.6.0\"}\n  ]\nend\n```\n\n### Backend tools\n\n**Please note:** Smee does not do all processing itself using Elixir - it sometimes cheats (OK, it often cheats) by sending data to\nexternal programs for processing. At the moment it requires the following commandline utilities:\n\n* `xmlsec1`\n* `xmllint`\n* `xsltproc`\n\nSmee now includes a Mix task to install the default backend software - just run:\n\n`mix deps.smee`\n\n(Currently only tested on Macs with Homebrew, but it *should* also work on Debian, Ubuntu, Red Hat and Alpine Linux)\n\nA future version of Smee will support alternative sets of backends.\n\n### Rambo and Rust\n\nSmee uses an Elixir library called [Rambo](https://hex.pm/packages/rambo) to run external utilities like xsltproc. Rambo \nrelies on a small compiled shim that is provided pre-compiled for various architectures or automatically built during installation.\n\n- If your system does not match any of the supplied binary shims, you will need to have a rust compiler installed so Rambo\n  can build the shim itself. \n- Version 0.3.4 of Rambo has an additional problem: it does not ship with a precompiled binary for M1 Macs, *and it also does\n  not automatically build one*. Smee works around that with an explicit compile step in `mix.exs`, but it prevent Smee from\n  being used in Elixir .exs scripts on M1 Macs unless you copy the compiled shim from Smee into your script's build directory.\n  Hopefully this will be fixed in later versions of Rambo. \n- If you are using a modern M1/M2/M3 Mac you may need to explicitly include Rambo as a dependency in your project.\n\n## Uses\n\nMetadata Database: Smee was extracted from an application that downloads, processes and stores SAML metadata in a database for access \nwith an API. The original code had become rather messy and was very difficult to test - hopefully Smee will be an \nimprovement and will be used to replace the original code. Smee doesn't currently handle digestion and storage.\n\nMDQ Service: We've made a streaming MDQ service using Smee that should soon be available \n\nReports: We've generated reports for clients using Smee\n\n## Possible Alternatives\n\nThese aren't quite the same but are mature, widely used projects that are a lot more stable than Smee. In many cases you \nshould definitely use these instead of Smee.\n\n* [SAML Library by Tom Scavo](https://github.com/trscavo/saml-library/) A collection of Bash and XSLT scripts \n* [PyFF](https://github.com/IdentityPython/pyFF) A metadata pipeline and server using Python and YAML\n* [Shibboleth Metadata Aggregator](https://shibboleth.atlassian.net/wiki/spaces/MA1/overview) Widely used Java metadata pipeline\n\n## References\n\n### SAML Metadata\n* [Wikipedia's summary](https://en.wikipedia.org/wiki/SAML_2.0)\n\n### MDQ \n* [UK Federation Explains MDQ](https://www.ukfederation.org.uk/content/Documents/MDQ)\n\n### Unusual types of SAML metadata\n* [Oauth/OIDC](https://shibboleth.atlassian.net/wiki/spaces/SC/pages/1912406916/OAuthRPMetadataProfile)\n* [ADFS does it's own thing](https://shibboleth.atlassian.net/wiki/spaces/SC/pages/1836417024/ADFSMetadataProfile)\n* [CAS!](https://shibboleth.atlassian.net/wiki/spaces/SC/pages/1879344655/CASMetadataProfile)\n\n## Contributors and Sources\n\nVarious Github repositories have been plundered in a search for XSLT stylesheets, in particular:\n\n* [SAML Library by Tom Scavo](https://github.com/trscavo/saml-library/)\n* [SWAMID Federation](https://swamid.se/swamid-metadata.git/tree/xslt)\n* [UK Access Management Federation's metadata processing pipeline](https://github.com/ukf/ukf-meta)\n\nhttps://github.com/ukf/ukf-meta/\n\nThe use of the Apache2 license in the above projects led to the Apache2 license for Smee.\n\nOther sources and inspirations:\n\n* [The XSLT package for Elixir](https://github.com/johnhamelink/xslt)\n\nSmee temporarily includes a copy of [EasySSL](https://github.com/CaliDog/EasySSL) (MIT Licensed) that was required to\nfix an Erlang 26 compatibility issue. This will revert to a normal dependency when an update is released.\n\n## Documentation\n\nDocumentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc)\nand published on [HexDocs](https://hexdocs.pm). Once published, the docs can\nbe found at \u003chttps://hexdocs.pm/smee\u003e.\n\n## Contributing\n\nYou can request new features by creating an [issue](https://github.com/Digital-Identity-Labs/smee/issues),\nor submit a [pull request](https://github.com/Digital-Identity-Labs/smee/pulls) with your contribution.\n\nIf you are comfortable working with Python but Smee's Elixir code is unfamiliar then this blog post may help: \n[Elixir For Humans Who Know Python](https://hibox.live/elixir-for-humans-who-know-python)\n\n## Copyright and License\n\nCopyright (c) 2023, 2026 Digital Identity Ltd, UK\n\nSmee is Apache 2.0 licensed.\n\n## Disclaimer\nSmee is not endorsed by The Shibboleth Foundation or any of the NREN's that may have had their XSLT borrowed.\nThe API will definitely change considerably in the first few releases after 0.1.0 - it is not stable!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-identity-labs%2Fsmee","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigital-identity-labs%2Fsmee","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigital-identity-labs%2Fsmee/lists"}