{"id":23395765,"url":"https://github.com/todofixthis/filters","last_synced_at":"2025-06-29T05:37:59.164Z","repository":{"id":65422719,"uuid":"122112096","full_name":"todofixthis/filters","owner":"todofixthis","description":"🤔 What if we took the UNIX philosophy and applied it to input validation?","archived":false,"fork":false,"pushed_at":"2025-05-10T22:27:12.000Z","size":1050,"stargazers_count":2,"open_issues_count":3,"forks_count":3,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-06-13T03:46:59.751Z","etag":null,"topics":["data-pipelines","input-validation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/todofixthis.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-02-19T19:46:09.000Z","updated_at":"2025-05-10T22:27:16.000Z","dependencies_parsed_at":"2024-10-23T20:45:54.268Z","dependency_job_id":"b1e596b8-aff8-466d-9cde-3b0962a6e9f5","html_url":"https://github.com/todofixthis/filters","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"purl":"pkg:github/todofixthis/filters","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/todofixthis%2Ffilters","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/todofixthis%2Ffilters/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/todofixthis%2Ffilters/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/todofixthis%2Ffilters/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/todofixthis","download_url":"https://codeload.github.com/todofixthis/filters/tar.gz/refs/heads/develop","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/todofixthis%2Ffilters/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262543971,"owners_count":23326632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-pipelines","input-validation"],"created_at":"2024-12-22T07:17:24.945Z","updated_at":"2025-06-29T05:37:59.158Z","avatar_url":"https://github.com/todofixthis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":".. image:: https://github.com/todofixthis/filters/actions/workflows/build.yml/badge.svg\n   :target: https://github.com/todofixthis/filters/actions/workflows/build.yml\n.. image:: https://readthedocs.org/projects/filters/badge/?version=latest\n   :target: https://filters.readthedocs.io/\n\nFilters\n=======\nThe Filters library provides an easy and readable way to create complex\ndata validation and processing pipelines, including:\n\n- Validating complex JSON structures in API requests or config files.\n- Parsing timestamps and converting to UTC.\n- Converting Unicode strings to NFC, normalizing line endings and removing\n  unprintable characters.\n- Decoding Base64, including URL-safe variants.\n\nAnd much more!\n\nThe output from one filter can be piped into the input of another, enabling you\nto chain filters together to quickly and easily create complex data schemas and\npipelines.\n\n\nExamples\n--------\nValidate a latitude position and round to manageable precision:\n\n.. code-block:: python\n\n   (\n       f.Required |\n       f.Decimal |\n       f.Min(Decimal(-90)) |\n       f.Max(Decimal(90)) |\n       f.Round(to_nearest=\"0.000001\")\n   ).apply(\"-12.0431842\")\n\nParse an incoming value as a datetime, convert to UTC and strip tzinfo:\n\n.. code-block:: python\n\n   f.Datetime(naive=True).apply(\"2015-04-08T15:11:22-05:00\")\n\nConvert every value in an iterable (e.g., list) to unicode and strip\nleading/trailing whitespace.\nThis also applies `Unicode normalization`_, strips unprintable characters and\nnormalizes line endings automatically.\n\n.. code-block:: python\n\n   f.FilterRepeater(f.Unicode | f.Strip).apply([\n       b\"\\xe2\\x99\\xaa \",\n       b\"\\xe2\\x94\\x8f(\\xc2\\xb0.\\xc2\\xb0)\\xe2\\x94\\x9b \",\n       b\"\\xe2\\x94\\x97(\\xc2\\xb0.\\xc2\\xb0)\\xe2\\x94\\x93 \",\n       b\"\\xe2\\x99\\xaa \",\n   ])\n\nParse a JSON string and check that it has correct structure:\n\n.. code-block:: python\n\n   (\n       f.JsonDecode |\n       f.FilterMapper(\n           {\n               \"birthday\":  f.Date,\n               \"gender\":    f.CaseFold | f.Choice(choices={\"f\", \"m\", \"n\"}),\n\n               \"utcOffset\":\n                   f.Decimal |\n                   f.Min(Decimal(\"-15\")) |\n                   f.Max(Decimal(\"+15\")) |\n                   f.Round(to_nearest=\"0.25\"),\n           },\n\n           allow_extra_keys   = False,\n           allow_missing_keys = False,\n       )\n   ).apply('{\"birthday\":\"1879-03-14\", \"gender\":\"M\", \"utcOffset\":\"1\"}')\n\n\nRequirements\n------------\nFilters is known to be compatible with the following Python versions:\n\n- 3.13\n- 3.12\n\n.. note::\n   I'm only one person, so to keep from getting overwhelmed, I'm only committing\n   to supporting the 3 most recent versions of Python.\n\nInstallation\n------------\nInstall the latest stable version via pip::\n\n    pip install phx-filters\n\n\n.. important::\n   Make sure to install `phx-filters`, **not** `filters`.  I created the latter\n   at a previous job years ago, and after I left they never touched that project\n   again and stopped responding to my emails — so in the end I had to fork it 🤷\n\nExtensions\n~~~~~~~~~~\nThe following extensions are available:\n\n- `Django Filters`_: Adds filters designed to work with Django applications.\n  To install::\n\n      pip install phx-filters[django]\n\n- `ISO Filters`_: Adds filters for interpreting standard codes and identifiers.\n  To install::\n\n      pip install phx-filters[iso]\n\n.. tip::\n   To install multiple extensions, separate them with commas, e.g.::\n\n      pip install phx-filters[django,iso]\n\nMaintainers\n-----------\nTo install the distribution for local development, some additional setup is required:\n\n#. `Install uv \u003chttps://docs.astral.sh/uv/getting-started/installation/\u003e`_ (only needs\n   to be done once).\n\n#. Run the following command to install additional dependencies::\n\n      uv sync --group=dev\n\n#. Activate pre-commit hook::\n\n      uv run autohooks activate --mode=pythonpath\n\nRunning Unit Tests and Type Checker\n-----------------------------------\nRun the tests for all supported versions of Python using\n`tox \u003chttps://tox.readthedocs.io/\u003e`_::\n\n   uv run tox -p\n\n.. note::\n\n   The first time this runs, it will take awhile, as mypy needs to build up its cache.\n   Subsequent runs should be much faster.\n\nIf you just want to run unit tests in the current virtualenv (using\n`pytest \u003chttps://docs.pytest.org\u003e`_)::\n\n   uv run pytest\n\nIf you just want to run type checking in the current virtualenv (using\n`mypy \u003chttps://mypy.readthedocs.io\u003e`_)::\n\n   uv run mypyc src test\n\nDocumentation\n-------------\nTo build the documentation locally:\n\n#. Switch to the ``docs`` directory::\n\n    cd docs\n\n#. Build the documentation::\n\n    uv run make html\n\nReleases\n--------\nSteps to build releases are based on\n`Packaging Python Projects Tutorial \u003chttps://packaging.python.org/en/latest/tutorials/packaging-projects/\u003e`_.\n\n.. important::\n\n   Make sure to build releases off of the ``main`` branch, and check that all changes\n   from ``develop`` have been merged before creating the release!\n\n1. Build the Project\n~~~~~~~~~~~~~~~~~~~~\n#. Delete artefacts from previous builds, if applicable::\n\n    rm dist/*\n\n#. Run the build::\n\n    uv build\n\n#. The build artefacts will be located in the ``dist`` directory at the top level of the\n   project.\n\n2. Upload to PyPI\n~~~~~~~~~~~~~~~~~\n#. `Create a PyPI API token \u003chttps://pypi.org/manage/account/token/\u003e`_ (you only have to\n   do this once).\n#. Increment the version number in ``pyproject.toml``.\n#. Upload build artefacts to PyPI::\n\n    uv publish\n\n3. Create GitHub Release\n~~~~~~~~~~~~~~~~~~~~~~~~\n#. Create a tag and push to GitHub::\n\n      git tag \u003cversion\u003e\n      git push \u003cversion\u003e\n\n   ``\u003cversion\u003e`` must match the updated version number in ``pyproject.toml``.\n\n#. Go to the `Releases page for the repo`_.\n#. Click ``Draft a new release``.\n#. Select the tag that you created in step 1.\n#. Specify the title of the release (e.g., ``{{ cookiecutter.project_name }} v1.2.3``).\n#. Write a description for the release.  Make sure to include:\n   - Credit for code contributed by community members.\n   - Significant functionality that was added/changed/removed.\n   - Any backwards-incompatible changes and/or migration instructions.\n   - SHA256 hashes of the build artefacts.\n#. GPG-sign the description for the release (ASCII-armoured).\n#. Attach the build artefacts to the release.\n#. Click ``Publish release``.\n\n.. _Django Filters: https://pypi.python.org/pypi/phx-filters-django\n.. _ISO Filters: https://pypi.python.org/pypi/phx-filters-iso\n.. _Releases page for the repo: https://github.com/todofixthis/filters/releases\n.. _Unicode normalization: https://en.wikipedia.org/wiki/Unicode_equivalence\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftodofixthis%2Ffilters","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftodofixthis%2Ffilters","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftodofixthis%2Ffilters/lists"}