{"id":16802759,"url":"https://github.com/mathause/filefisher","last_synced_at":"2025-03-22T02:31:17.009Z","repository":{"id":268406794,"uuid":"897750113","full_name":"mathause/filefisher","owner":"mathause","description":"find and parse file and folder names","archived":false,"fork":true,"pushed_at":"2025-03-14T15:23:38.000Z","size":216,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-14T15:31:03.665Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"mpytools/filefisher","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mathause.png","metadata":{},"created_at":"2024-12-03T07:07:54.000Z","updated_at":"2024-12-18T07:29:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mathause/filefisher","commit_stats":null,"previous_names":["mathause/filefinder","mathause/filefisher"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathause%2Ffilefisher","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathause%2Ffilefisher/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathause%2Ffilefisher/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mathause%2Ffilefisher/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mathause","download_url":"https://codeload.github.com/mathause/filefisher/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244897992,"owners_count":20528329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T09:40:35.363Z","updated_at":"2025-03-22T02:31:17.004Z","avatar_url":"https://github.com/mathause.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FileFinder\n\n_Find and parse file and folder names._\n\nDefine regular folder and file patterns with the intuitive python syntax:\n\n```python\nfrom filefinder import FileFinder\n\npath_pattern = \"/root/{category}\"\nfile_pattern = \"{category}_file_{number}\"\n\nff = FileFinder(path_pattern, file_pattern)\n```\n\n## Create file and path names\n\nEverything enclosed in curly brackets is a placeholder. Thus, you can create file and\npath names like so:\n\n```python\nff.create_path_name(category=\"a\")\n\u003e\u003e\u003e /root/a/\n\nff.create_file_name(category=\"a\", number=1)\n\u003e\u003e\u003e a_file_1\n\nff.create_full_name(category=\"a\", number=1)\n\u003e\u003e\u003e /root/a/a_file_1\n```\n\n## Find files on disk\n\nHowever, the strength of filefinder is parsing file names on disk. Assuming you have the\nfollowing folder structure:\n\n```\n/root/a1/a1_file_1\n/root/a1/a1_file_2\n/root/b2/b2_file_1\n/root/b2/b2_file_2\n/root/c3/c3_file_1\n/root/c3/c3_file_2\n```\n\nYou can then look for paths:\n\n```python\nff.find_paths()\n\u003e\u003e\u003e \u003cFileContainer\u003e\n\u003e\u003e\u003e      filename category\n\u003e\u003e\u003e 0  /root/a1/*       a1\n\u003e\u003e\u003e 1  /root/b2/*       b2\n\u003e\u003e\u003e 2  /root/c3/*       c3\n```\nThe placeholders (here `{category}`) is parsed and returned. You can also look for\nfiles:\n\n```python\nff.find_files()\n\u003e\u003e\u003e \u003cFileContainer\u003e\n\u003e\u003e\u003e              filename category number\n\u003e\u003e\u003e 0  /root/a1/a1_file_1       a1      1\n\u003e\u003e\u003e 1  /root/a1/a1_file_2       a1      2\n\u003e\u003e\u003e 2  /root/b2/b2_file_1       b2      1\n\u003e\u003e\u003e 3  /root/b2/b2_file_2       b2      2\n\u003e\u003e\u003e 4  /root/c3/c3_file_1       c3      1\n\u003e\u003e\u003e 5  /root/c3/c3_file_2       c3      2\n```\n\nIt's also possible to filter for certain files:\n```python\nff.find_files(category=[\"a1\", \"b2\"], number=1)\n\u003e\u003e\u003e \u003cFileContainer\u003e\n\u003e\u003e\u003e              filename category number\n\u003e\u003e\u003e 0  /root/a1/a1_file_1       a1      1\n\u003e\u003e\u003e 2  /root/b2/b2_file_1       b2      1\n```\n\nOften we need to be sure to find _exactly one_ file or path. This can be achieved using\n\n```python\nff.find_single_file(category=\"a1\", number=1)\n\u003e\u003e\u003e \u003cFileContainer\u003e\n\u003e\u003e\u003e              filename category number\n\u003e\u003e\u003e 0  /root/a1/a1_file_1       a1      1\n```\n\nIf none or more than one file is found a `ValueError` is raised.\n\n## Format syntax\n\nYou can pass format specifiers to allow more complex formats, see\n[format-specification](https://github.com/r1chardj0n3s/parse#format-specification) for details.\nUsing format specifiers, you can parse names that are not possible otherwise.\n\n### Example\n\n```python\nfrom filefinder import FileFinder\n\npaths = [\"a1_abc\", \"ab200_abcdef\",]\n\nff = FileFinder(\"\", \"{letters:l}{num:d}_{beg:2}{end}\", test_paths=paths)\n\nfc = ff.find_files()\n\nfc\n```\n\nwhich results in the following:\n\n```python\n\u003cFileContainer\u003e\n       filename letters  num beg   end\n0        a1_abc       a    1  ab     c\n1  ab200_abcdef      ab  200  ab  cdef\n```\n\nNote that `fc.df.num` has now a data type of `int` while without the `:d` it would be an\nstring (or more precisely an object as pandas uses this dtype to represent strings).\n\n\n## Filters\n\nFilters can postprocess the found paths in `\u003cFileContainer\u003e`. Currently only a `priority_filter`\nis implemented.\n\n### Example\n\nAssuming you have data for several models with different time resolution, e.g., 1 hourly\n(`\"1h\"`), 6 hourly (`\"6h\"`), and daily (`\"1d\"`), but not all models have all time resolutions:\n\n```\n/root/a/a_1h\n/root/a/a_6h\n/root/a/a_1d\n\n/root/b/b_1h\n/root/b/b_6h\n\n/root/c/c_1h\n```\n\nYou now want to get the `\"1d\"` data if available, and then the `\"6h\"` etc.. This can be achieved with the `priority filter`. Let's first parse the file names:\n\n```python\nff = FileFinder(\"/root/{model}\", \"{model}_{time_res}\")\n\nfiles = ff.find_files()\nfiles\n```\n\nwhich yields:\n\n```\n\u003cFileContainer\u003e\n       filename model time_res\n0  /root/a/a_1d     a       1d\n1  /root/a/a_1h     a       1h\n2  /root/a/a_6h     a       6h\n3  /root/b/b_1h     b       1h\n4  /root/b/b_6h     b       6h\n5  /root/c/c_1h     c       1h\n```\n\nWe can now apply a `priority_filter` as follows:\n\n```python\nfrom filefinder.filters import priority_filter\n\nfiles = priority_filter(files, \"time_res\", [\"1d\", \"6h\", \"1h\"])\nfiles\n```\n\nResulting in the desired selection:\n\n```\n       filename model time_res\n0  /root/a/a_1d     a       1d\n1  /root/b/b_6h     b       6h\n2  /root/c/c_1h     c       1h\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathause%2Ffilefisher","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmathause%2Ffilefisher","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmathause%2Ffilefisher/lists"}