{"id":20182322,"url":"https://github.com/ywangd/pybufrkit","last_synced_at":"2025-04-07T10:19:24.021Z","repository":{"id":59961904,"uuid":"66844307","full_name":"ywangd/pybufrkit","owner":"ywangd","description":"Pure Python toolkit to work with WMO BUFR messages","archived":false,"fork":false,"pushed_at":"2025-03-28T11:07:04.000Z","size":4628,"stargazers_count":78,"open_issues_count":4,"forks_count":29,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-03-31T09:03:12.732Z","etag":null,"topics":["bufr","fm-94","pure","python","wmo"],"latest_commit_sha":null,"homepage":"http://pybufrkit.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ywangd.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-08-29T13:05:24.000Z","updated_at":"2025-03-28T11:03:46.000Z","dependencies_parsed_at":"2023-10-15T11:42:16.777Z","dependency_job_id":"e48267b4-3911-445e-8e3e-64f79293f3eb","html_url":"https://github.com/ywangd/pybufrkit","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ywangd%2Fpybufrkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ywangd%2Fpybufrkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ywangd%2Fpybufrkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ywangd%2Fpybufrkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ywangd","download_url":"https://codeload.github.com/ywangd/pybufrkit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247631834,"owners_count":20970069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bufr","fm-94","pure","python","wmo"],"created_at":"2024-11-14T02:38:14.667Z","updated_at":"2025-04-07T10:19:23.994Z","avatar_url":"https://github.com/ywangd.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Python Toolkit for WMO BUFR Messages\n====================================\n\n.. image:: https://github.com/ywangd/pybufrkit/actions/workflows/ci.yml/badge.svg\n    :target: https://github.com/ywangd/pybufrkit\n\n`PyBufrKit \u003chttps://github.com/ywangd/pybufrkit\u003e`_ is a **pure** Python package\nto work with WMO BUFR (FM-94) messages. It can be used as both a\ncommand line tool or library to decode and encode BUFR messages. Here is a brief\nlist of some of the features:\n\n* Pure Python\n* Handles both compressed and un-compressed messages\n* Handles all practical operator descriptors, including data quality info,\n  stats, bitmaps, etc.\n* Option to construct hierarchial structure of a message, e.g. associate\n  first order stats data to their owners.\n* Convenient subsetting support for BUFR messages\n* Comprehensive query support for BUFR messages\n* Script support enables flexible extensions, e.g. filtering through large number of files.\n* Tested with the same set of BUFR files used by\n  `ecCodes \u003chttps://software.ecmwf.int/wiki/display/ECC/ecCodes+Home\u003e`_\n  and `BUFRDC \u003chttps://software.ecmwf.int/wiki/display/BUFR/BUFRDC+Home\u003e`_.\n\nMore documentation at http://pybufrkit.readthedocs.io/\n\nAn `online BUFR decoder \u003chttp://aws-bufr-webapp.s3-website-ap-southeast-2.amazonaws.com/\u003e`_ powered by PyBufrKit, \n`Serverless \u003chttps://serverless.com/\u003e`_ and `AWS Lambda \u003chttps://aws.amazon.com/lambda/\u003e`_.\n\nInstallation\n------------\nPyBufrKit is compatible with Python 2.7, 3.5+, and `PyPy \u003chttps://pypy.org/\u003e`_.\nTo install from PyPi::\n\n    pip install pybufrkit\n\nOr from `conda-forge \u003chttps://conda-forge.org\u003e`_::\n\n    conda install -c conda-forge pybufrkit\n\nOr from source::\n\n    python setup.py install\n\nCommand Line Usage\n------------------\n\nThe command line usage of the toolkit takes the following form::\n\n    pybufrkit [OPTIONS] command ...\n\nwhere the ``command`` is one of following actions that can be performed by the tool:\n\n* ``decode`` - Decode a BUFR file to outputs of various format, e.g. JSON\n* ``encode`` - Encode a BUFR file from a JSON input\n* ``info`` - Decode only the metadata sections (i.e. section 0, 1, 2, 3) of given BUFR files\n* ``split`` - Split given BUFR files into one message per file\n* ``subset`` - Subset the given BUFR file and save as new file\n* ``query`` - Query metadata or data of given BUFR files\n* ``script`` - Embed BUFR query expressions into normal Python script\n* ``lookup`` - Look up information about the given list of comma separated BUFR descriptors\n* ``compile`` - Compile the given comma separated BUFR descriptors\n\nHere are a few examples using the tool from command line. For more details, please refer\nto the help option, e.g. ``pybufrkit decode -h``. Also checkout the\n`documentation \u003chttp://pybufrkit.readthedocs.io/\u003e`_.\n\n.. code-block:: Bash\n\n    # Decode a BUFR file and output in the default flat text format\n    pybufrkit decode BUFR_FILE\n\n    # Decode a file that is a concatenation of multiple BUFR messages,\n    # skipping any erroneous messages and continue on next one\n    pybufrkit decode -m --continue-on-error FILE\n\n    # Filter through a multi-message file and only decode messages\n    # that have data_category equals to 2. See below for details\n    # about usable filter expressions.\n    pybufrkit decode -m --filter '${%data_category} == 2' FILE\n\n    # Decode a BUFR file and display it in a hierarchical structure\n    # corresponding to the BUFR Descriptors. In addition, the attribute\n    # descriptors are associated to their (bitmap) corresponding descriptors.\n    pybufrkit decode -a BUFR_FILE\n\n    # Decode a BUFR file and output in the flat JSON format\n    pybufrkit decode -j BUFR_FILE\n\n    # Encode from a flat JSON file to BUFR\n    pybufrkit encode -j JSON_FILE BUFR_FILE\n\n    # Decode a BUFR file, pipe it to the encoder to encode it back to BUFR\n    pybufrkit decode BUFR_FILE | pybufrkit encode -\n\n    # Decode only the metadata sections of a BUFR file\n    pybufrkit info BUFR_FILE\n\n    # Split a BUFR file into one message per file\n    pybufrkit split BUFR_FILE\n\n    # Subset from a given BUFR file\n    pybufrkit subset 0,3,6,9 BUFR_FILE\n\n    # Query values from the metadata sections (section 0, 1, 2, 3):\n    pybufrkit query %n_subsets BUFR_FILE\n\n    # Query all values for descriptor 001002 of the data section\n    pybufrkit query 001002 BUFR_FILE\n\n    # Query for those root level 001002 of the BUFR Template\n    pybufrkit query /001002 BUFR_FILE\n\n    # Query for 001002 that is a direct child of 301001\n    pybufrkit query /301001/001002 BUFR_FILE\n\n    # Query for all 001002 of the first subset\n    pybufrkit query '@[0] \u003e 001002' BUFR_FILE\n\n    # Query for associated field of 021062\n    pybufrkit query 021062.A21062 BUFR_FILE\n\n    # Filtering through a number of BUFR files with Script support\n    # (find files that have multiple subsets):\n    pybufrkit script 'if ${%n_subsets} \u003e 1: print(PBK_FILENAME)' DIRECTORY/*.bufr\n\n    # Lookup information for a Element Descriptor (along with its code table)\n    pybufrkit lookup -l 020003\n\n    # Compile a BUFR Template composed as a comma separated list of descriptors\n    pybufrkit compile 309052,205060\n\n\nLibrary Usage\n-------------\n\nThe following code shows an example of basic library usage\n\n.. code-block:: Python\n\n    # Decode a BUFR file\n    from pybufrkit.decoder import Decoder\n    decoder = Decoder()\n    with open(SOME_BUFR_FILE, 'rb') as ins:\n        bufr_message = decoder.process(ins.read())\n\n    # Convert the BUFR message to JSON\n    from pybufrkit.renderer import FlatJsonRenderer\n    json_data = FlatJsonRenderer().render(bufr_message)\n\n    # Encode the JSON back to BUFR file\n    from pybufrkit.encoder import Encoder\n    encoder = Encoder()\n    bufr_message_new = encoder.process(json_data)\n    with open(BUFR_OUTPUT_FILE, 'wb') as outs:\n        outs.write(bufr_message_new.serialized_bytes)\n\n    # Decode for multiple messages from a single file\n    from pybufrkit.decoder import generate_bufr_message\n    with open(SOME_FILE, 'rb') as ins:\n        for bufr_message in generate_bufr_message(decoder, ins.read()):\n            pass  # do something with the decoded message object\n\n    # Query the metadata\n    from pybufrkit.mdquery import MetadataExprParser, MetadataQuerent\n    n_subsets = MetadataQuerent(MetadataExprParser()).query(bufr_message, '%n_subsets')\n\n    # Query the data\n    from pybufrkit.dataquery import NodePathParser, DataQuerent\n    query_result = DataQuerent(NodePathParser()).query(bufr_message, '001002')\n\n    # Script\n    from pybufrkit.script import ScriptRunner\n    # NOTE: must use the function version of print (Python 3), NOT the statement version\n    code = \"\"\"print('Multiple' if ${%n_subsets} \u003e 1 else 'Single')\"\"\"\n    runner = ScriptRunner(code)\n    runner.run(bufr_message)\n\n**For more help, please check the documentation site at** http://pybufrkit.readthedocs.io/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fywangd%2Fpybufrkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fywangd%2Fpybufrkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fywangd%2Fpybufrkit/lists"}