{"id":13775266,"url":"https://github.com/opencomputeproject/Project-Zipline","last_synced_at":"2025-05-11T07:32:29.896Z","repository":{"id":52970330,"uuid":"174593531","full_name":"opencomputeproject/Project-Zipline","owner":"opencomputeproject","description":"Defines a lossless compressed data format that is independent of CPU type, operating system, file system, and character set, and is suitable for compression using the XP10 algorithm.","archived":false,"fork":false,"pushed_at":"2023-04-11T15:22:03.000Z","size":35992,"stargazers_count":285,"open_issues_count":9,"forks_count":47,"subscribers_count":35,"default_branch":"master","last_synced_at":"2025-02-20T11:44:46.108Z","etag":null,"topics":["huffman-compression-algorithm"],"latest_commit_sha":null,"homepage":"","language":"Verilog","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencomputeproject.png","metadata":{"files":{"readme":"README.md","changelog":"HISTORY.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-03-08T19:05:02.000Z","updated_at":"2025-02-09T13:24:55.000Z","dependencies_parsed_at":"2024-01-07T01:29:36.481Z","dependency_job_id":null,"html_url":"https://github.com/opencomputeproject/Project-Zipline","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencomputeproject%2FProject-Zipline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencomputeproject%2FProject-Zipline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencomputeproject%2FProject-Zipline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencomputeproject%2FProject-Zipline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencomputeproject","download_url":"https://codeload.github.com/opencomputeproject/Project-Zipline/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253533986,"owners_count":21923515,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["huffman-compression-algorithm"],"created_at":"2024-08-03T17:01:36.127Z","updated_at":"2025-05-11T07:32:24.881Z","avatar_url":"https://github.com/opencomputeproject.png","language":"Verilog","funding_links":[],"categories":["others","Accelerators"],"sub_categories":[],"readme":"## Project Zipline\r\n\r\n## Introduction\r\n\r\nProject Zipline is a program to accelerate innovation in lossless compression.  With this release Microsoft is making available:\r\n\r\n* A new compression format called XP10 which is tailored for modern cloud datasets.\r\n* All specifications for the entire pipeline.\r\n* All RTL to support implementation for the entire pipeline.\r\n* A testbench environment specifically developed for the VCS simulator.\r\n\r\nProject Zipline is open-sourced under the MIT License, see the LICENSE file.\r\n\r\n## Specifications\r\nThe following Project Zipline specifications have been included under the “specs” directory in this repository:\r\n\r\n* Project_Zipline_Compression_Specification\r\n* Project_Zipline_Huffman_Encoder_Micro_Architecture_Specification\r\n* Project_Zipline_Comp_lz77_Micro_Architecture_Specification\r\n* Project_Zipline_Crypto_Engine_Micro_Architecture_Specification\r\n* Project_Zipline_Decompression_Top_Micro_Architecture_Specification\r\n* Project_Zipline_Keyblob_Micro_Architecture_Specification\r\n* Project_Zipline_KME_Micro_Architecture_Specification\r\n* Project_Zipline_Prefix_Attach_Micro_Architecture_Specification\r\n* Project_Zipline_Prefix_Micro_Architecture_Specification\r\n* Project_Zipline_SSB_Micro_Architecture_Specification\r\n* Project_Zipline_Top_Micro_Architecture_Specification\r\n\r\n## Simulation Notes\r\n\r\nEdit the file **`./zipline.setup`** for VCS, SYNTH, and VERDI environment settings.\r\n\r\nThen source the file as follows:\r\n\r\n**`source zipline.setup`**\r\n\r\nGo to either the CCE_64, CDD_64, or KME run directory:\r\n\r\n**`cd dv/CCE_64/run`**\r\n\r\n**`cd dv/CDD_64/run`**\r\n\r\n**`cd dv/KME/run`**\r\n\r\nTo build the simv executable:\r\n\r\n**`make build_simv`**\r\n\r\nExample for simulating a test:\r\n\r\n**`make run_simv TESTNAME=xp10`**\r\n\r\nExample for simulating a test with waves (.vpd):\r\n\r\n**`make run_simv TESTNAME=xp10 WAVES=1`**\r\n\r\nExample for simulating a test with Verdi waves (.fsdb):\r\n\r\n**`make run_simv TESTNAME=xp10 VERDI_WAVES=1`**\r\n\r\nAll of the tests for the CCE_64, CDD_64, and KME can be run via a regress\r\nscript in the associated \"run\" directory:\r\n\r\n**`dv/CCE_64/run/regress`**\r\n\r\n**`dv/CDD_64/run/regress`**\r\n\r\n**`dv/KME/run/regress`**\r\n\r\nAll test files are located in the associated engine directory \"tests\".\r\nEach directory also contains a README file with a description of each test.\r\n\r\n**`dv/CCE_64/tests`**\r\n\r\n**`dv/CDD_64/tests`**\r\n\r\n**`dv/KME/tests`**\r\n\r\nA description of the programmable registers in the CCE/CDD/KME can be found\r\nin the register_doc directory. Please read register_doc/README for more details.\r\n\r\n## KME Notes\r\n\r\nThe KME RTL (rtl/cr_kme) has been modified to remove the following modules:\r\n* AES engine in the random GUID generator\r\n* SHA engines within the KDF function\r\n* AES engine within the Key Decryption logic\r\n\r\nWith these reductions, the RTL only supports Key Types 1-6 without KDF and \r\nKey Type 0 without encryption/authentication\r\n \r\nAdditionally, the KME output has been reduced to support a single engine.\r\n \r\nAll modifications are identified with \"KME_MODIFICATION_NOTE\" comments\r\nin the RTL code.\r\n\r\n## Synthesis Notes\r\n\r\nNote: For synthesis, please edit syn/Makefile to choose an LSF or dedicated\r\nmachine with at least 16G of memory.  Edit the contents of this line,\r\nbefore \"dc_shell\":\r\n\r\n**`bsub -Is -q irv-cpx-M16 -R \"rusage [mem=16000]\" dc_shell -f syn_example.tcl | tee ./LOGS/syn_eample.tcl.`date '+%m%d.%H:%M'`.log`**\r\n\r\nTo run synthesis for CDD\r\n\r\n**`cd syn/cr_cddip`**\r\n\r\n**`make syn`**\r\n\r\nTo run synthesis for CCE\r\n\r\n**`cd syn/cr_cceip_64`**\r\n\r\n**`make syn`**\r\n\r\nTo run synthesis for KME\r\n\r\n**`cd syn/cr_kme`**\r\n\r\n**`make syn`**\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencomputeproject%2FProject-Zipline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencomputeproject%2FProject-Zipline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencomputeproject%2FProject-Zipline/lists"}