{"id":20492490,"url":"https://github.com/mumez/sqnappy","last_synced_at":"2025-04-13T17:01:58.991Z","repository":{"id":7863524,"uuid":"9235791","full_name":"mumez/sqnappy","owner":"mumez","description":"Squeak/Pharo binding for Snappy compression library","archived":false,"fork":false,"pushed_at":"2013-12-24T02:27:58.000Z","size":3288,"stargazers_count":8,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2023-03-24T14:32:47.511Z","etag":null,"topics":["compressed","pharo-binding","smalltalk","snappy","stream-api"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mumez.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-04-05T07:15:49.000Z","updated_at":"2021-08-28T00:29:51.000Z","dependencies_parsed_at":"2022-07-11T01:32:00.089Z","dependency_job_id":null,"html_url":"https://github.com/mumez/sqnappy","commit_stats":null,"previous_names":[],"tags_count":4,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mumez%2Fsqnappy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mumez%2Fsqnappy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mumez%2Fsqnappy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mumez%2Fsqnappy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mumez","download_url":"https://codeload.github.com/mumez/sqnappy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224819443,"owners_count":17375269,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compressed","pharo-binding","smalltalk","snappy","stream-api"],"created_at":"2024-11-15T17:29:22.312Z","updated_at":"2024-11-15T17:29:22.979Z","avatar_url":"https://github.com/mumez.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"Sqnappy\r\n=======\r\n\r\nSqueak/Pharo binding for [Snappy](https://code.google.com/p/snappy/ \"Snappy\") compression library.\r\nSqnappy is using a VM plugin for calling Snappy C API. (It is not a pure Smalltalk implementation).\r\n\r\nYou can see these directories:\r\n\r\n- plugin-build\r\n + plugin build settings\r\n- plugin-binary\r\n + pre-built shared libraries (SnappyPlugin.dll, bundle, so, etc)\r\n- repository\r\n + [Cypress](https://github.com/CampSmalltalk/Cypress) style Smalltalk source tree\r\n\r\nFor MCZ packages, visit \r\n\u003ca href=\"http://smalltalkhub.com/#!/~MasashiUmezawa/Sqnappy\"\u003eSmalltalkHub Sqnappy site\u003c/a\u003e.\r\n\r\n## Performance ##\r\nJust a simple code snippet. But it would be helpful for showing Sqnappy's characteristics.\r\n\r\n```Smalltalk\r\n\"## Test data ##\"\r\noriginalBytes := Morph allSelectors asString asByteArray.\r\noriginalBytes size. \"30363 bytes on my image\"\r\n\r\n\"## Compressed size ##\"\r\n\" GZipWriteStream \"\r\noriginalBytes asString zipped size. \"11036\"\r\n\" Sqnappy\"\r\n(SnappyCore compress: originalBytes) size.  \"17461 -- 1.6x bigger\"\r\n\r\n\"## Round-trip time ##\"\r\n\" GZipWriteStream/GZipReadStream \"\r\n[100 timesRepeat: [originalBytes asString zipped unzipped asByteArray]] timeToRun.  \"487\"\r\n\" Sqnappy \"\r\n[100 timesRepeat: [SnappyCore uncompress: (SnappyCore compress: originalBytes)]] timeToRun \"45 -- 10.8x faster\"\r\n```\r\n\r\n## Features ##\r\n- Basic compress/uncompress API\r\n- Stream API with [framing format](http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt) support (interoperability was checked with [snzip](https://github.com/kubo/snzip))\r\n- Custom stream format (.sqn - sqnappy)\r\n\t- Block size is changeable from 32k to 4M (default is 64k)\r\n\t- No CRC32C checking for speed\r\n\t- Data size is represented in big-endian\r\n\r\n## Usage ##\r\nBasic:\r\n```Smalltalk\r\n\"## Compress/Uncompress ##\"\r\ncompressed := SnappyCore compress: data.\r\nuncompressed := SnappyCore uncompress: compressed.\r\n```\r\n\r\nStreaming:\r\n```Smalltalk\r\n\"## Streaming (write) ##\"\r\nwstrm := SnappyFraming sz writeStreamOn: ByteArray new.\r\nwstrm nextPutAll: #[49 50 51 52 53 10 49 50 51 52 53 10].\r\nwstrm nextPutAll: #[49 50 51 52 53 10 123 49 50 51 52 53 10 0].\r\ncompressed := wstrm contents.\r\nwstrm close.\r\n```\r\n```Smalltalk\r\n\"## Streaming (read) ##\"\r\nrstrm := SnappyFraming sz readStreamOn: compressed.\r\nuncompressed := rstrm contents.\r\nrstrm close.\r\n```\r\n```Smalltalk\r\n\"## Streaming (partial read) ##\"\r\nrstrm := SnappyFraming sz readStreamOn: compressed.\r\n[rstrm atEnd] whileFalse: [\r\n  uncompressedPart := rstrm upTo: 10.\r\n  Transcript cr; show: uncompressedPart asString\r\n].\r\nrstrm close.\r\n```\r\n\r\nStreaming with files: (This snippet is using [FileMan](https://github.com/mumez/FileMan) for simplifying file access).\r\n```Smalltalk\r\n\"## Compress 'SqueakV41.sources -\u003e 'SqueakV41.sources.sqn' with Sqnappy format ##\"\r\nreadStr := ('.' asDirectoryEntry / 'SqueakV41.sources') readStream binary.\r\nwriteEnt := '.' asDirectoryEntry / 'SqueakV41.sources.sqn'.\r\nwriter := SnappyFraming sqn writeStreamOn: writeEnt writeStream.\r\nwriter repeatWrite:[:w | w nextPutAll:(readStr next: w blockSize)] until:[readStr atEnd] finally:[:w | w close. readStr close].\r\n\r\n\"## Uncompress 'SqueakV41.sources.sqn -\u003e 'SqueakV41.sources' ##\"\r\nreadEnt := '.' asDirectoryEntry / 'SqueakV41.sources.sqn'.\r\nreader := SnappyFraming sqn readStreamOn: readEnt readStream.\r\nwriteStr := ('.' asDirectoryEntry / 'SqueakV41-trip.sources') writeStream.\r\nreader repeatReadUntilEnd:[:r | ] out: writeStr finally:[:r | r close. writeStr close].\r\n```\r\n\r\nOn my windows laptop (Core-i5 2430M, Samsong SSD 840), Compressed file('SqueakV41.sources.sqn') was written in about **250 msecs**. Uncompressed file('SqueakV41-trip.sources') was written in about **100 msecs**.\r\nOriginal file size was **26,005,504 bytes**. Compressed file size was **10,612,736 bytes**.\r\n\r\nStream API enables you to handle pretty big data. I've also tested the code above with a big pg_dump file ( **1.3G** - 1,304,485,888 bytes). There were no annoying GCs. Compressed size was 631,627,776 bytes. **9844 msecs** for compress; **6496 msecs** for uncompress.\r\n\r\n## Installation ##\r\n1. Copy the [pre-built](https://github.com/mumez/sqnappy/blob/master/plugin-binary/) Snappy.dll (.so, .bundle) to your VM directory\r\n2. Load Sqnappy\r\n\r\n```Smalltalk\r\n\"Squeak\"\r\nInstaller squeaksource\r\n    project: 'MetacelloRepository';\r\n    install: 'ConfigurationOfSqnappy'. \r\n(Smalltalk at: #ConfigurationOfSqnappy) load.\r\n\r\n\"Pharo\"\r\nGofer new\r\n      url: 'http://smalltalkhub.com/mc/MasashiUmezawa/Sqnappy/main';\r\n      package: 'ConfigurationOfSqnappy';\r\n      load.\r\n(Smalltalk at: #ConfigurationOfSqnappy) load.\r\n```\r\n\r\nOpen TestRunner and see the results. If tests are red, please make sure step.1 was done properly.\r\n \r\nNote: Plugin binaries were built for 32 bit Cog VM. For other VM (legacy VM, 64 bit VM, etc), probably you need to recompile plugin source by yourself.\r\n\r\n## License ##\r\n[MIT license](http://opensource.org/licenses/MIT)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmumez%2Fsqnappy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmumez%2Fsqnappy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmumez%2Fsqnappy/lists"}