Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vapier/chrome-ext-wicked-good-unarchiver
Chrome OS Files app plugin for reading many archives/compression formats
https://github.com/vapier/chrome-ext-wicked-good-unarchiver
chrome chrome-os javascript
Last synced: 3 months ago
JSON representation
Chrome OS Files app plugin for reading many archives/compression formats
- Host: GitHub
- URL: https://github.com/vapier/chrome-ext-wicked-good-unarchiver
- Owner: vapier
- License: bsd-3-clause
- Created: 2016-03-04T04:38:35.000Z (almost 9 years ago)
- Default Branch: wgu
- Last Pushed: 2020-12-14T10:31:11.000Z (about 4 years ago)
- Last Synced: 2024-04-16T03:53:59.233Z (10 months ago)
- Topics: chrome, chrome-os, javascript
- Language: JavaScript
- Homepage: https://chrome.google.com/webstore/detail/mljpablpddhocfbnokacjggdbmafjnon
- Size: 1.01 MB
- Stars: 50
- Watchers: 12
- Forks: 26
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Wicked Good Unpacker extension
This is a fork of the bundled Chrome OS ZIP Unpacker extension.
It enables support for a wide variety of archive and compression formats.
It also supports files that have only been compressed (e.g. foo.gz).
All of this is thanks to the great [libarchive] project.
You can install it via the CWS:
https://chrome.google.com/webstore/detail/mljpablpddhocfbnokacjggdbmafjnon
## Supported Formats
Note that we support archives (compressed or uncompressed), and we support
single compressed files (that have no archiving, e.g. `foo.gz`).
Here's the list of supported archive formats:
* 7z: The [7-Zip][7z] format.
* ar: Simple [UNIX archive][ar], usually for developers.
* cab: [Microsoft's cabinet archive][cab] format.
* cpio: [Classic UNIX archive][cpio] that still shows up, but has been largely
replaced by tarballs.
* crx: Google Chrome extensions.
* deb: [Debian package archive][deb] format used by Linux distros based on
Debian (like Ubuntu).
* iso: [ISO 9660][iso] CD disk images.
Note: [UDF][udf] DVD disk images are *not* supported.
* jar: [Java ARchives][jar] used by programmers.
* lha/lzh: [LHA][lha] and [LZH][lzh] are common formats in Japan, and used by
many old school video games.
* mtree: The BSD [mtree] format for mapping a directory tree.
* pax: [Portable Archive Exchange format][pax] is a UNIX format meant to
replace cpio and tar.
* rar: [RAR][rar] archives produced by [WinRAR](https://www.rarlab.com). This
is mostly for testing, so the native CrOS support should be used by default
instead.
* rpm: [RPM Package Manager archive][rpm] used by Linux distros like RedHat,
Fedora, CentOS, SUSE, and more.
* tar: [UNIX tarballs][tar] that are common in the Linux computing world.
* warc: The [Web ARChive][warc] format for archiving web sites.
* zip: The venerable [ZIP][zip]. This is mostly for testing, so the native CrOS
support should be used by default instead.
[7z]: https://en.wikipedia.org/wiki/7z
[ar]: https://en.wikipedia.org/wiki/Ar_(Unix)
[cab]: https://en.wikipedia.org/wiki/Cabinet_(file_format)
[cpio]: https://en.wikipedia.org/wiki/Cpio
[deb]: https://en.wikipedia.org/wiki/Deb_(file_format)
[iso]: https://en.wikipedia.org/wiki/ISO_9660
[jar]: https://en.wikipedia.org/wiki/JAR_(file_format)
[lha]: https://en.wikipedia.org/wiki/LHA_(file_format)
[lzh]: https://en.wikipedia.org/wiki/LHA_(file_format)
[mtree]: https://www.freebsd.org/cgi/man.cgi?mtree(5)
[pax]: https://en.wikipedia.org/wiki/Pax_(Unix)
[rar]: https://en.wikipedia.org/wiki/RAR_(file_format)
[rpm]: https://en.wikipedia.org/wiki/RPM_Package_Manager
[tar]: https://en.wikipedia.org/wiki/Tar_(computing)
[udf]: https://en.wikipedia.org/wiki/Universal_Disk_Format
[warc]: https://en.wikipedia.org/wiki/Web_ARChive
[zip]: https://en.wikipedia.org/wiki/Zip_(file_format)
Here's the list of supported compression/encoding formats:
* bz2/bzip/bzip2: The [bzip2] compression format common in the UNIX world.
* gz/gzip: The [gzip] compression format based on [zlib]. Common
in the UNIX world, although zlib is also used in many places.
* lzma: The [lzma] compression format that has been largely replaced by xz.
The compression algorithm is used by other formats, but the standalone format
is not.
* lz4: The [LZ4][lz4] compression algorithm.
* lzip: The [lzma] compression algorithm in the [lzip] format.
* lzop: The [LZO][lzo] compression algorithm in the [lzop] format.
* uu: The [unix-to-unix][uu] text encoding format.
* xz: The [xz] compression format that is common in the UNIX world.
* Z: The [compress][Z] legacy format that still shows up at random.
* zstd: The [Zstandard][zstd] algorithm developed by Facebook.
[bzip2]: https://en.wikipedia.org/wiki/Bzip2
[gzip]: https://en.wikipedia.org/wiki/Gzip
[lz4]: https://en.wikipedia.org/wiki/LZ4_(compression_algorithm)
[lzip]: https://en.wikipedia.org/wiki/Lzip
[lzma]: https://en.wikipedia.org/wiki/LZMA
[lzo]: https://en.wikipedia.org/wiki/Lempel–Ziv–Oberhumer
[lzop]: https://en.wikipedia.org/wiki/Lzop
[uu]: https://en.wikipedia.org/wiki/Uuencoding
[xz]: https://en.wikipedia.org/wiki/Xz
[Z]: https://en.wikipedia.org/wiki/Compress
[zlib]: https://en.wikipedia.org/wiki/Zlib
[zstd]: https://en.wikipedia.org/wiki/Zstandard
## Known Issues
### Speed/Performance
Most archive formats don't include an index. This means we need to decompress
the entire file just to get a directory listing. The formats allow any ordering
by design. For example, it could be `./bar.txt`, `./foo/blah.txt`, `./asdf.txt`.
Or it could be `./asdf.txt`, `./foo/blah.txt`, and `./bar.txt`. The only way we
can produce a complete directory listing is by looking through the entire file.
This slows things down overall (like in tarballs) and there isn't much that can
be done about it.
However, there are a some file formats that do have indexes and we don't (yet)
support using those. 7-zip is the most notable one here.
A similar issue comes up with single compressed files. Many formats do not know
the uncompressed file size, so the only way to calculate it is by decompressing
the entire file. If we were to report a fake file size (like zero bytes, or a
really large file size) to the Files app, it wouldn't be able to copy the result
out. It would try to read the number of bytes that it was told were available.
For the few formats that do include the uncompressed size in their header (like
the gzip format), we can skip the decompression overhead.
### Passwords
Some formats can be encrypted with passwords, but we don't prompt the user, so
the files aren't decrypted. Oops.
### Spanning Archives
Some formats can span multiple files, but we don't yet support those.
### RAR Support
*"It's complicated."*
The WGU extension doesn't support the [RAR][rar] format today.
Chrome OS supports it natively via [cros-disks] -> [AVFS][avfs] -> official
[unrar] program.
We can't replace that stack until we have comparable coverage.
The [RAR][rar] format has gone through a number of major revisions (at least 5
so far). A smart Russian came up with it long ago and continues to develop it
as a company (RARLAB). It's a proprietary format and, while some code has been
released by them, they are hostile to reverse engineering. As such, only the
v1, v2, and v3 formats are supported. Unfortunately, v4 and v5 formats are
common and users tend to use those more.
There is an open source unrar library released by RARLAB, but the API is not
documented, and its runtime model does not mesh well with libarchive's runtime
model. It's possible, but it's not trivial.
[avfs]: http://avf.sourceforge.net/
[cros-disks]: https://chromium.googlesource.com/chromiumos/platform2/+/master/cros-disks/
[rar]: https://en.wikipedia.org/wiki/RAR_(file_format)
[unrar]: https://www.rarlab.com/rar_add.htm
## Chrome OS Bundling
Sometimes people ask, since WGU is based on the official Chrome OS Zip unpacker
that is bundled with Chrome OS today, why don't we just merge the two so that
Chrome OS supports everything WGU does out of the box?
*"It's complicated."*
From the product team's perspective, they don't want to support an extensive set
of formats if there is not high user demand for them. If users run into problems
(and they inevitably will), the engineering costs aren't justified.
Similarly, they don't want to say "ZIP is officially supported, but all other
formats are 'best effort'". Most users don't care about those trade-offs --
they just want their system to work. All they see is that they tried to open
a 7z file and it didn't work even though opening a different 7z file worked.
Trying to explain these nuances doesn't really scale.
Thus the status quo is to not support the formats at all. Users can try and
locate alternatives (like WGU), and in the process of doing so, understand that
the resulting software might be buggy. And those bugs are not the fault of the
Chrome OS product (although some will still complain that Chrome OS should have
included support out of the box).
Everyone has a reasonable position taken in isolation. But the end result is
that everyone loses. Offering best-effort support makes users unhappy, but
offering nothing also makes them unhappy. At least this way, the blow back on
the Chrome OS product is lower.
## Bug reports / Feature requests
Please use the [issues](https://github.com/vapier/chrome-ext-wicked-good-unarchiver/issues)
link here to report any issues you might run into.
# ZIP Unpacker extension
This is the ZIP Unpacker extension used in Chrome OS to support reading and
unpacking of zip archives.
## Build steps
### NaCl SDK
Since the code is built with [NaCl](https://developer.chrome.com/native-client/),
you'll need its toolchain.
```
$ cd third-party
$ make nacl_sdk
```
### Webports (a.k.a. NaCl ports)
We'll use libraries from [webports](https://chromium.googlesource.com/webports/).
```
$ cd third-party
$ make depot_tools
$ make webports
```
### npm Setup
First install [npm](https://www.npmjs.com/) using your normal packaging system.
On Debian, you'll want something like:
```
$ sudo apt-get install npm
```
Your distro might have old versions of npm, so you'd have to install it
yourself.
Then install the npm modules that we require. Do this in the root of the
unpacker repo.
```
$ npm install bower vulcanize crisper
```
### Unpacker Build
Once done, install the [libarchive-fork/] from [third-party/] of the unpacker
project. Note that you cannot use [libarchive] nor libarchive-dev packages from
webports at this moment, as not all patches in the fork are upstreamed.
```
$ cd third-party
$ make libarchive-fork
```
Polymer is used for UI. In order to fetch it, in the same directory type:
```
$ make polymer
```
Build the PNaCl module.
```
$ cd unpacker
$ make [debug]
```
## Use
The package can be found in the release or debug directory. You can run it
directly from there using Chrome's "Load unpacked extension" feature, or you
can zip it up for posting to the Chrome Web Store.
```
$ zip -r release.zip release/
```
Once it's loaded, you should be able to open ZIP archives in the Files app.
## Source Layout
Paths that aren't linked below are dynamically created at build time.
* node_modules/: All the locally installed npm modules used for building.
* [third-party/]: The source for third-party NaCl & Polymer code.
* [libarchive-fork/]: The [libarchive] NaCl module (w/custom patches).
* [polymer/]: [Polymer](https://www.polymer-project.org/) code for UI objects.
* [unpacker/]: The extension CSS/HTML/JS/NaCl source code.
* [cpp/]: The NaCl module source.
* [css/]: Any CSS needed for styling UI.
* debug/: A debug build of the Chrome extension.
* [html/]: Any HTML needed for UI.
* [icons/]: Various extension images.
* [js/]: The JavaScript code.
* [_locales/]: [Translations](https://developer.chrome.com/extensions/i18n) of strings shown to the user.
* pnacl/: Compiled NaCl objects & module (debug & release).
* release/: A release build of the Chrome extension.
* [unpacker-test/]: Code for running NaCl & JavaScript unittests.
### NaCl/JS Life Cycle
Some high level points to remember: the JS side reacts to user events and is
the only part that has access to actual data on disk. It uses the NaCl module
to do all the data parsing (e.g. gzip & tar), but it has to both send a request
to the module ("parse this archive"), and respond to requests from the module
when the module needs to read actual bytes on disk.
When the extension loads, [background.js] registers everything and goes idle.
When the Files app wants to mount an archive, callbacks in [app.js]
`unpacker.app` are called to initialize the NaCl runtime. Creates an
`unpacker.Volume` object for each mounted archive.
Requests on the archive (directory listing, metadata lookups, reading files)
are routed through [app.js] `unpacker.app` and to [volume.js] `unpacker.Volume`.
Then they are sent to the low level [decompressor.js] `unpacker.Decompressor`
which talks to the NaCl module using the [request.js] `unpacker.request`
protocol. Responses are passed back up.
When the NaCl module is loaded, [module.cc] `NaclArchiveModule` is instantiated.
That instantiates `NaclArchiveInstance` for initial JS message entry points.
It instantiates `JavaScriptMessageSender` for sending requests back to JS.
When JS requests come in, [module.cc] `NaclArchiveInstance` will create
[volume.h] `Volume` objects on the fly, and pass requests down to them (using
the protocol defined in [request.h] `request::*`).
[volume.h] `Volume` objects in turn use the [volume_archive.h] `VolumeArchive`
abstract interface to handle requests from the JS side (using the protocol
defined in [request.h] `request:**`). This way the lower levels don't have to
deal with JS directly.
[volume_archive_libarchive.cc] `VolumeArchiveLibarchive` implements the
`VolumeArchive` interface and uses [libarchive] as its backend to do all the
decompression & archive format processing.
But NaCl code doesn't have access to any files or data itself. So the
[volume_reader.h] `VolumeReader` abstract interface is passed to it to provide
the low level data read functions. The [volume_reader_javascript_stream.cc]
`VolumeReaderJavaScriptStream` implements that by passing requests back up
to the JS side via the [javascript_requestor_interface.h]
`JavaScriptRequestorInterface` interface (which was passed down to it).
So requests (mount an archive, read a file, etc...) generally follow the path:
* JavaScript -> `request::*`
* [module.cc] `NaclArchiveModule` -> `NaclArchiveModule` -> `request::*`
* [volume.cc] `Volume`
* [volume_archive.h] `VolumeArchive`
* [volume_archive_libarchive.cc] `VolumeArchiveLibarchive`
* [volume_reader.h] `VolumeReader`
* [volume_reader_javascript_stream.cc] `VolumeReaderJavaScriptStream`
* [javascript_requestor_interface.h] `JavaScriptRequestorInterface`
* [volume.cc] `JavaScriptRequestor`
* [javascript_message_sender_interface.h] `JavaScriptMessageSenderInterface`
* [module.cc] `JavaScriptMessageSender` -> `request::*`
* JavaScript (to read actual bytes of data)
Then once `VolumeArchive` has processed the raw data stream, it can return
results to the `Volume` object which takes care of posting JS status messages
back to the Chrome side.
#### Source Layout
Here's the JavaScript code that matters. A few files have very specific
purposes and can be ignored at a high level, so they're in a separate section.
* [background.js]
* Main entry point.
* Initializes the module/runtime.
* Registers the extension with Chrome filesystem/runtime.
* [app.js] `unpacker.app`
* Main runtime for the extension.
* Loads/unloads NaCl modules on demand (to save runtime memory).
* Loads/unloads volumes as Chrome has requested.
* Responds to Chrome filesystem callbacks.
* Passes data back to Chrome from `unpacker.Volume` objects.
* [volume.js] `unpacker.Volume`
* Every mounted archive has a `unpacker.Volume` instance.
* Provides high level interface to requests (like reading files & metadata).
* [decompressor.js] `unpacker.Decompressor`
* Provides low level interface for `unpacker.Volume` requests.
* Talks to the NaCl module using the `unpacker.request` protocol.
* [request.js] `unpacker.request`
* Handle the JS<->NaCl protocol communication.
* [passphrase-manager.js] `unpacker.PassphraseManager`
* Interface for plumbing password requests between UI & JS & NaCl.
These are the boilerplate/simple JavaScript files you can generally ignore.
* [build-config.js]
* Unused?
* [passphrase-dialog.js]
* Polymer code for managing the password dialog.
* [types.js] `unpacker.types`
* Basic file for setting up custom types/constants.
* [unpacker.js] `unpacker`
* Basic file for setting up `unpacker.*` namespace.
Here's the NaCl layout.
* [module.cc]
* Main entry point to the module.
* Implements the ppapi interface.
* Routes requests between the JS & NaCl layers.
* Implements the `JavaScriptMessageSenderInterface` interface so the rest of
NaCl code can easily send messages back up.
* [javascript_message_sender_interface.h]
* API for the nacl code to easily send messages back up.
* Implemented in [module.cc].
* [request.cc] [request.h]
* Defines the protocol used to communicate between JS & NaCl.
* [javascript_requestor_interface.h]
* Abstract `JavaScriptRequestorInterface` interface for talking to JS side.
* Implemented in [volume.cc].
* [volume.cc] [volume.h]
* Defines the `Volume` class that encompasses a high level volume.
* Every mount request has a Volume.
* Takes care of plumbing JS requests to lower `VolumeArchive`.
* Implements `JavaScriptRequestorInterface` interface.
* [volume_archive.h]
* Abstract `VolumeArchive` interface for handling specific archive formats.
* [volume_archive_libarchive.cc] [volume_archive_libarchive.h]
* Implements `VolumeArchive` using the libarchive project.
* [volume_reader.h]
* Abstract `VolumeReader` interface for low level reading of data.
* Used to read raw streams of data from somewhere.
* [volume_reader_javascript_stream.cc] [volume_reader_javascript_stream.h]
* Implements `VolumeReader`.
* Uses `JavaScriptRequestorInterface` to get data from the JS side.
## Debugging
To see debug messages open chrome from a terminal and check the output.
For output redirection see
https://developer.chrome.com/native-client/devguide/devcycle/debugging.
## Testing
Install [Karma](https://karma-runner.github.io/0.12/index.html) for tests
runner, [Mocha](https://mochajs.org/) for asynchronous testings,
[Chai](https://www.chaijs.com/) for assertions, and
[Sinon](https://sinonjs.org/) for spies and stubs.
```
$ npm install --save-dev \
karma karma-chrome-launcher karma-cli \
mocha karma-mocha karma-chai chai karma-sinon sinon
# Run tests:
$ cd unpacker-test
$ ./run_js_tests.sh # JavaScript tests.
$ ./run_cpp_tests.sh # C++ tests.
# Check JavaScript code using the Closure JS Compiler.
# See https://www.npmjs.com/package/closurecompiler
$ cd unpacker
$ npm install google-closure-compiler
$ bash check_js_for_errors.sh
```
[libarchive]: https://www.libarchive.org/
[third-party/]: ./third-party/
[libarchive-fork/]: ./third-party/libarchive-fork/
[polymer/]: ./third-party/polymer/
[unpacker/]: ./unpacker/
[cpp/]: ./unpacker/cpp/
[css/]: ./unpacker/css/
[html/]: ./unpacker/html/
[icons/]: ./unpacker/icons/
[js/]: ./unpacker/js/
[_locales/]: ./unpacker/_locales/
[unpacker-test/]: ./unpacker-test/
[javascript_message_sender_interface.h]: ./unpacker/cpp/javascript_message_sender_interface.h
[javascript_requestor_interface.h]: ./unpacker/cpp/javascript_requestor_interface.h
[module.cc]: ./unpacker/cpp/module.cc
[request.cc]: ./unpacker/cpp/request.cc
[request.h]: ./unpacker/cpp/request.h
[volume_archive.h]: ./unpacker/cpp/volume_archive.h
[volume_archive_libarchive.cc]: ./unpacker/cpp/volume_archive_libarchive.cc
[volume_archive_libarchive.h]: ./unpacker/cpp/volume_archive_libarchive.h
[volume.cc]: ./unpacker/cpp/volume.cc
[volume.h]: ./unpacker/cpp/volume.h
[volume_reader.h]: ./unpacker/cpp/volume_reader.h
[volume_reader_javascript_stream.cc]: ./unpacker/cpp/volume_reader_javascript_stream.cc
[volume_reader_javascript_stream.h]: ./unpacker/cpp/volume_reader_javascript_stream.h
[app.js]: ./unpacker/js/app.js
[background.js]: ./unpacker/js/background.js
[build-config.js]: ./unpacker/js/build-config.js
[decompressor.js]: ./unpacker/js/decompressor.js
[passphrase-dialog.js]: ./unpacker/js/passphrase-dialog.js
[passphrase-manager.js]: ./unpacker/js/passphrase-manager.js
[request.js]: ./unpacker/js/request.js
[types.js]: ./unpacker/js/types.js
[unpacker.js]: ./unpacker/js/unpacker.js
[volume.js]: ./unpacker/js/volume.js