{"id":28140703,"url":"https://github.com/unnoted/fileb0x","last_synced_at":"2025-05-14T18:14:03.719Z","repository":{"id":3596014,"uuid":"50256578","full_name":"UnnoTed/fileb0x","owner":"UnnoTed","description":"a better customizable tool to embed files in go; also update embedded files remotely without restarting the server","archived":false,"fork":false,"pushed_at":"2023-03-07T00:10:45.000Z","size":603,"stargazers_count":632,"open_issues_count":14,"forks_count":53,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-05-14T18:12:10.708Z","etag":null,"topics":["embed","embed-files","embedded-files","filebox","files","go","go-bindata","http","http-filesystem","update","virtual-file-system"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UnnoTed.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2016-01-23T20:19:33.000Z","updated_at":"2025-05-13T09:49:14.000Z","dependencies_parsed_at":"2022-08-09T09:30:18.831Z","dependency_job_id":null,"html_url":"https://github.com/UnnoTed/fileb0x","commit_stats":{"total_commits":121,"total_committers":17,"mean_commits":7.117647058823529,"dds":0.6446280991735538,"last_synced_commit":"822a7ad39f15f3b0bfb5d96dc7a9657209357549"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnnoTed%2Ffileb0x","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnnoTed%2Ffileb0x/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnnoTed%2Ffileb0x/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UnnoTed%2Ffileb0x/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UnnoTed","download_url":"https://codeload.github.com/UnnoTed/fileb0x/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254198424,"owners_count":22030966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embed","embed-files","embedded-files","filebox","files","go","go-bindata","http","http-filesystem","update","virtual-file-system"],"created_at":"2025-05-14T18:12:10.529Z","updated_at":"2025-05-14T18:14:03.704Z","avatar_url":"https://github.com/UnnoTed.png","language":"Go","readme":"fileb0x [![Circle CI](https://circleci.com/gh/UnnoTed/fileb0x.svg?style=svg)](https://circleci.com/gh/UnnoTed/fileb0x) [![GoDoc](https://godoc.org/github.com/UnnoTed/fileb0x?status.svg)](https://godoc.org/github.com/UnnoTed/fileb0x) [![GoReportCard](https://goreportcard.com/badge/unnoted/fileb0x)](https://goreportcard.com/report/unnoted/fileb0x)\n-------\n\n### What is fileb0x?\nA better customizable tool to embed files in go.\n\nIt is an alternative to `go-bindata` that have better features and organized configuration.\n\n###### TL;DR\na better `go-bindata`\n\n-------\n### How does it compare to `go-bindata`?\nFeature                               | fileb0x                       | go-bindata\n---------------------                 | -------------                 | ------------------\ngofmt                                 | yes (optional)                | no\ngolint                                | safe                          | unsafe\ngzip compression                      | yes                           | yes\ngzip decompression                    | yes (optional: runtime)       | yes (on read)\ngzip compression levels               | yes                           | no\nseparated prefix / base for each file | yes                           | no (all files only)\ndifferent build tags for each file    | yes                           | no\nexclude / ignore files                | yes (glob)                    | yes (regex)\nspread files                          | yes                           | no (single file only)\nunexported vars/funcs                 | yes (optional)                | no\nvirtual memory file system            | yes                           | no\nhttp file system / handler            | yes                           | no\nreplace text in files                 | yes                           | no\nglob support                          | yes                           | no (walk folders only)\nregex support                         | no                            | yes (ignore files only)\nconfig file                           | yes (config file only)        | no (cmd args only)\nupdate files remotely                 | yes                           | no\n\n-------\n### What are the benefits of using a Virtual Memory File System?\nBy using a virtual memory file system you can have access to files like when they're stored in a hard drive instead of a `map[string][]byte` you would be able to use IO writer and reader.\nThis means you can `read`, `write`, `remove`, `stat` and `rename` files also `make`, `remove` and `stat` directories.\n\n###### TL;DR \nVirtual Memory File System has similar functions as a hdd stored files would have.\n\n\n\n### Features\n\n- [x] golint safe code output\n\n- [x] optional: gzip compression (with optional run-time decompression)\n\n- [x] optional: formatted code (gofmt)\n\n- [x] optional: spread files\n\n- [x] optional: unexporTed variables, functions and types\n\n- [x] optional: include multiple files and folders\n\n- [x] optional: exclude files or/and folders\n\n- [x] optional: replace text in files\n\n- [x] optional: custom base and prefix path\n\n- [x] Virtual Memory FileSystem - [webdav](https://godoc.org/golang.org/x/net/webdav)\n\n- [x] HTTP FileSystem and Handler\n\n- [x] glob support - [doublestar](https://github.com/bmatcuk/doublestar)\n\n- [x] json / yaml / toml support\n\n- [x] optional: Update files remotely\n\n- [x] optional: Build tags for each file\n\n\n### License\nMIT\n\n\n### Get Started\n\n###### TL;DR QuickStart\u0026trade;\n\nHere's the get-you-going in 30 seconds or less:\n\n```bash\ngit clone https://github.com/UnnoTed/fileb0x.git\ncd fileb0x\ncd _example/simple\ngo generate\ngo build\n./simple\n```\n\n* `mod.go` defines the package as `example.com/foo/simple`\n* `b0x.yaml` defines the sub-package `static` from the folder `public`\n* `main.go` includes the comment `//go:generate go run github.com/UnnoTed/fileb0x b0x.yaml`\n* `main.go` also includes the import `example.com/foo/simple/static`\n* `go generate` locally installs `fileb0x` which generates `./static` according to `bax.yaml`\n* `go build` creates the binary `simple` from `package main` in the current folder\n* `./simple` runs the self-contained standalone webserver with built-in files from `public`\n\n\u003cdetails\u003e \n\n\u003csummary\u003eHow to use it?\u003c/summary\u003e\n\n##### 1. Download\n\n```bash\ngo get -u github.com/UnnoTed/fileb0x\n```\n\n##### 2. Create a config file\nFirst you need to create a config file, it can be `*.json`, `*.yaml` or `*.toml`. (`*` means any file name)\n\nNow write into the file the configuration you wish, you can use the example files as a start.\n\njson config file example [b0x.json](https://raw.githubusercontent.com/UnnoTed/fileb0x/master/_example/simple/b0x.json)\n\nyaml config file example [b0x.yaml](https://github.com/UnnoTed/fileb0x/blob/master/_example/simple/b0x.yaml)\n\ntoml config file example [b0x.toml](https://github.com/UnnoTed/fileb0x/blob/master/_example/simple/b0x.toml)\n\n##### 3. Run\nif you prefer to use it from the `cmd or terminal` edit and run the command below.\n\n```bash\nfileb0x YOUR_CONFIG_FILE.yaml\n```\n\nor if you wish to generate the embedded files through `go generate` just add and edit the line below into your `main.go`.\n```go\n//go:generate fileb0x YOUR_CONFIG_FILE.yaml\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eWhat functions and variables fileb0x let me access and what are they for?\u003c/summary\u003e\n\n#### HTTP\n```go\nvar HTTP http.FileSystem\n```\n\n##### Type \n[`http.FileSystem`](https://golang.org/pkg/net/http/#FileSystem)\n\n##### What is it?\n\nA In-Memory HTTP File System.\n\n##### What it does?\n\nServe files through a HTTP FileServer.\n\n##### How to use it?\n```go\n// http.ListenAndServe will create a server at the port 8080\n// it will take http.FileServer() as a param\n//\n// http.FileServer() will use HTTP as a file system so all your files\n// can be avialable through the port 8080\nhttp.ListenAndServe(\":8080\", http.FileServer(myEmbeddedFiles.HTTP))\n```\n\u003c/details\u003e\n\u003cdetails\u003e \n  \u003csummary\u003eHow to use it with `echo`?\u003c/summary\u003e\n\n```go\npackage main\n\nimport (\n\t\"github.com/labstack/echo\"\n\t\"github.com/labstack/echo/engine/standard\"\n\t// your embedded files import here ...\n\t\"github.com/UnnoTed/fileb0x/_example/echo/myEmbeddedFiles\"\n)\n\nfunc main() {\n\te := echo.New()\n\n\t// enable any filename to be loaded from in-memory file system\n\te.GET(\"/*\", echo.WrapHandler(myEmbeddedFiles.Handler))\n\n\t// http://localhost:1337/public/README.md\n\te.Start(\":1337\")\n}\n```\n\n##### How to serve a single file through `echo`?\n```go\npackage main\n\nimport (\n\t\"github.com/labstack/echo\"\n\n\t// your embedded files import here ...\n\t\"github.com/UnnoTed/fileb0x/_example/echo/myEmbeddedFiles\"\n)\n\nfunc main() {\n\te := echo.New()\n\n\t// read ufo.html from in-memory file system\n\thtmlb, err := myEmbeddedFiles.ReadFile(\"ufo.html\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// convert to string\n\thtml := string(htmlb)\n\n\t// serve ufo.html through \"/\"\n\te.GET(\"/\", func(c echo.Context) error {\n\n\t\t// serve as html\n\t\treturn c.HTML(http.StatusOK, html)\n\t})\n\n\te.Start(\":1337\")\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eExamples\u003c/summary\u003e\n\n[simple example](https://github.com/UnnoTed/fileb0x/tree/master/_example/simple) -\n[main.go](https://github.com/UnnoTed/fileb0x/blob/master/_example/simple/main.go)\n\n[echo example](https://github.com/UnnoTed/fileb0x/tree/master/_example/echo) -\n[main.go](https://github.com/UnnoTed/fileb0x/blob/master/_example/echo/main.go)\n\n```go\npackage main\n\nimport (\n\t\"log\"\n\t\"net/http\"\n\n  // your generaTed package\n\t\"github.com/UnnoTed/fileb0x/_example/simple/static\"\n)\n\nfunc main() {\n\tfiles, err := static.WalkDirs(\"\", false)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tlog.Println(\"ALL FILES\", files)\n\n  // here we'll read the file from the virtual file system\n\tb, err := static.ReadFile(\"public/README.md\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n  // byte to str\n  s := string(b)\n  s += \"#hello\"\n\n  // write file back into the virtual file system\n  err := static.WriteFile(\"public/README.md\", []byte(s), 0644)\n  if err != nil {\n    log.Fatal(err)\n  }\n\n\n\tlog.Println(string(b))\n\n\t// true = handler\n\t// false = file system\n\tas := false\n\n\t// try it -\u003e http://localhost:1337/public/secrets.txt\n\tif as {\n\t\t// as Handler\n\t\tpanic(http.ListenAndServe(\":1337\", static.Handler))\n\t} else {\n\t\t// as File System\n\t\tpanic(http.ListenAndServe(\":1337\", http.FileServer(static.HTTP)))\n\t}\n}\n```\n\u003c/details\u003e\n\u003cdetails\u003e \n\n\u003csummary\u003eUpdate files remotely\u003c/summary\u003e\n\nHaving to upload an entire binary just to update some files in a b0x and restart a server isn't something that i like to do...\n\n##### How it works?\nBy enabling the updater option, the next time that you generate a b0x, it will include a http server, this http server will use a http basic auth and it contains 1 endpoint `/` that accepts 2 methods: `GET, POST`.\n\nThe `GET` method responds with a list of file names and sha256 hash of each file.\nThe `POST` method is used to upload files, it creates the directory tree of a new file and then creates the file or it updates an existing file from the virtual memory file system... it responds with a `ok` string when the upload is successful.\n  \n##### How to update files remotely?\n\n1. First enable the updater option in your config file:\n```yaml\n##################\n## yaml example ##\n##################\n\n# updater allows you to update a b0x in a running server\n# without having to restart it\nupdater:\n  # disabled by default\n  enabled: false\n\n  # empty mode creates a empty b0x file with just the \n  # server and the filesystem, then you'll have to upload\n  # the files later using the cmd:\n  # fileb0x -update=http://server.com:port b0x.yaml\n  #\n  # it avoids long compile time\n  empty: false\n\n  # amount of uploads at the same time\n  workers: 3\n\n  # to get a username and password from a env variable\n  # leave username and password blank (username: \"\")\n  # then set your username and password in the env vars \n  # (no caps) -\u003e fileb0x_username and fileb0x_password\n  #\n  # when using env vars, set it before generating a b0x \n  # so it can be applied to the updater server.\n  username: \"user\" # username: \"\"\n  password: \"pass\" # password: \"\"\n  port: 8041\n```\n2. Generate a b0x with the updater option enabled, don't forget to set the username and password for authentication.\n3. When your files update, just run `fileb0x -update=http://yourServer.com:8041 b0x.toml` to update the files in the running server.\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eBuild Tags\u003c/summary\u003e\n\nTo use build tags for a b0x package just add the tags to the `tags` property in the main object of your config file\n```yaml\n# default: main\npkg: static\n\n# destination\ndest: \"./static/\"\n\n# build tags for the main b0x.go file\ntags: \"!linux\"\n```\n\nYou can also have different build tags for a list of files, you must enable the `spread` property in the main object of your config file, then at the `custom` list, choose the set of files which you want a different build tag \n```yaml\n# default: main\npkg: static\n\n# destination\ndest: \"./static/\"\n\n# build tags for the main b0x.go file\ntags: \"windows darwin\"\n\n# [spread] means it will make a file to hold all fileb0x data\n# and each file into a separaTed .go file\n#\n# example:\n# theres 2 files in the folder assets, they're: hello.json and world.txt\n# when spread is activaTed, fileb0x will make a file: \n# b0x.go or [output]'s data, assets_hello.json.go and assets_world.txt.go\n#\n#\n# type: bool\n# default: false\nspread: true\n\n# type: array of objects\ncustom:\n  # type: array of strings\n  - files: \n    - \"start_space_ship.exe\"\n\n    # build tags for this set of files\n    # it will only work if spread mode is enabled\n    tags: \"windows\"\n\n  # type: array of strings\n  - files: \n    - \"ufo.dmg\"\n\n    # build tags for this set of files\n    # it will only work if spread mode is enabled\n    tags: \"darwin\"\n```\n\nthe config above will make:\n```yaml\nab0x.go                         # // +build windows darwin\n\nb0xfile_ufo.exe.go              # // +build windows\nb0xfile_start_space_ship.bat.go # // +build darwin\n```\n\u003c/details\u003e\n\n### Functions and Variables\n\n\u003cdetails\u003e \n  \u003csummary\u003eFS (File System)\u003c/summary\u003e\n\n```go\nvar FS webdav.FileSystem\n```\n\n##### Type\n[`webdav.FileSystem`](https://godoc.org/golang.org/x/net/webdav#FileSystem)\n\n##### What is it?\n\nIn-Memory File System.\n\n##### What it does?\n\nLets you `read, write, remove, stat and rename` files and `make, remove and stat` directories...\n\n##### How to use it?\n```go\nfunc main() {\n\n\t// you have the following functions available\n\t// they all control files/dirs from/to the in-memory file system!\n\tfunc Mkdir(name string, perm os.FileMode) error\n\tfunc OpenFile(name string, flag int, perm os.FileMode) (File, error)\n\tfunc RemoveAll(name string) error\n\tfunc Rename(oldName, newName string) error\n\tfunc Stat(name string) (os.FileInfo, error)\n\t// you should remove those lines ^\n\n\t// 1. creates a directory\n\terr := myEmbeddedFiles.FS.Mkdir(myEmbeddedFiles.CTX, \"assets\", 0777)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// 2. creates a file into the directory we created before and opens it\n\t// with fileb0x you can use ReadFile and WriteFile instead of this complicaTed thing\n\tf, err := myEmbeddedFiles.FS.OpenFile(myEmbeddedFiles.CTX, \"assets/memes.txt\", os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\tdata := []byte(\"I are programmer I make computer beep boop beep beep boop\")\n\n\t// write the data into the file\n\tn, err := f.Write(data)\n\tif err == nil \u0026\u0026 n \u003c len(data) {\n\t\terr = io.ErrShortWrite\n\t}\n\n\t// close the file\n\tif err1 := f.Close(); err == nil {\n\t\tlog.Fatal(err1)\n\t}\n\n\t// 3. rename a file\n\t// can also move files\n\terr = myEmbeddedFiles.FS.Rename(myEmbeddedFiles.CTX, \"assets/memes.txt\", \"assets/programmer_memes.txt\")\n\tif err != nil {\n\t\tlog.Fatal(err)\n\t}\n\n\t// 4. checks if the file we renamed exists\n\tif _, err = myEmbeddedFiles.FS.Stat(myEmbeddedFiles.CTX, \"assets/programmer_memes.txt\"); os.IsExist(err) {\n\t\t// exists!\n\n\t\t// tries to remove the /assets/ directory\n\t\t// from the in-memory file system\n\t\terr = myEmbeddedFiles.FS.RemoveAll(myEmbeddedFiles.CTX, \"assets\")\n\t\tif err != nil {\n\t\t\tlog.Fatal(err)\n\t\t}\n\t}\n\n\t// 5. checks if the dir we removed exists\n\tif _, err = myEmbeddedFiles.FS.Stat(myEmbeddedFiles.CTX, \"public/\"); os.IsNotExist(err) {\n\t\t// doesn't exists!\n\t\tlog.Println(\"works!\")\n\t}\n}\n```\n\u003c/details\u003e\n\u003cdetails\u003e \n  \u003csummary\u003eHandler\u003c/summary\u003e\n\n```go\nvar Handler *webdav.Handler\n```\n\n##### Type\n[`webdav.Handler`](https://godoc.org/golang.org/x/net/webdav#Handler)\n\n##### What is it?\n\nA HTTP Handler implementation.\n\n##### What it does?\n\nServe your embedded files.\n\n##### How to use it?\n```go\n// ListenAndServer will create a http server at port 8080\n// and use Handler as a http handler to serve your embedded files\nhttp.ListenAndServe(\":8080\", myEmbeddedFiles.Handler)\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eReadFile\u003c/summary\u003e\n\n```go\nfunc ReadFile(filename string) ([]byte, error)\n```\n\n##### Type\n[`ioutil.ReadFile`](https://godoc.org/io/ioutil#ReadFile)\n\n##### What is it?\n\nA Helper function to read your embedded files.\n\n##### What it does?\n\nReads the specified file from the in-memory file system and return it as a byte slice.\n\n##### How to use it?\n```go\n// it works the same way that ioutil.ReadFile does.\n// but it will read the file from the in-memory file system\n// instead of the hard disk!\n//\n// the file name is passwords.txt\n// topSecretFile is a byte slice ([]byte)\ntopSecretFile, err := myEmbeddedFiles.ReadFile(\"passwords.txt\")\nif err != nil {\n\tlog.Fatal(err)\n}\n\nlog.Println(string(topSecretFile))\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eWriteFile\u003c/summary\u003e\n\n```go\nfunc WriteFile(filename string, data []byte, perm os.FileMode) error\n```\n\n##### Type\n[`ioutil.WriteFile`](https://godoc.org/io/ioutil#WriteFile)\n\n##### What is it?\n\nA Helper function to write a file into the in-memory file system.\n\n##### What it does?\n\nWrites the `data` into the specified `filename` in the in-memory file system, meaning you embedded a file!\n\n-- IMPORTANT --\nIT WON'T WRITE THE FILE INTO THE .GO GENERATED FILE, IT WILL BE TEMPORARY, WHILE YOUR APP IS RUNNING THE FILE WILL BE AVAILABLE,\nAFTER IT SHUTDOWN, IT IS GONE.\n\n##### How to use it?\n```go\n// it works the same way that ioutil.WriteFile does.\n// but it will write the file into the in-memory file system\n// instead of the hard disk!\n//\n// the file name is secret.txt\n// data should be a byte slice ([]byte)\n// 0644 is a unix file permission\n\ndata := []byte(\"jet fuel can't melt steel beams\")\nerr := myEmbeddedFiles.WriteFile(\"secret.txt\", data, 0644)\nif err != nil {\n\tlog.Fatal(err)\n}\n```\n\u003c/details\u003e\n\n\u003cdetails\u003e \n  \u003csummary\u003eWalkDirs\u003c/summary\u003e\n\n```go\nfunc WalkDirs(name string, includeDirsInList bool, files ...string) ([]string, error) {\n```\n\n##### Type\n`[]string`\n\n##### What is it?\n\nA Helper function to walk dirs from the in-memory file system.\n\n##### What it does?\n\nReturns a list of files (with option to include dirs) that are currently in the in-memory file system.\n\n##### How to use it?\n```go\nincludeDirsInTheList := false\n\n// WalkDirs returns a string slice with all file paths\nfiles, err := myEmbeddedFiles.WalkDirs(\"\", includeDirsInTheList)\nif err != nil {\n\tlog.Fatal(err)\n}\n\nlog.Println(\"List of all my files\", files)\n```\n\n\u003c/details\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funnoted%2Ffileb0x","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funnoted%2Ffileb0x","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funnoted%2Ffileb0x/lists"}