{"id":13412788,"url":"https://github.com/bigfile/bigfile","last_synced_at":"2025-03-14T18:32:17.989Z","repository":{"id":57496522,"uuid":"196979364","full_name":"bigfile/bigfile","owner":"bigfile","description":"Bigfile -- a file transfer system that supports http, rpc and ftp protocol   https://bigfile.site  ","archived":true,"fork":false,"pushed_at":"2023-02-15T02:22:16.000Z","size":568,"stargazers_count":250,"open_issues_count":3,"forks_count":49,"subscribers_count":15,"default_branch":"master","last_synced_at":"2024-07-31T20:51:26.291Z","etag":null,"topics":["bigfile","file-system","ftp","golang","http","rpc"],"latest_commit_sha":null,"homepage":"https://learnku.com/docs/bigfile","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bigfile.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-07-15T10:43:50.000Z","updated_at":"2024-06-18T14:46:36.000Z","dependencies_parsed_at":"2023-10-20T18:24:35.710Z","dependency_job_id":null,"html_url":"https://github.com/bigfile/bigfile","commit_stats":null,"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigfile%2Fbigfile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigfile%2Fbigfile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigfile%2Fbigfile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bigfile%2Fbigfile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bigfile","download_url":"https://codeload.github.com/bigfile/bigfile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221495311,"owners_count":16832457,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigfile","file-system","ftp","golang","http","rpc"],"created_at":"2024-07-30T20:01:29.207Z","updated_at":"2024-10-26T04:31:13.211Z","avatar_url":"https://github.com/bigfile.png","language":"Go","readme":"### Bigfile ———— a file transfer system that supports http, rpc and ftp protocol [简体中文](https://learnku.com/docs/bigfile) ∙ [English](https://bigfile.site)\n----\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://travis-ci.org/bigfile/bigfile\"\u003e\u003cimg src=\"https://travis-ci.org/bigfile/bigfile.svg?branch=master\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://codecov.io/gh/bigfile/bigfile\"\u003e\u003cimg src=\"https://codecov.io/gh/bigfile/bigfile/branch/master/graph/badge.svg\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://godoc.org/github.com/bigfile/bigfile\"\u003e\u003cimg src=\"https://godoc.org/github.com/bigfile/bigfile?status.svg\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/bigfile/bigfile\"\u003e\u003cimg src=\"https://img.shields.io/badge/release-v1.0.11-blue\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://app.fossa.io/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile?ref=badge_shield\"\u003e\u003cimg src=\"https://app.fossa.io/api/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile.svg?type=shield\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://goreportcard.com/report/github.com/bigfile/bigfile\"\u003e\u003cimg src=\"https://goreportcard.com/badge/github.com/bigfile/bigfile\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://golangci.com/r/github.com/bigfile/bigfile\"\u003e\u003cimg src=\"https://golangci.com/badges/github.com/bigfile/bigfile.svg\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/avelino/awesome-go\"\u003e\u003cimg src=\"https://awesome.re/mentioned-badge.svg\"/\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://bigfile.site/bigfile.png\" /\u003e\n\u003c/p\u003e\n\n**Bigfile** is a file transfer system, supports http, ftp and rpc protocol. Designed to provide a file management service and give developers more help. At the bottom, bigfile splits the file into small pieces of **1MB**, the same slice will only be stored once.\n\nIn fact, we built a file organization system based on the database. Here you can find familiar files and folders.\n\nSince the rpc and http protocols are supported, those languages supported by [grpc](https://grpc.io/) and other languages can be quickly accessed.\n\nMore detailed documents can be found here\n\n### Features\n\n* Support HTTP(s) protocol\n\n    * Support rate limit by ip\n    * Support cors\n    * Support to avoid replay attack\n    * Support to validate parameter signature\n    * Support Http Single Range Protocol\n\n* Support FTP(s) protocol\n\n* Support RPC protocol\n\n* Support to deploy by [docker](https://hub.docker.com/r/bigfile/bigfile)\n\n* Provide document with English and Chinese\n\n\n### Install Bigfile\n\nThere are kinds of ways to install Bigfile, you can find more detailed documentation here [English](https://bigfile.site/start/) [简体中文](https://learnku.com/docs/bigfile)\n\n### Start Bigfile\n\n1. generate certificates\n\n\u003e bigfile rpc:make-cert\n\n2. start server\n\n\u003e  bigfile multi:server\n\u003e\n\nThis will print some information as follows:\n\n        [2019/09/19 15:38:32.817] 56628 DEBUG  bigfile http service listening on: https://0.0.0.0:10985\n        [2019/09/19 15:38:32.818] 56628 DEBUG   Go FTP Server listening on 2121\n        [2019/09/19 15:38:32.819] 56628 DEBUG  bigfile rpc service listening on: tcp://[::]:10986\n\n\n### HTTP Example (Token Create)\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"io/ioutil\"\n\tlibHttp \"net/http\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/bigfile/bigfile/databases/models\"\n\t\"github.com/bigfile/bigfile/http\"\n)\n\nfunc main() {\n\tappUid := \"42c4fcc1a620c9e97188f50b6f2ab199\"\n\tappSecret := \"f8f2ae1fe4f70b788254dcc991a35558\"\n\tbody := http.GetParamsSignBody(map[string]interface{}{\n\t\t\"appUid\":         appUid,\n\t\t\"nonce\":          models.RandomWithMD5(128),\n\t\t\"path\":           \"/images/png\",\n\t\t\"expiredAt\":      time.Now().AddDate(0, 0, 2).Unix(),\n\t\t\"secret\":         models.RandomWithMD5(44),\n\t\t\"availableTimes\": -1,\n\t\t\"readOnly\":       false,\n\t}, appSecret)\n\trequest, err := libHttp.NewRequest(\n\t\t\"POST\", \"https://127.0.0.1:10985/api/bigfile/token/create\", strings.NewReader(body))\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\trequest.Header.Set(\"Content-Type\", \"application/x-www-form-urlencoded\")\n\tresp, err := libHttp.DefaultClient.Do(request)\n\tif err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\tif bodyBytes, err := ioutil.ReadAll(resp.Body); err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t} else {\n\t\tfmt.Println(string(bodyBytes))\n\t}\n}\n```\n\n### RPC Example (Token Create)\n\n```go\npackage main\n\nimport (\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\n\t\"google.golang.org/grpc\"\n\t\"github.com/bigfile/bigfile/rpc\"\n\t\"google.golang.org/grpc/credentials\"\n)\n\nfunc createConnection() (*grpc.ClientConn, error) {\n\tvar (\n\t\terr           error\n\t\tconn          *grpc.ClientConn\n\t\tcert          tls.Certificate\n\t\tcertPool      *x509.CertPool\n\t\trootCertBytes []byte\n\t)\n\tif cert, err = tls.LoadX509KeyPair(\"client.pem\", \"client.key\"); err != nil {\n\t\treturn nil, err\n\t}\n\n\tcertPool = x509.NewCertPool()\n\tif rootCertBytes, err = ioutil.ReadFile(\"ca.pem\"); err != nil {\n\t\treturn nil, err\n\t}\n\n\tif !certPool.AppendCertsFromPEM(rootCertBytes) {\n\t\treturn nil, err\n\t}\n\n\tif conn, err = grpc.Dial(\"192.168.0.103:10986\", grpc.WithTransportCredentials(credentials.NewTLS(\u0026tls.Config{\n\t\tCertificates: []tls.Certificate{cert},\n\t\tRootCAs:      certPool,\n\t}))); err != nil {\n\t\treturn nil, err\n\t}\n\treturn conn, err\n}\n\nfunc main() {\n\tvar (\n\t\terr  error\n\t\tconn *grpc.ClientConn\n\t)\n\n\tif conn, err = createConnection(); err != nil {\n\t\tfmt.Println(err)\n\t\treturn\n\t}\n\tdefer conn.Close()\n\tgrpcClient := rpc.NewTokenCreateClient(conn)\n    \tfmt.Println(grpcClient.TokenCreate(context.TODO(), \u0026rpc.TokenCreateRequest{\n    \t\tAppUid:    \"42c4fcc1a620c9e97188f50b6f2ab199\",\n    \t\tAppSecret: \"f8f2ae1fe4f70b788254dcc991a35558\",\n    }))\n}\n```\n\n### FTP Example\n\n![ftp-example](https://bigfile.site/ftp-connect-succesfully.png)\n\n[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile?ref=badge_large)\n[![Get it from the Snap Store](https://snapcraft.io/static/images/badges/en/snap-store-white.svg)](https://snapcraft.io/bigfile)\n","funding_links":[],"categories":["File Handling","开源类库","Files","Open source library","文件处理`处理文件和文件系统操作的库`","文件处理","Relational Databases"],"sub_categories":["Search and Analytic Databases","文件/存储","Advanced Console UIs","Files/Storage","SQL 查询语句构建库","检索及分析资料库"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigfile%2Fbigfile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbigfile%2Fbigfile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbigfile%2Fbigfile/lists"}