Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/bigfile/bigfile
Bigfile -- a file transfer system that supports http, rpc and ftp protocol https://bigfile.site
https://github.com/bigfile/bigfile
bigfile file-system ftp golang http rpc
Last synced: 14 days ago
JSON representation
Bigfile -- a file transfer system that supports http, rpc and ftp protocol https://bigfile.site
- Host: GitHub
- URL: https://github.com/bigfile/bigfile
- Owner: bigfile
- License: mit
- Archived: true
- Created: 2019-07-15T10:43:50.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2023-02-15T02:22:16.000Z (over 1 year ago)
- Last Synced: 2024-07-31T20:51:26.291Z (3 months ago)
- Topics: bigfile, file-system, ftp, golang, http, rpc
- Language: Go
- Homepage: https://learnku.com/docs/bigfile
- Size: 555 KB
- Stars: 250
- Watchers: 15
- Forks: 49
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
- awesome-go - bigfile - A file transfer system, support to manage files with http api, rpc call and ftp client. (File Handling / Search and Analytic Databases)
- go-awesome - Bigfile - file transfer management system (Open source library / Files/Storage)
- awesome-go-extra - bigfile - - a file transfer system that supports http, rpc and ftp protocol https://bigfile.site |229|43|2|2019-07-15T10:43:50Z|2020-02-26T01:29:46Z| (File Handling / Advanced Console UIs)
README
### Bigfile ———— a file transfer system that supports http, rpc and ftp protocol [简体中文](https://learnku.com/docs/bigfile) ∙ [English](https://bigfile.site)
----
**Bigfile** is a file transfer system, supports http, ftp and rpc protocol. Designed to provide a file management service and give developers more help. At the bottom, bigfile splits the file into small pieces of **1MB**, the same slice will only be stored once.
In fact, we built a file organization system based on the database. Here you can find familiar files and folders.
Since the rpc and http protocols are supported, those languages supported by [grpc](https://grpc.io/) and other languages can be quickly accessed.
More detailed documents can be found here
### Features
* Support HTTP(s) protocol
* Support rate limit by ip
* Support cors
* Support to avoid replay attack
* Support to validate parameter signature
* Support Http Single Range Protocol
* Support FTP(s) protocol
* Support RPC protocol
* Support to deploy by [docker](https://hub.docker.com/r/bigfile/bigfile)
* Provide document with English and Chinese
### Install Bigfile
There are kinds of ways to install Bigfile, you can find more detailed documentation here [English](https://bigfile.site/start/) [简体中文](https://learnku.com/docs/bigfile)
### Start Bigfile
1. generate certificates
> bigfile rpc:make-cert
2. start server
> bigfile multi:server
>
This will print some information as follows:
[2019/09/19 15:38:32.817] 56628 DEBUG bigfile http service listening on: https://0.0.0.0:10985
[2019/09/19 15:38:32.818] 56628 DEBUG Go FTP Server listening on 2121
[2019/09/19 15:38:32.819] 56628 DEBUG bigfile rpc service listening on: tcp://[::]:10986
### HTTP Example (Token Create)
```go
package main
import (
"fmt"
"io/ioutil"
libHttp "net/http"
"strings"
"time"
"github.com/bigfile/bigfile/databases/models"
"github.com/bigfile/bigfile/http"
)
func main() {
appUid := "42c4fcc1a620c9e97188f50b6f2ab199"
appSecret := "f8f2ae1fe4f70b788254dcc991a35558"
body := http.GetParamsSignBody(map[string]interface{}{
"appUid": appUid,
"nonce": models.RandomWithMD5(128),
"path": "/images/png",
"expiredAt": time.Now().AddDate(0, 0, 2).Unix(),
"secret": models.RandomWithMD5(44),
"availableTimes": -1,
"readOnly": false,
}, appSecret)
request, err := libHttp.NewRequest(
"POST", "https://127.0.0.1:10985/api/bigfile/token/create", strings.NewReader(body))
if err != nil {
fmt.Println(err)
return
}
request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := libHttp.DefaultClient.Do(request)
if err != nil {
fmt.Println(err)
return
}
if bodyBytes, err := ioutil.ReadAll(resp.Body); err != nil {
fmt.Println(err)
return
} else {
fmt.Println(string(bodyBytes))
}
}
```
### RPC Example (Token Create)
```go
package main
import (
"crypto/tls"
"crypto/x509"
"fmt"
"io/ioutil"
"google.golang.org/grpc"
"github.com/bigfile/bigfile/rpc"
"google.golang.org/grpc/credentials"
)
func createConnection() (*grpc.ClientConn, error) {
var (
err error
conn *grpc.ClientConn
cert tls.Certificate
certPool *x509.CertPool
rootCertBytes []byte
)
if cert, err = tls.LoadX509KeyPair("client.pem", "client.key"); err != nil {
return nil, err
}
certPool = x509.NewCertPool()
if rootCertBytes, err = ioutil.ReadFile("ca.pem"); err != nil {
return nil, err
}
if !certPool.AppendCertsFromPEM(rootCertBytes) {
return nil, err
}
if conn, err = grpc.Dial("192.168.0.103:10986", grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
Certificates: []tls.Certificate{cert},
RootCAs: certPool,
}))); err != nil {
return nil, err
}
return conn, err
}
func main() {
var (
err error
conn *grpc.ClientConn
)
if conn, err = createConnection(); err != nil {
fmt.Println(err)
return
}
defer conn.Close()
grpcClient := rpc.NewTokenCreateClient(conn)
fmt.Println(grpcClient.TokenCreate(context.TODO(), &rpc.TokenCreateRequest{
AppUid: "42c4fcc1a620c9e97188f50b6f2ab199",
AppSecret: "f8f2ae1fe4f70b788254dcc991a35558",
}))
}
```
### FTP Example
[](https://app.fossa.io/projects/git%2Bgithub.com%2Fbigfile%2Fbigfile?ref=badge_large)
[](https://snapcraft.io/bigfile)