Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/johannesboyne/gofakes3
A simple fake AWS S3 object storage (used for local test-runs against AWS S3 APIs)
https://github.com/johannesboyne/gofakes3
Last synced: 4 days ago
JSON representation
A simple fake AWS S3 object storage (used for local test-runs against AWS S3 APIs)
- Host: GitHub
- URL: https://github.com/johannesboyne/gofakes3
- Owner: johannesboyne
- License: mit
- Created: 2016-02-24T09:13:09.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2024-09-14T15:02:48.000Z (3 months ago)
- Last Synced: 2024-09-15T00:53:30.276Z (3 months ago)
- Language: Go
- Homepage:
- Size: 633 KB
- Stars: 358
- Watchers: 8
- Forks: 81
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- stars - johannesboyne/gofakes3 - runs against AWS S3 APIs) (HarmonyOS / Windows Manager)
README
[](https://circleci.com/gh/johannesboyne/gofakes3)
[](https://codecov.io/gh/johannesboyne/gofakes3)
# AWS (GOFAKE)S3
AWS S3 fake server and testing library for comprehensive S3 integration testing.
This tool can be used to run a test server, for example, to support testing AWS Lambda functions that interact with S3. It also serves as a straightforward and convenient S3 mock and test server for various development needs.
## Intended Use
**GOFAKE)S3** is primarily designed for:
- Local development of S3-dependent AWS Lambda functions.
- Testing implementations with AWS S3 access.
- Facilitating browser-based direct uploads to S3 in a local testing environment.
## When Not to Use (GOFAKE)S3?
**(GOFAKE)S3** should not be used as a production service. Its primary purpose is to aid in development and testing:
- **(GOFAKE)S3** is not designed for production-level data storage or handling.
- It lacks the robustness required for safe, persistent access to production data.
- The tool is still under development with significant portions of the AWS S3 API yet to be implemented. Consequently, breaking changes are expected.
For production environments, consider more established solutions. Some recommended alternatives can be found in the "Similar Notable Projects" section below.
## How to use it?
### Example (aws-sdk-go version 1)
```golang
// fake s3
backend := s3mem.New()
faker := gofakes3.New(backend)
ts := httptest.NewServer(faker.Server())
defer ts.Close()
// configure S3 client
s3Config := &aws.Config{
Credentials: credentials.NewStaticCredentials("YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY", ""),
Endpoint: aws.String(ts.URL),
Region: aws.String("eu-central-1"),
DisableSSL: aws.Bool(true),
S3ForcePathStyle: aws.Bool(true),
}
newSession := session.New(s3Config)
s3Client := s3.New(newSession)
cparams := &s3.CreateBucketInput{
Bucket: aws.String("newbucket"),
}
// Create a new bucket using the CreateBucket call.
_, err := s3Client.CreateBucket(cparams)
if err != nil {
// Message from an error.
fmt.Println(err.Error())
return
}
// Upload a new object "testobject" with the string "Hello World!" to our "newbucket".
_, err = s3Client.PutObject(&s3.PutObjectInput{
Body: strings.NewReader(`{"configuration": {"main_color": "#333"}, "screens": []}`),
Bucket: aws.String("newbucket"),
Key: aws.String("test.txt"),
})
// ... accessing of test.txt through any S3 client would now be possible
```
### Example for V2 (aws-sdk-go-v2)
```golang
backend := s3mem.New()
faker := gofakes3.New(s3Backend, gofakes3.WithHostBucket(true))
ts := httptest.NewServer(faker.Server())
defer ts.Close()
// Difference in configuring the client
// Setup a new config
cfg, _ := config.LoadDefaultConfig(
context.TODO(),
config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider("KEY", "SECRET", "SESSION")),
config.WithHTTPClient(&http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
// Override the dial address because the SDK uses the bucket name as a subdomain.
DialContext: func(ctx context.Context, network, _ string) (net.Conn, error) {
dialer := net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}
s3URL, _ := url.Parse(s3Server.URL)
return dialer.DialContext(ctx, network, s3URL.Host)
},
},
}),
config.WithEndpointResolverWithOptions(
aws.EndpointResolverWithOptionsFunc(func(_, _ string, _ ...interface{}) (aws.Endpoint, error) {
return aws.Endpoint{URL: ts.URL}, nil
}),
),
)
// Create an Amazon S3 v2 client, important to use o.UsePathStyle
// alternatively change local DNS settings, e.g., in /etc/hosts
// to support requests to http://.127.0.0.1:32947/...
client := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.UsePathStyle = true
})
```
Please feel free to check it out and to provide useful feedback (using github
issues), but be aware, this software is used internally and for the local
development only. Thus, it has no demand for correctness, performance or
security.
There are different ways to run locally: e.g., using DNS, using S3 path mode, or V2 setting the ENV-Var:
```
os.Setenv("AWS_ENDPOINT_URL_S3", "http://localhost:9000")
```
S3 path mode is the most flexible and least restrictive, but it does require that you
are able to modify your client code. In Go, the modification would look like so:
config := aws.Config{}
config.WithS3ForcePathStyle(true)
S3 path mode works over the network by default for all bucket names.
If you are unable to modify the code, DNS mode can be used, but it comes with further
restrictions and requires you to be able to modify your local DNS resolution.
If using `localhost` as your endpoint, you will need to add the following to
`/etc/hosts` for *every bucket you want to fake*:
127.0.0.1 .localhost
It is trickier if you want other machines to be able to use your fake S3 server
as you need to be able to modify their DNS resolution as well.
## Exemplary usage
### Lambda Example
```javascript
var AWS = require('aws-sdk')
var ep = new AWS.Endpoint('http://localhost:9000');
var s3 = new AWS.S3({endpoint: ep});
exports.handle = function (e, ctx) {
s3.createBucket({
Bucket: '',
}, function(err, data) {
if (err) return console.log(err, err.stack);
ctx.succeed(data)
});
}
```
### Upload Example
```html
Key to upload:
Tags for File:
File:
```
## Similar notable projects
- https://github.com/minio/minio **not similar but powerfull ;-)**
- https://github.com/andrewgaul/s3proxy by @andrewgaul
## Contributors
A big thank you to all the [contributors](https://github.com/johannesboyne/gofakes3/graphs/contributors),
especially [Blake @shabbyrobe](https://github.com/shabbyrobe) who pushed this
little project to the next level!
**Help wanted**