https://github.com/threefoldtech/qss-ipfs-pinningserver
ipfs pinning service
https://github.com/threefoldtech/qss-ipfs-pinningserver
Last synced: about 2 months ago
JSON representation
ipfs pinning service
- Host: GitHub
- URL: https://github.com/threefoldtech/qss-ipfs-pinningserver
- Owner: threefoldtech
- License: apache-2.0
- Created: 2022-09-14T15:15:13.000Z (over 2 years ago)
- Default Branch: development
- Last Pushed: 2023-10-12T04:09:29.000Z (over 1 year ago)
- Last Synced: 2024-06-21T06:27:21.063Z (11 months ago)
- Language: Go
- Size: 346 KB
- Stars: 0
- Watchers: 19
- Forks: 1
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![MIT License][license-shield]][license-url]
[![LinkedIn][linkedin-shield]][linkedin-url]
TF Pinning Service
IPFS Pinning Service over TF Quantum-Safe Storage
Explore the docs »
View Demo
·
Report Bug
·
Request Feature
Table of Contents
-
About The Project
-
Getting Started
- Usage
- Interacting With The Pinning Service
- API Specs
- COMPLIANCE TEST SUITE
- Roadmap
- Contributing
- License
- Contact
- Acknowledgments
## About The Project
IPFS pinning service backed by ThreeFold decentralized grid and Quantum Safe Filesystem.
### Built With
* [Go](https://go.dev/)
* [IPFS](https://ipfs.io/)
* [QSFS](https://github.com/threefoldtech/0-stor_v2)
* [Vue](https://vuejs.org/)
## Getting Started
Threefold pinning service is a IPFS pinning service that complies with the IPFS Pinning Service API specification. it combines the content addressing capabilities of IPFS, the decentralized, Peer To Peer Internet infrastructure of Threefold, and Threefold's distributed and ultra-secure storage backend QSFS(Quantum-Safe Storage Filesystem) in order to achieve a powerful decentralized storage solution, wrapping the power of these systems into an easy-to-use service with simple rest API.
### Prerequisites
- An IPFS cluster with at least one IPFS peer.
- For development you can spin up a local IPFS Cluster instance. [setup instructions](https://docs.ipfs.tech/install/server-infrastructure/#create-a-local-cluster).
- For setup IPFS and IPFS Cluster on a production environment see [here](https://ipfscluster.io/documentation/deployment/)
- The tfpin web service binary.
- You can download it from releases or compile from the source code.
If you need to compile from the source code, you will need also:
- An installation of Go 1.16 or later. [installation instructions](https://go.dev/doc/install)
- Git client
### Compile
To compile tfpin binary from the source code, follow below instructions
1 - Clone the repository:
- Open Terminal, Change the current working directory to the location where you want the cloned directory, and type
```sh
git clone https://github.com/threefoldtech/tf-pinning-service.git
```
2- Compile:
- Change to `./tf-pinning-service` and type
```sh
make build
```
Then find the compiled binary file in `./bin` for current os and `./bin/linux_amd64` for linux os.
Other make scripts available.
```sh
make run # run the web server
```
```sh
make build_run # compile and run the web server
```
Make sure to set correctly the required environment variables. see below usage section.
## Usage
1- set the required environment variables (see next section), then run the compiled binary
```sh
TFPIN_CLUSTER_HOSTNAME="cluster-host-name" TFPIN_CLUSTER_USERNAME="usname" TFPIN_CLUSTER_PASSWORD="password" ./tfpin
```
2- You will need API token to interact with the service, for development run the script below to add a token of your choice (the development of the real sign-up flow blocked on the the chain part for now):
```sh
go run ./scripts/add_test_tokens.go BestTokenEver
Token `BestTokenEver` stored successfully.
```
### Environment Variables
| Environment Variable | Description | Default Value |
|--- |--- |--- |
| TFPIN_CLUSTER_HOSTNAME | | 127.0.0.1 |
| TFPIN_CLUSTER_PORT | | 9097 |
| TFPIN_CLUSTER_USERNAME | | "" |
| TFPIN_CLUSTER_PASSWORD | | "" |
| TFPIN_CLUSTER_REPLICA_MIN | | -1 (Pin on all cluster IPFS peers) |
| TFPIN_CLUSTER_REPLICA_MAX | | -1 (Pin on all cluster IPFS peers) |
| TFPIN_DB_DSN | | pins.db |
| TFPIN_DB_LOG_LEVEL | | 1 (silent) |
| TFPIN_SERVER_ADDR | | :8000 |
| TFPIN_SERVER_LOG_LEVEL | | 3 (warn) |
| TFPIN_AUTH_HEADER_KEY | | Authorization |
### Docker
#### Building Image
building binaries with version information and other metadata will improve the monitoring, logging, and debugging processes by adding identifying information to help track the builds over time.
this why you should provide a value for the build arg `VERSION`.
```sh
VERSION=$([ ! -z "$(git tag --points-at HEAD)" ] && git tag --points-at HEAD || git rev-parse --short HEAD) # use current tag, if none, use commit hash
docker build --build-arg VERSION="$VERSION" -t abouelsaad/tfpinsvc .
```
#### Running the container
```sh
docker run -d --name tf-pin-svc --env TFPIN_CLUSTER_HOSTNAME="HOSTNAME" --env TFPIN_CLUSTER_PORT="9094" --env TFPIN_CLUSTER_USERNAME="USERNAME" --env TFPIN_CLUSTER_PASSWORD="PASSWORD" --env TFPIN_SERVER_ADDR=":80" -p 8000:80 abouelsaad/tfpinsvc
```
Adding test token
```sh
docker exec tf-pin-svc ./add_test_tokens BestTokenEver
```
### Docker Compose
For development and testing, with one command you can spin up
- an IPFS cluster, and
- Threefold pinning service
which is already configured to communicate, so you just need to get/add a test token, and start interacting with the service.
```sh
docker compose up -d
docker exec tf-pin-svc ./add_test_tokens BestTokenEver
```
## Interacting With The Pinning Service
To interact with the pinning service you can use any of:
- IPFS Desktop or IPFS Web UI [installation instructions](https://github.com/ipfs/ipfs-desktop#install)
- IPFS CLI [installation instructions](https://docs.ipfs.tech/install/command-line/)
- Any http client, like `curl`
See Use pinning service [instructions](https://docs.ipfs.tech/how-to/work-with-pinning-services/#use-an-existing-pinning-service)
The threefold pinning service endpoint for all requests is
https://[hostname]/api/v1/pins
### Using the HTTP API
#### Add a pin
```sh
curl -X POST 'https://[HOSTNAME]/api/v1/pins' \
--header 'Accept: */*' \
--header 'Authorization: Bearer ' \
--header 'Content-Type: application/json' \
-d '{
"cid": "",
"name": "PreciousData.pdf"
}'
```
#### List successful pins
```sh
curl -X GET 'https://[HOSTNAME]/api/v1/pins' \
--header 'Accept: */*' \
--header 'Authorization: Bearer '
```
#### Delete a pin
```sh
curl -X DELETE 'https://[HOSTNAME]/api/v1/pins/' \
--header 'Accept: */*' \
--header 'Authorization: Bearer '
```
### Using the IPFS CLI
The IPFS CLI can be used to maintain pins by first adding the threefold pinning service.
```sh
ipfs pin remote service add tfpinsvc https://[HOSTNAME]/api/v1/
```
#### Add a pin
```sh
ipfs pin remote add --service=tfpinsvc --name=
```
#### List successful pins
```sh
ipfs pin remote ls --service=tfpinsvc
```
#### Delete a pin
```sh
ipfs pin remote rm --service=tfpinsvc --cid=
```
### Using the IPFS Desktop GUI app
see [here](https://docs.ipfs.tech/how-to/work-with-pinning-services/#ipfs-desktop-or-ipfs-web-ui)
## API Specs
Threefold pinning service is compatible with the IPFS Pinning Service API (1.0.0) OpenAPI spec.
see [here](./pinning-api/README.md)
## COMPLIANCE TEST SUITE
This tool should be used as compliance test suite for IPFS Pinning Service API Spec.
https://github.com/ipfs-shipyard/pinning-service-compliance
## Roadmap
- [ ] simple ipfs pining service server compatible with this OpenAPI spec (https://github.com/ipfs/pinning-services-api-spec).
- [ ] registration using smart contract
- [ ] payment
- [ ] Use QSFS as blocks storage layer
See the [open issues](https://github.com/threefoldtech/tf-pinning-service/issues) for a full list of proposed features (and known issues).
## Contributing
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are **greatly appreciated**.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement".
Don't forget to give the project a star! Thanks again!
1. Fork the Project
2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`)
3. Commit your Changes (`git commit -m 'Add some AmazingFeature'`)
4. Push to the Branch (`git push origin feature/AmazingFeature`)
5. Open a Pull Request
## License
Distributed under the Apache License. See [`LICENSE`](LICENSE) for more information.
## Contact
Threefold - [@threefold_io](https://twitter.com/threefold_io)
Project Link: [https://github.com/threefoldtech/tf-pinning-service](https://github.com/threefoldtech/tf-pinning-service)
## Acknowledgments
* [Pinning services api spec](https://ipfs.github.io/pinning-services-api-spec/)
* [Work with pinning services](https://docs.ipfs.tech/how-to/work-with-pinning-services/#use-an-existing-pinning-service)
* [Threefold quantum safe filesystem](https://library.threefold.me/info/manual/#/technology/threefold__qsfs)
[contributors-shield]: https://img.shields.io/github/contributors/threefoldtech/tf-pinning-service.svg?style=for-the-badge
[contributors-url]: https://github.com/threefoldtech/tf-pinning-service/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/threefoldtech/tf-pinning-service.svg?style=for-the-badge
[forks-url]: https://github.com/threefoldtech/tf-pinning-service/network/members
[stars-shield]: https://img.shields.io/github/stars/threefoldtech/tf-pinning-service.svg?style=for-the-badge
[stars-url]: https://github.com/threefoldtech/tf-pinning-service/stargazers
[issues-shield]: https://img.shields.io/github/issues/threefoldtech/tf-pinning-service.svg?style=for-the-badge
[issues-url]: https://github.com/threefoldtech/tf-pinning-service/issues
[license-shield]: https://img.shields.io/github/license/threefoldtech/tf-pinning-service.svg?style=for-the-badge
[license-url]: https://github.com/threefoldtech/tf-pinning-service/blob/master/LICENSE.txt
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
[linkedin-url]: https://linkedin.com/in/linkedin_username