Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/vgrichina/nearfs
Distributed file system compatible with IPFS using NEAR blockchain as a backend.
https://github.com/vgrichina/nearfs
blockchain ipfs nearprotocol web3 web4
Last synced: 14 days ago
JSON representation
Distributed file system compatible with IPFS using NEAR blockchain as a backend.
- Host: GitHub
- URL: https://github.com/vgrichina/nearfs
- Owner: vgrichina
- Created: 2022-12-09T01:51:17.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-10-02T21:42:11.000Z (about 1 month ago)
- Last Synced: 2024-10-18T12:56:23.213Z (28 days ago)
- Topics: blockchain, ipfs, nearprotocol, web3, web4
- Language: JavaScript
- Homepage:
- Size: 3.56 MB
- Stars: 23
- Watchers: 2
- Forks: 3
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# NEARFS
NEARFS is a distributed file system compatible with IPFS that uses the NEAR blockchain as a backend. It is a work in progress.
## TLDR
- IPFS blocks are recorded in transaction history as `fs_store` function calls
- custom indexer can be used to collect all blocks and store them in a local file system (see `scripts/load-from-near-lake.js`)
- IPFS-compatible gateway can be used to access files (see `app.js`)
- Public gateways are available:
- https://ipfs.web4.near.page. Access data stored on NEAR mainnet.
- https://ipfs.web4.testnet.page. Access data stored on NEAR testnet.
## Useful tools
- [web4-deploy](https://github.com/vgrichina/web4-deploy) Deploy your website to NEAR blockchain + IPFS/NEARFS.
- [fast-ipfs](https://github.com/vgrichina/fast-ipfs) Low level utilities to work with IPFS blocks, etc.
- https://near.page Web4 gateway which uses NEARFS to resolve `ipfs://` links when possible.
## How to run gateway
- `yarn install`
- `yarn start`
## How to run indexer
- `yarn install`
- `node scripts/load-from-near-lake.js --bucket-name [near-lake-bucket-name]`
## Environment variables
- `PORT` - port to run gateway on
- `NEARFS_STORAGE_PATH` - path to NEARFS block storage
- `NEARFS_LOAD_NEAR_LAKE` - if set to `true` (or `yes`) gateway will also load blocks from NEAR Lake. Use this if you want to avoid running `load-from-near-lake.js` script separately.
- `NEARFS_LAKE_BUCKET_NAME` - name of the NEAR Lake S3 bucket to load blocks from
- `NEARFS_LAKE_REGION_NAME` - region of the NEAR Lake S3 bucket to load blocks from
- `NEARFS_LAKE_ENDPOINT` - endpoint of the NEAR Lake S3 bucket to load blocks from
- `NEARFS_LAKE_BATCH_SIZE` - number of blocks to load from NEAR Lake in one batch
- `NEARFS_LAKE_INCLUDE` - comma-separated list of account glob patterns to include when loading blocks from NEAR Lake
- `NEARFS_LAKE_EXCLUDE` - comma-separated list of account glob patterns to exclude when loading blocks from NEAR Lake
- `AWS_ACCESS_KEY_ID` - AWS access key ID to use when loading blocks from NEAR Lake
- `AWS_SECRET_ACCESS_KEY` - AWS secret access key to use when loading blocks from NEAR Lake
## API
Following IPFS gateway APIs are currently implemented.
Note that there might be some differences in details like error handling, etc.
Most notable difference with regular IPFS is that all calls would either return immediately or 404. There is no wait for content to be fetched from other nodes, etc.
Also only CIDv1 is currently supported (see examples).
### `GET /ipfs/:cid`
Returns IPFS file with given CID.
**Example:**
https://ipfs.web4.near.page/ipfs/bafybeicit72w2sl3agal2jftpkrzwd773fjgdk4dym7pq2pbojyif72v5e
### `GET /ipfs/:cid/:path`
Returns IPFS file with given CID and path.
**Example:**
https://ipfs.web4.near.page/ipfs/bafybeiepywlzwr2yzyin2bo7k2v5oi37lsgleyvfrf6erjvlze2qec6wkm/privacy.html
### Directory listing
If directory is requested and it has no `index.html` – HTML with its contents is returned.
Otherwise `index.html` is served.
**Example directory**:
https://ipfs.web4.near.page/ipfs/bafybeiepywlzwr2yzyin2bo7k2v5oi37lsgleyvfrf6erjvlze2qec6wkm/
**Example index.html**:
https://ipfs.web4.near.page/ipfs/bafybeidg3ohf4kscsf6cjbgg7vttcvu7q4olena3kwhpl5wl3trhhougyi/dist/
### Subdomain Support
NEARFS supports accessing content via subdomains. This allows serving full websites via NEARFS gateway with isolated security context.
#### How it works
- The gateway checks if the hostname contains a subdomain.
- If the subdomain is a valid CID, it serves the content associated with that CID.
- The path after the domain is treated as the path within the IPFS directory structure.
#### `GET http://.ipfs.web4.near.page/`
Returns the content of the root directory for the given CID.
**Example:**
http://bafybeicit72w2sl3agal2jftpkrzwd773fjgdk4dym7pq2pbojyif72v5e.ipfs.web4.near.page/
#### `GET http://.ipfs.web4.near.page/:path`
Returns the file at the specified path within the directory structure of the given CID.
**Example:**
http://bafybeiepywlzwr2yzyin2bo7k2v5oi37lsgleyvfrf6erjvlze2qec6wkm.ipfs.web4.near.page/privacy.html
This feature provides a more intuitive way to share and access IPFS content through the NEARFS gateway.