{"id":28178431,"url":"https://github.com/mirasaki-development/navi-fs","last_synced_at":"2025-05-16T01:14:01.554Z","repository":{"id":188947670,"uuid":"679742587","full_name":"Mirasaki-Development/navi-fs","owner":"Mirasaki-Development","description":"Transform your local files into a secure, powerful API. Stream, serve, and execute using simple configuration and REST-style access.","archived":false,"fork":false,"pushed_at":"2025-04-16T12:59:30.000Z","size":409,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-16T14:06:30.218Z","etag":null,"topics":["api","file","file-api","file-sharing","local-files","local-filesystem","remote","remote-access","remote-files","sharing"],"latest_commit_sha":null,"homepage":"https://navifs.mirasaki.dev","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Mirasaki-Development.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-17T14:12:14.000Z","updated_at":"2025-04-16T12:59:34.000Z","dependencies_parsed_at":"2023-08-17T15:55:23.947Z","dependency_job_id":"5329df12-d81b-47ee-a167-a45dd85bb7e4","html_url":"https://github.com/Mirasaki-Development/navi-fs","commit_stats":{"total_commits":23,"total_committers":3,"mean_commits":7.666666666666667,"dds":"0.17391304347826086","last_synced_commit":"7af981235d75889ed9223b13fd9b0a7662ad1fa4"},"previous_names":["mirasaki/remote-file-access-api","mirasaki-development/remote-file-access-api","mirasaki-development/navi-fs"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirasaki-Development%2Fnavi-fs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirasaki-Development%2Fnavi-fs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirasaki-Development%2Fnavi-fs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Mirasaki-Development%2Fnavi-fs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Mirasaki-Development","download_url":"https://codeload.github.com/Mirasaki-Development/navi-fs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254448572,"owners_count":22072765,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","file","file-api","file-sharing","local-files","local-filesystem","remote","remote-access","remote-files","sharing"],"created_at":"2025-05-16T01:12:23.999Z","updated_at":"2025-05-16T01:14:01.543Z","avatar_url":"https://github.com/Mirasaki-Development.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg src=\".github/markdown-media/logo.png\" alt=\"NaviFS Logo\" height=\"60\" style=\"border-radius:50px\"/\u003e\u003c/p\u003e\n\u003ch1 align=\"center\"\u003eNaviFS\u003c/h1\u003e\n\n\u003cdiv align='center'\u003e\n\n[![CodeFactor](https://www.codefactor.io/repository/github/Mirasaki-Development/navi-fs/badge)](https://www.codefactor.io/repository/github/Mirasaki-Development/navi-fs)\n![GitHub](https://img.shields.io/github/license/Mirasaki-Development/navi-fs)\n![GitHub release](https://img.shields.io/github/v/release/Mirasaki-Development/navi-fs)\n\n\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  NaviFS (short for Navigate File System) transforms your local files into a secure, powerful API.\n  Stream, serve, and execute using simple configuration and REST-style access.\n\u003c/p\u003e\n\n## 🔨 Installation\n\n### 📦 Run as a Docker container (preferred)\n\nThe quickest, and easiest, way to host/use this application is by deploying it inside of a [Docker](https://www.docker.com/) container. We recommend [Docker Desktop](https://www.docker.com/products/docker-desktop/).\n\n1. Download the [latest release](https://github.com/Mirasaki-Development/navi-fs/releases`) or `git clone git@github.com:Mirasaki-Development/navi-fs.git` the repo\n2. Run `npm run setup:linux` or `npm run setup:windows` (depending on your OS) in the project root folder\n3. Edit the newly created `/config.yaml` file and provide your configuration\n4. Start the application: `docker compose up -d` (you can view logs with `docker compose logs -f`)\n\n### 🖥️ Run as a plain NodeJS app\n\n1. Install the additional pre-requisites:\n   - [Node.js](https://nodejs.org/en/) v16.6.0 or newer\n2. Download the [latest release](https://github.com/Mirasaki-Development/navi-fs/releases`) or `git clone git@github.com:Mirasaki-Development/navi-fs.git` the repo\n3. Run `npm run setup:linux` or `npm run setup:windows` in the project root folder\n4. Edit the newly created `/config.yaml` file and provide your configuration\n5. Start the application: `npm run start` for production, and `npm run dev` for local development\n\n## ⚙️ Configuration\n\nThe configuration for this project can be found [here](/config.example.yaml), and should be very straight-forward. Please see below for \"advanced\" usage examples.\n\n```yaml\nport: 9000 # Port to run the application on\nlog-level: info # debug, info, warn, or error\n\n# Master API key (must be strong, unique, and kept secret)\nmaster-api-key: \"a7fd8324d81e45a99e33b1c9a77ddca1c97f04a4e9aafb2ffb9273b7713ce1cb\"\n\n# List of resources to expose via the API\nresources:\n  - slug: project-files\n    target: /home/navi/projects/my-site\n    extensions:\n      - .html\n      - .css\n      - .js\n    size-policy:\n      - unit: bytes\n        limit: 1048576  # 1 MB\n        mode: truncate\n        cumulative: false\n        extensions: [\".log\", \".txt\"]\n        pattern: \".*\\\\.log$\"\n    cache-control:\n      max-age: 3600\n      s-maxage: 7200\n      no-cache: false\n      no-store: false\n      stale-while-revalidate: 86400\n      stale-if-error: 604800\n      public: true\n      private: false\n      immutable: false\n    permissions:\n      - type: [read, write]\n        api-key: \"6cbf9926bc7c4ea49ad0ffcb3a6d31e103eb8c7209c69df173bfe1f5f90aa9a3\"\n\n  - slug: run-script\n    target: /usr/local/bin/scripts\n    executable:\n      command: \"bash\"\n      args: [\"hello-world.sh\"]\n      cwd: \"/usr/local/bin/scripts\"\n      env:\n        MODE: \"production\"\n      inject-current-env: true\n      timeout: 300\n      detached: false\n      shell: \"/bin/bash\"\n    permissions:\n      - type: execute\n        api-key: \"4fa02b5dcfe84f2694ef7cbb6a198a3a94bd6b3e3f4696845edb4a983be70c26\"\n\n  - slug: config-json\n    target: /etc/myapp/config.json\n    extensions:\n      - .json\n    permissions:\n      - type: read\n        api-key: \"79b29829cebf4b65bb63ff51cdbeaad4f9b8e59aee93e2e5b709b2ef943f3468\"\n```\n## 📚 API Documentation\n\nNaviFS exposes a RESTful API to interact with your local files via secure endpoints.\n\nAll requests must include a valid `x-api-key` header.\n\n---\n\n### 🔐 Authentication\n\nThere are two types of API keys:\n\n- **Master API Key** – Grants access to list all available slugs.\n- **Per-Resource API Key** – Grants access to specific actions on a specific resource.\n\n---\n\n### 🌐 Endpoints\n\n#### `GET /`\n\nBasic health check. No authentication required.\n\n**Response:**\n\n```json\n{\n  \"data\": {\n    \"status\": \"alive\"\n  }\n}\n```\n\n## 📂 `GET /api/v1/:slug`\n\nRetrieves file contents or lists directory contents.\n\n**Query Parameters (optional):**\n\n- `cursor` – Subpath relative to resource root.\n- `type` – Output format: `stream`, `json`, `text`, `html`, `xml`, `csv`, `yaml`, `toml`\n- `recursive` – If `true`, list all nested files.\n\n**Headers:**\n\n- `x-api-key: \u003cRESOURCE_API_KEY\u003e`\n\n**Behavior:**\n\n- If resource is a file: streams file or parses to requested format.\n- If directory: lists contents (recursively if requested).\n- Sets `Cache-Control`, `ETag`, `Last-Modified` headers.\n- Responds `304` if `If-Modified-Since` or `If-None-Match` headers match.\n\n**Response:**\n\n```json\n{\n  \"data\": \"...\",  // string or list\n  \"meta\": {\n    \"slug\": \"logs/system\",\n    \"query\": { ... }\n  }\n}\n```\n\n---\n\n## 📥 `POST /api/v1/:slug`\n\nWrites to a file.\n\n**Query Parameters (optional):**\n\n- `cursor` – Path of file to write to.\n- `encoding` – Optional, e.g. `utf-8`, `base64`, etc.\n\n**Headers:**\n\n- `Content-Type: text/plain`\n- `x-api-key: \u003cRESOURCE_API_KEY\u003e`\n\n**Body:**  \nPlain string content to be written to file.\n\n**Behavior:**\n\n- Only works if target is a file.\n- Responds with `400` if request body is invalid.\n\n**Response:**  \n`201 Created` on success.\n\n---\n\n## 🗑️ `DELETE /api/v1/:slug`\n\nDeletes a file or directory.\n\n**Query Parameters (optional):**\n\n- `cursor` – Path to target file or directory.\n- `force` – Required to delete directories.\n\n**Headers:**\n\n- `x-api-key: \u003cRESOURCE_API_KEY\u003e`\n\n**Behavior:**\n\n- Deletes file directly.\n- If target is a directory, requires `force=true`.\n\n**Response:**  \n`204 No Content` on success.\n\n---\n\n## 🧠 `EXECUTE /api/v1/:slug`\n\nExecutes a command via SSE (Server-Sent Events).\n\n**Query Parameters (optional):**\n\n- `cursor` – Optional execution path override.\n\n**Headers:**\n\n- `Accept: text/event-stream`\n- `x-api-key: \u003cRESOURCE_API_KEY\u003e`\n\n**Behavior:**\n\n- Only allowed for resources that have an `executable` configuration defined.\n- Streams `stdout`, `stderr`, `exit`, and `error` events.\n\n**Response Format (SSE):**\n\n```json\nevent: stdout\ndata: Hello from stdout\n\nevent: stderr\ndata: Warning: something happened\n\nevent: exit\ndata: Process exited with code 0\n```\n\n---\n\n## 🧪 `HEAD /api/v1/:slug`\n\nReturns basic info and cache headers.\n\n**Headers:**\n\n- `x-api-key: \u003cRESOURCE_API_KEY\u003e`\n\n**Behavior:**\n\n- Sets `Cache-Control` based on resource configuration.\n- Returns `200 OK`.\n\n---\n\n## ⚙️ `OPTIONS /api/v1/:slug`\n\nReturns allowed methods.\n\n**Response Headers:**\n\n```json\nAllow: HEAD, OPTIONS, GET, POST, DELETE, EXECUTE\n```\n\n---\n\n### ⚠️ Error Handling\n\nAll errors return a consistent format:\n\n```json\n{\n  \"error\": {\n    \"code\": \"BAD_REQUEST\" | \"NOT_FOUND\" | \"METHOD_NOT_ALLOWED\" | \"INTERNAL_ERROR\",\n    \"message\": \"Human-readable error message\",\n    \"details\": {\n      \"slug\": \"...\",\n      \"query\": { ... },\n      \"error\": \"Optional extra error string\"\n    }\n  }\n}\n```\n\n\u003e In development mode, error messages may include stack traces or exception details.\n\n## 🛣️ Roadmap\n\nThis section outlines the current and upcoming features and enhancements planned for the project. Here's where we plan to go:\n\n### 🚀 Current Milestones\n\n- [ ] **Improved Error Handling:** Better error messages with more specific error codes.\n- [ ] **File Streaming Enhancements:** Optimizing file streaming (through Ranges) for large file resources.\n- [ ] **Search Functionality:** Add advanced search and filtering capabilities for resource queries.\n- [ ] **Cloud Storage Integrations:** Support for external cloud storage providers (e.g., AWS S3, Google Cloud Storage).\n- [ ] **Frontend Panel/Dashboard** for administrators (long-term vision).\n    - [ ] **User Authentication \u0026 Authorization:** Allow users to create accounts, authenticate via OAuth, and manage their API keys.\n    - [ ] **Resource Management Dashboard:** Build an admin interface for better management and exploration of resources.\n\n### 📖 Roadmap Entry: OS-Level File Caching Documentation\n\n**Description**: Document the operating system's file caching mechanism, focusing on how the OS optimizes file reads by caching frequently accessed data in memory. This will include the use of page caches, filesystem caches, and memory-mapped files for improving read performance.\n\n- [ ] Explain how the OS stores file data in memory after the first read to optimize subsequent file access.\n- [ ] Detail the page cache and filesystem cache mechanisms.\n- [ ] Provide insights into write-back vs. write-through caching strategies.\n- [ ] Include a section on memory-mapped files (mmap) and how they interact with caching.\n- [ ] Describe cache eviction and limitations (e.g., when memory is low, and files are evicted).\n- [ ] Discuss the benefits of caching for performance and disk wear reduction, and explain scenarios where caching may not be effective.\n\nIf you have any suggestions or would like to see a specific feature, feel free to open an issue or submit a pull request!\n\n## 🤝 Contributing\n\nWe welcome contributions of all kinds! Whether it's fixing a bug, improving documentation, suggesting a new feature, or helping with tests — every bit helps make NaviFS better for everyone.\n\nTo contribute: (Please note the project is in an early stage, and currently doesn't have any tests set-up - any PRs for this are appreciated!)\n\n1. **Fork the repository** and create your branch from `main`\n2. **Make your changes** and ensure existing tests pass\n3. If applicable, add tests for your changes\n4. **Submit a pull request** with a clear description of your work\n\nBefore submitting, please review our [Code of Conduct](./CODE_OF_CONDUCT.md) and [Contribution Guidelines](./CONTRIBUTING.md) (coming soon).\n\n\u003e Found a bug or have an idea? Open an issue — let’s talk!\n\n## ⭐ Credits and Attribution\n\n- Logo by [Flaticon](https://www.flaticon.com/free-icons/distance-education)\n\n\u003cbr /\u003e\n\n\u003e Open source, self-hosted, and [Free Culture licensed](https://creativecommons.org/share-your-work/public-domain/freeworks), meaning you're in full control.\n\n\u003cbr /\u003e\n\n\u003cp align=\"center\"\u003e\u003ca href=\"https://github.com/Mirasaki-Development/navi-fs#navi-fs\"\u003e\u003cimg src=\"http://randojs.com/images/backToTopButton.png\" alt=\"Back to top\" height=\"29\"/\u003e\u003c/a\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirasaki-development%2Fnavi-fs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmirasaki-development%2Fnavi-fs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmirasaki-development%2Fnavi-fs/lists"}