{"id":29843329,"url":"https://github.com/stackql/stackql-provider-utils","last_synced_at":"2025-07-29T15:43:02.682Z","repository":{"id":304140520,"uuid":"1017806165","full_name":"stackql/stackql-provider-utils","owner":"stackql","description":"Shared library of utilities for building and documenting StackQL providers from OpenAPI specifications. Works with Node.js and Deno.","archived":false,"fork":false,"pushed_at":"2025-07-11T09:05:37.000Z","size":164,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-11T12:25:23.819Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stackql.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-11T06:06:48.000Z","updated_at":"2025-07-11T09:05:41.000Z","dependencies_parsed_at":"2025-07-11T12:25:40.490Z","dependency_job_id":"38a3f8a7-e189-463e-a78d-89979d9567ce","html_url":"https://github.com/stackql/stackql-provider-utils","commit_stats":null,"previous_names":["stackql/stackql-provider-utils"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/stackql/stackql-provider-utils","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-provider-utils","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-provider-utils/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-provider-utils/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-provider-utils/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stackql","download_url":"https://codeload.github.com/stackql/stackql-provider-utils/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-provider-utils/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267710239,"owners_count":24131936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-29T15:43:00.418Z","updated_at":"2025-07-29T15:43:02.668Z","avatar_url":"https://github.com/stackql.png","language":"JavaScript","readme":"# StackQL Provider Utils\n\nA comprehensive toolkit for transforming OpenAPI specs into StackQL providers. Includes parsing, mapping, validation, testing, and documentation generation utilities. Compatible with both Node.js and Deno.\n\n## Table of Contents\n\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Local Development Setup](#local-development-setup)\n- [Testing with Node.js](#testing-with-nodejs)\n- [Testing with Deno](#testing-with-deno)\n- [Using the Documentation Generator](#using-the-documentation-generator)\n- [API Reference](#api-reference)\n- [Contributing](#contributing)\n\n## Prerequisites\n\n### For Node.js\n- Node.js \u003e= 20\n- npm or yarn\n- StackQL server (for documentation generation)\n\n### For Deno\n- Deno \u003e= 1.30.0\n- StackQL server (for documentation generation)\n\n### Installing StackQL\n\nDownload and install StackQL from [stackql.io/downloads](https://stackql.io/downloads)\n\n```bash\n# macOS\nbrew install stackql\n\n# Linux\ncurl -L https://bit.ly/stackql-zip -O \u0026\u0026 unzip stackql-zip\n\n# Windows\n# Download from https://stackql.io/downloads\n```\n\n## Installation\n\n### For Node.js Projects\n\n```bash\nnpm install @stackql/provider-utils\n# or\nyarn add @stackql/provider-utils\n```\n\n### For Deno Projects\n\n```typescript\nimport { docgen } from \"https://deno.land/x/stackql_provider_utils/mod.ts\";\n```\n\n## Local Development Setup\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/stackql/stackql-provider-utils.git\ncd stackql-provider-utils\n```\n\n2. Install dependencies (Node.js):\n```bash\nnpm install\n```\n\n## Testing with Node.js\n\n### 1. Create a Test Script\n\nCreate a file `test-docgen.js`:\n\n```javascript\nimport { docgen } from './src/index.js';\n\n// Test the documentation generator\nasync function testDocGen() {\n    try {\n        const result = await docgen.generateDocs({\n            providerName: 'myservice',\n            providerDir: './test-data/output/src/myservice/v00.00.00000',\n            outputDir: './test-output',\n            providerDataDir: './test-data/provider-data',\n            stackqlConfig: {\n                host: 'localhost',\n                port: 5444,\n                user: 'stackql',\n                database: 'stackql'\n            }\n        });\n        \n        console.log('Documentation generated successfully:', result);\n    } catch (error) {\n        console.error('Error generating documentation:', error);\n    }\n}\n\ntestDocGen();\n```\n\n### 2. Set Up Test Data\n\nCreate the required directory structure:\n\n```bash\nmkdir -p test-data/output/src/myservice/v00.00.00000/services\nmkdir -p test-data/provider-data\n```\n\nAdd test files:\n\n`test-data/provider-data/headerContent1.txt`:\n```\n---\ntitle: myservice\nhide_title: false\nhide_table_of_contents: false\nkeywords:\n  - myservice\n  - stackql\n  - infrastructure-as-code\n  - configuration-as-data\ndescription: Query and manage myservice resources using SQL\n---\n\n# myservice Provider\n\nThe myservice provider for StackQL allows you to query, deploy, and manage myservice resources using SQL.\n```\n\n`test-data/provider-data/headerContent2.txt`:\n```\nSee the [myservice provider documentation](https://myservice.com/docs) for more information.\n```\n\n`test-data/output/src/myservice/v00.00.00000/services/example.yaml`:\n```yaml\nopenapi: 3.0.0\ninfo:\n  title: Example Service\n  version: 1.0.0\npaths:\n  /examples:\n    get:\n      operationId: listExamples\n      responses:\n        '200':\n          description: Success\ncomponents:\n  x-stackQL-resources:\n    examples:\n      id: myservice.example.examples\n      name: examples\n      title: Examples\n      methods:\n        list:\n          operation:\n            $ref: '#/paths/~1examples/get'\n          response:\n            mediaType: application/json\n            openAPIDocKey: '200'\n      sqlVerbs:\n        select:\n          - $ref: '#/components/x-stackQL-resources/examples/methods/list'\n```\n\n### 3. Start StackQL Server\n\n```bash\n# In a separate terminal\nstackql srv \\\n  --pgsrv.port=5444 \\\n  --pgsrv.tls=false \\\n  --loglevel=INFO\n```\n\n### 4. Run the Test\n\n```bash\nnode test-docgen.js\n```\n\n## Testing with Deno\n\n### 1. Create a Test Script\n\nCreate a file `test-docgen.ts`:\n\n```typescript\nimport { docgen } from './src/mod.ts';\n\n// Test the documentation generator\nasync function testDocGen() {\n    try {\n        const result = await docgen.generateDocs({\n            providerName: 'myservice',\n            providerDir: './test-data/output/src/myservice/v00.00.00000',\n            outputDir: './test-output',\n            providerDataDir: './test-data/provider-data',\n            stackqlConfig: {\n                host: 'localhost',\n                port: 5444,\n                user: 'stackql',\n                database: 'stackql'\n            }\n        });\n        \n        console.log('Documentation generated successfully:', result);\n    } catch (error) {\n        console.error('Error generating documentation:', error);\n    }\n}\n\ntestDocGen();\n```\n\n### 2. Run the Test\n\n```bash\n# With permissions\ndeno run --allow-read --allow-write --allow-net test-docgen.ts\n```\n\n## Using the Documentation Generator\n\n### Basic Example\n\n```javascript\nimport { docgen } from '@stackql/provider-utils';\n\nconst options = {\n    providerName: 'github',\n    providerDir: './output/src/github/v00.00.00000',\n    outputDir: './docs',\n    providerDataDir: './config/provider-data',\n    stackqlConfig: {\n        host: 'localhost',\n        port: 5444,\n        user: 'stackql',\n        database: 'stackql'\n    }\n};\n\nconst result = await docgen.generateDocs(options);\nconsole.log(`Generated docs for ${result.totalServices} services and ${result.totalResources} resources`);\nconsole.log(`Output location: ${result.outputPath}`);\n```\n\n### Options\n\n| Option | Type | Description | Default |\n|--------|------|-------------|---------|\n| `providerName` | string | Name of the provider (e.g., 'github', 'aws') | Required |\n| `providerDir` | string | Path to provider spec directory | Required |\n| `outputDir` | string | Directory for generated documentation | Required |\n| `providerDataDir` | string | Directory containing provider header files | Required |\n| `stackqlConfig` | object | StackQL server connection configuration | See below |\n\n#### StackQL Config Options\n\n```javascript\n{\n    host: 'localhost',  // StackQL server host\n    port: 5444,         // StackQL server port\n    user: 'stackql',    // Database user\n    database: 'stackql' // Database name\n}\n```\n\n## Directory Structure Requirements\n\n### Provider Data Directory\n```\nprovider-data/\n├── headerContent1.txt    # Provider introduction\n├── headerContent2.txt    # Additional provider info\n└── stackql-provider-registry.mdx (optional)\n```\n\n### Provider Spec Directory\n```\noutput/src/{provider}/v00.00.00000/\n├── provider.yaml\n└── services/\n    ├── service1.yaml\n    ├── service2.yaml\n    └── ...\n```\n\n### Generated Output\n```\ndocs/{provider}-docs/\n├── index.md\n├── stackql-provider-registry.mdx\n└── providers/\n    └── {provider}/\n        └── {service}/\n            ├── index.md\n            └── {resource}/\n                └── index.md\n```\n\n## Running Tests\n\n### Node.js\n```bash\nnpm test\n```\n\n### Deno\n```bash\ndeno task test\n```\n\n## Troubleshooting\n\n### StackQL Server Connection Issues\n- Ensure StackQL server is running: `stackql srv --pgsrv.port=5444`\n- Check if port 5444 is available\n- Verify connection settings in `stackqlConfig`\n\n### Missing Provider Data\n- Ensure `headerContent1.txt` and `headerContent2.txt` exist in provider data directory\n- Check file permissions\n\n### Empty Documentation\n- Verify provider specs have `x-stackQL-resources` components\n- Check that resources have proper method definitions\n\n## API Reference\n\n### `docgen.generateDocs(options)`\n\nGenerates documentation for a StackQL provider.\n\n**Parameters:**\n- `options` (Object): Configuration options\n\n**Returns:**\n- Promise\u003cObject\u003e: Result object containing:\n  - `totalServices`: Number of services processed\n  - `totalResources`: Number of resources documented\n  - `outputPath`: Path to generated documentation\n\n**Example:**\n```javascript\nconst result = await docgen.generateDocs({\n    providerName: 'aws',\n    providerDir: './providers/src/aws/v00.00.00000',\n    outputDir: './documentation',\n    providerDataDir: './config/aws',\n    stackqlConfig: {\n        host: 'localhost',\n        port: 5444\n    }\n});\n```\n\n### `docgen.createResourceIndexContent(...)`\n\nCreates markdown content for a single resource. This is a lower-level function used internally by `generateDocs`.\n\n## Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/amazing-feature`)\n3. Commit your changes (`git commit -m 'Add some amazing feature'`)\n4. Push to the branch (`git push origin feature/amazing-feature`)\n5. Open a Pull Request\n\n## License\n\nMIT\n\n## Support\n\n- [StackQL Documentation](https://stackql.io/docs)\n- [GitHub Issues](https://github.com/stackql/stackql-provider-utils/issues)\n- [StackQL Discord](https://discord.gg/stackql)","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackql%2Fstackql-provider-utils","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstackql%2Fstackql-provider-utils","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackql%2Fstackql-provider-utils/lists"}