{"id":31144094,"url":"https://github.com/ankan97dutta/profilis","last_synced_at":"2026-03-10T22:17:08.830Z","repository":{"id":312411164,"uuid":"1042267328","full_name":"ankan97dutta/profilis","owner":"ankan97dutta","description":"High performance, non blocking profiler for modern Python web apps. Flask, FastAPI, Sanic, SQLAlchemy, MongoDB and more.","archived":false,"fork":false,"pushed_at":"2025-09-11T15:33:24.000Z","size":796,"stargazers_count":1,"open_issues_count":12,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-11T18:34:10.857Z","etag":null,"topics":["application-performance-monitoring","developer-tools","fastapi","flask","mongodb","neo4j","observability","performance-monitoring","profiler","profiling-tools","prometheus-metrics","pyodbc","python","python-library","sanic","spyglass","sqlalchemy","tracing"],"latest_commit_sha":null,"homepage":"https://ankan97dutta.github.io/profilis/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ankan97dutta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-21T18:35:29.000Z","updated_at":"2025-09-06T08:17:10.000Z","dependencies_parsed_at":"2025-08-30T13:18:30.745Z","dependency_job_id":"9afeef80-8e06-424d-9072-5c51df9aac55","html_url":"https://github.com/ankan97dutta/profilis","commit_stats":null,"previous_names":["ankan97dutta/spyglass","ankan97dutta/profilis"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ankan97dutta/profilis","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankan97dutta%2Fprofilis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankan97dutta%2Fprofilis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankan97dutta%2Fprofilis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankan97dutta%2Fprofilis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ankan97dutta","download_url":"https://codeload.github.com/ankan97dutta/profilis/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ankan97dutta%2Fprofilis/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275781194,"owners_count":25527351,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-18T02:00:09.552Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["application-performance-monitoring","developer-tools","fastapi","flask","mongodb","neo4j","observability","performance-monitoring","profiler","profiling-tools","prometheus-metrics","pyodbc","python","python-library","sanic","spyglass","sqlalchemy","tracing"],"created_at":"2025-09-18T14:11:20.584Z","updated_at":"2026-03-10T22:17:08.815Z","avatar_url":"https://github.com/ankan97dutta.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg width=\"64\" height=\"64\" alt=\"image\" src=\"https://github.com/user-attachments/assets/663b4497-d023-49a6-9ce9-60c50c86df02\" /\u003e\n\n# Profilis\n\n\u003e A high performance, non-blocking profiler for Python web applications.\n\n[![Docs](https://github.com/ankan97dutta/profilis/actions/workflows/docs.yml/badge.svg)](https://ankan97dutta.github.io/profilis/)\n[![CI](https://github.com/ankan97dutta/profilis/actions/workflows/ci.yml/badge.svg)](https://github.com/ankan97dutta/profilis/actions/workflows/ci.yml)\n[![PyPI Downloads](https://static.pepy.tech/personalized-badge/profilis?period=total\u0026units=INTERNATIONAL_SYSTEM\u0026left_color=black\u0026right_color=BRIGHTGREEN\u0026left_text=downloads)](https://pepy.tech/projects/profilis)\n---\n\n## Overview\n\nProfilis provides drop-in observability across APIs, functions, and database queries with minimal performance impact. It's designed to be:\n\n- **Non blocking**: Async collection with configurable batching and backpressure handling\n- **Framework agnostic**: Flask, FastAPI, and Sanic with optional ASGI middleware for any ASGI app\n- **Database aware**: SQLAlchemy (sync \u0026 async), MongoDB (PyMongo), Neo4j, and pyodbc\n- **Production ready**: Configurable sampling, error tracking, and multiple export formats\n\n\u003cimg width=\"1126\" height=\"642\" alt=\"Screenshot 2025-09-01 at 12 38 50 PM\" src=\"https://github.com/user-attachments/assets/7c9d541b-4984-4575-92fb-8c0ec48dff55\" /\u003e\n\n## Star This Repository\n\nIf you find Profilis helpful for your projects, please consider giving it a star! It helps others discover this tool and motivates continued development.\n\n[![GitHub stars](https://img.shields.io/github/stars/ankan97dutta/profilis?style=social)](https://github.com/ankan97dutta/profilis)\n\n## Features\n\n- **Request Profiling**: Automatic HTTP request/response timing and status tracking\n- **Frameworks**: Flask, FastAPI (ASGI middleware), and Sanic with built-in dashboard (Flask blueprint, FastAPI router, Sanic blueprint)\n- **Function Profiling**: Decorator-based function timing with exception tracking\n- **Database Instrumentation**: SQLAlchemy (sync \u0026 async), MongoDB (PyMongo), Neo4j, pyodbc with query/command monitoring\n- **Built-in UI**: Real-time dashboard for monitoring and debugging\n- **Multiple Exporters**: JSONL (with rotation), Console\n- **Runtime Context**: Distributed tracing with trace/span ID management\n- **Configurable Sampling**: Control data collection volume (Flask, ASGI, Sanic)\n\n\n## Installation\n\nInstall the core package with optional dependencies for your specific needs:\n\n### Option 1: Using pip with extras (Recommended)\n\n```bash\n# Core package only\npip install profilis\n\n# With Flask support\npip install profilis[flask]\n\n# With FastAPI support\npip install profilis[fastapi]\n\n# With Sanic support\npip install profilis[sanic]\n\n# With database support\npip install profilis[flask,sqlalchemy]\n\n# With all integrations\npip install profilis[all]\n```\n\n### Option 2: Using requirements files\n\n```bash\n# Minimal setup (core only)\npip install -r requirements-minimal.txt\n\n# Flask integration\npip install -r requirements-flask.txt\n\n# SQLAlchemy integration\npip install -r requirements-sqlalchemy.txt\n\n# All integrations\npip install -r requirements-all.txt\n```\n\n### Option 3: Manual installation\n\n```bash\n# Core dependencies\npip install typing_extensions\u003e=4.0\n\n# Flask support\npip install flask[async]\u003e=3.0\n\n# FastAPI support\npip install fastapi\u003e=0.110 starlette\u003e=0.37 httpx\u003e=0.24.0\n\n# Sanic support\npip install sanic\u003e=23.0\n\n# SQLAlchemy support\npip install sqlalchemy\u003e=2.0 aiosqlite greenlet\n\n# Performance optimization\npip install orjson\u003e=3.8\n```\n\n## Quick Start\n\n### Flask Integration\n\n```python\nfrom flask import Flask\nfrom profilis.flask.adapter import ProfilisFlask\nfrom profilis.exporters.jsonl import JSONLExporter\nfrom profilis.core.async_collector import AsyncCollector\n\n# Setup exporter and collector\nexporter = JSONLExporter(dir=\"./logs\", rotate_bytes=1024*1024, rotate_secs=3600)\ncollector = AsyncCollector(exporter, queue_size=2048, batch_max=128, flush_interval=0.1)\n\n# Create Flask app and integrate Profilis\napp = Flask(__name__)\nprofilis = ProfilisFlask(\n    app,\n    collector=collector,\n    exclude_routes=[\"/health\", \"/metrics\"],\n    sample=1.0  # 100% sampling\n)\n\n@app.route('/api/users')\ndef get_users():\n    return {\"users\": [\"alice\", \"bob\"]}\n\n# Visit /_profilis for the dashboard (if you mount the UI blueprint)\nif __name__ == \"__main__\":\n    app.run(debug=True)\n```\n\n### FastAPI Integration\n\n```python\nfrom fastapi import FastAPI\nfrom profilis.fastapi.adapter import instrument_fastapi\nfrom profilis.fastapi.ui import make_ui_router\nfrom profilis.exporters.jsonl import JSONLExporter\nfrom profilis.core.async_collector import AsyncCollector\nfrom profilis.core.emitter import Emitter\nfrom profilis.core.stats import StatsStore\n\nexporter = JSONLExporter(dir=\"./logs\", rotate_bytes=1024*1024, rotate_secs=3600)\ncollector = AsyncCollector(exporter, queue_size=2048, batch_max=128, flush_interval=0.1)\nemitter = Emitter(collector)\nstats = StatsStore()\n\napp = FastAPI()\ninstrument_fastapi(app, emitter, route_excludes=[\"/profilis\"])\napp.include_router(make_ui_router(stats, prefix=\"/profilis\"))\n\n@app.get(\"/api/users\")\nasync def get_users():\n    return {\"users\": [\"alice\", \"bob\"]}\n\n# Run with: uvicorn your_module:app --reload\n# Visit http://localhost:8000/profilis for the dashboard\n```\n\n### Function Profiling\n\n```python\nfrom profilis.decorators.profile import profile_function\nfrom profilis.core.emitter import Emitter\nfrom profilis.exporters.console import ConsoleExporter\nfrom profilis.core.async_collector import AsyncCollector\n\n# Setup profiling\nexporter = ConsoleExporter(pretty=True)\ncollector = AsyncCollector(exporter, queue_size=128, flush_interval=0.2)\nemitter = Emitter(collector)\n\n@profile_function(emitter)\ndef expensive_calculation(n: int) -\u003e int:\n    \"\"\"This function will be automatically profiled.\"\"\"\n    result = sum(i * i for i in range(n))\n    return result\n\n@profile_function(emitter)\nasync def async_operation(data: list) -\u003e list:\n    \"\"\"Async functions are also supported.\"\"\"\n    processed = [item * 2 for item in data]\n    return processed\n\n# Use the profiled functions\nresult = expensive_calculation(1000)\n```\n\n### Manual Event Emission\n\n```python\nfrom profilis.core.emitter import Emitter\nfrom profilis.exporters.jsonl import JSONLExporter\nfrom profilis.core.async_collector import AsyncCollector\nfrom profilis.runtime import use_span, span_id\n\n# Setup\nexporter = JSONLExporter(dir=\"./logs\")\ncollector = AsyncCollector(exporter)\nemitter = Emitter(collector)\n\n# Create a trace context\nwith use_span(trace_id=span_id()):\n    # Emit custom events\n    emitter.emit_req(\"/api/custom\", 200, dur_ns=15000000)  # 15ms\n    emitter.emit_fn(\"custom_function\", dur_ns=5000000)      # 5ms\n    emitter.emit_db(\"SELECT * FROM users\", dur_ns=8000000, rows=100)\n\n# Close collector to flush remaining events\ncollector.close()\n```\n\n### Built-in Dashboard\n\nDashboard is available per framework:\n\n- **Flask**: `make_ui_blueprint(stats, ui_prefix=\"/_profilis\")` → `app.register_blueprint(ui_bp)`\n- **FastAPI**: `make_ui_router(stats, prefix=\"/profilis\")` → `app.include_router(router)`\n- **Sanic**: `make_ui_blueprint(stats, ui_prefix=\"/profilis\")` → `app.blueprint(bp)`\n\n```python\n# Example: Flask\nfrom flask import Flask\nfrom profilis.flask.ui import make_ui_blueprint\nfrom profilis.core.stats import StatsStore\n\napp = Flask(__name__)\nstats = StatsStore()\nui_bp = make_ui_blueprint(stats, ui_prefix=\"/_profilis\")\napp.register_blueprint(ui_bp)\n# Visit http://localhost:5000/_profilis\n```\n\n## Advanced Usage\n\n### Custom Exporters\n\n```python\nfrom profilis.core.async_collector import AsyncCollector\nfrom profilis.exporters.base import BaseExporter\n\nclass CustomExporter(BaseExporter):\n    def export(self, events: list[dict]) -\u003e None:\n        for event in events:\n            # Custom export logic\n            print(f\"Custom export: {event}\")\n\n# Use custom exporter\nexporter = CustomExporter()\ncollector = AsyncCollector(exporter)\n```\n\n### Runtime Context Management\n\n```python\nfrom profilis.runtime import use_span, span_id, get_trace_id, get_span_id\n\n# Create distributed trace context\nwith use_span(trace_id=\"trace-123\", span_id=\"span-456\"):\n    current_trace = get_trace_id()  # \"trace-123\"\n    current_span = get_span_id()    # \"span-456\"\n\n    # Nested spans inherit trace context\n    with use_span(span_id=\"span-789\"):\n        nested_span = get_span_id()  # \"span-789\"\n        parent_trace = get_trace_id() # \"trace-123\"\n```\n\n### Performance Tuning\n\n```python\nfrom profilis.core.async_collector import AsyncCollector\n\n# High-throughput configuration\ncollector = AsyncCollector(\n    exporter,\n    queue_size=8192,        # Large queue for high concurrency\n    batch_max=256,          # Larger batches for efficiency\n    flush_interval=0.05,    # More frequent flushing\n    drop_oldest=True        # Drop events under backpressure\n)\n\n# Low-latency configuration\ncollector = AsyncCollector(\n    exporter,\n    queue_size=512,         # Smaller queue for lower latency\n    batch_max=32,           # Smaller batches for faster processing\n    flush_interval=0.01,    # Very frequent flushing\n    drop_oldest=False       # Don't drop events\n)\n```\n\n## Configuration\n\n### Environment Variables\n\n```bash\n# Note: Environment variable support is planned for future releases\n# Currently, all configuration is done programmatically\n```\n\n### Sampling Strategies\n\n```python\n# Random sampling\nprofilis = ProfilisFlask(app, collector=collector, sample=0.1)  # 10% of requests\n\n# Route-based sampling\nprofilis = ProfilisFlask(\n    app,\n    collector=collector,\n    exclude_routes=[\"/health\", \"/metrics\", \"/static\"],\n    sample=1.0\n)\n```\n\n## Exporters\n\n### JSONL Exporter\n```python\nfrom profilis.exporters.jsonl import JSONLExporter\n\n# With rotation\nexporter = JSONLExporter(\n    dir=\"./logs\",\n    rotate_bytes=1024*1024,  # 1MB per file\n    rotate_secs=3600         # Rotate every hour\n)\n```\n\n### Console Exporter\n```python\nfrom profilis.exporters.console import ConsoleExporter\n\n# Pretty-printed output for development\nexporter = ConsoleExporter(pretty=True)\n\n# Compact output for production\nexporter = ConsoleExporter(pretty=False)\n```\n\n## Performance Characteristics\n\n- **Event Creation**: ≤15µs per event\n- **Memory Overhead**: ~100 bytes per event\n- **Throughput**: 100K+ events/second on modern hardware\n- **Latency**: Sub-millisecond collection overhead\n\n## Documentation\n\nFull documentation is available at: [Profilis Docs](https://ankan97dutta.github.io/profilis/)\n\nDocs are written in Markdown under [`docs/`](./docs) and built with [MkDocs Material](https://squidfunk.github.io/mkdocs-material/).\n\n### Available Documentation\n\n- **[Getting Started](https://ankan97dutta.github.io/profilis/guides/getting-started/)** - Quick setup and basic usage\n- **[Configuration](https://ankan97dutta.github.io/profilis/guides/configuration/)** - Tuning and customization\n- **[Flask Integration](https://ankan97dutta.github.io/profilis/adapters/flask/)** - Flask adapter\n- **[FastAPI Integration](https://ankan97dutta.github.io/profilis/adapters/fastapi/)** - FastAPI/ASGI adapter\n- **[Sanic Integration](https://ankan97dutta.github.io/profilis/adapters/sanic/)** - Sanic adapter\n- **[SQLAlchemy Support](https://ankan97dutta.github.io/profilis/databases/sqlalchemy/)** - Database instrumentation\n- **[MongoDB](https://ankan97dutta.github.io/profilis/databases/mongodb/) · [Neo4j](https://ankan97dutta.github.io/profilis/databases/neo4j/) · [pyodbc](https://ankan97dutta.github.io/profilis/databases/pyodbc/)** - Additional databases\n- **[JSONL Exporter](https://ankan97dutta.github.io/profilis/exporters/jsonl/)** - Log file output\n- **[Built-in UI](https://ankan97dutta.github.io/profilis/ui/ui/)** - Dashboard documentation\n- **[Architecture](https://ankan97dutta.github.io/profilis/architecture/architecture/)** - System design\n\nTo preview locally:\n```bash\npip install mkdocs mkdocs-material mkdocs-mermaid2-plugin\nmkdocs serve\n```\n\n## Development\n\n### Setting up the project\n\n1. **Clone and enter the repo**\n   ```bash\n   git clone https://github.com/ankan97dutta/profilis.git\n   cd profilis\n   ```\n\n2. **Create a virtual environment and install in editable mode with dev dependencies**\n   ```bash\n   python -m venv .venv\n   source .venv/bin/activate   # Windows: .venv\\Scripts\\activate\n   pip install -e \".[dev]\"\n   ```\n\n3. **Install pre-commit hooks** (optional but recommended)\n   ```bash\n   pre-commit install\n   ```\n\n4. **Run the test suite**\n   ```bash\n   pytest\n   ```\n\n   Use `pytest -v` for verbose output, `pytest path/to/test_file.py` to run a single file, or `pytest -k \"test_name\"` to run tests matching a pattern. Coverage: `pytest --cov=profilis --cov-report=term-missing`.\n\n### Working with TDD\n\nWe encourage **test-driven development (TDD)**:\n\n1. **Red** — Write a failing test that describes the behaviour you want.\n2. **Green** — Implement the minimum code to make the test pass.\n3. **Refactor** — Improve the implementation while keeping tests green.\n\nRun tests frequently (e.g. `pytest` or `pytest tests/ -q`) as you work. See [Development Guidelines](./docs/meta/development-guidelines.md#test-driven-development-tdd) for the full TDD workflow and test layout.\n\n### Branching and commits\n\n- See [Contributing](./docs/meta/contributing.md) and [Development Guidelines](./docs/meta/development-guidelines.md).\n- Branch strategy: trunk‑based (`feat/*`, `fix/*`, `perf/*`, `chore/*`).\n- Commits follow [Conventional Commits](https://www.conventionalcommits.org/).\n\n## Roadmap\n\nSee [Profilis – v0 Roadmap Project](https://github.com/ankan97dutta/profilis/projects) and [`docs/overview/roadmap.md`](./docs/overview/roadmap.md).\n\n## License\n\n[MIT](./LICENSE)\n\n## Contact\n\n- **Email**: [connect@ankandutta.in](mailto:connect@ankandutta.in)\n- **Website**: [https://www.ankandutta.in](https://www.ankandutta.in)\n- **Blog**: [Signals \u0026 Noise](https://blog.ankandutta.in/)\n- **GitHub**: [@ankan97dutta](https://github.com/ankan97dutta)\n\nFeel free to reach out if you have questions, suggestions, or would like to contribute to Profilis!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankan97dutta%2Fprofilis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fankan97dutta%2Fprofilis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fankan97dutta%2Fprofilis/lists"}