{"id":27938826,"url":"https://github.com/aamir-hullur/script-visualizer","last_synced_at":"2025-05-07T08:58:00.467Z","repository":{"id":287479815,"uuid":"964872998","full_name":"Aamir-Hullur/script-visualizer","owner":"Aamir-Hullur","description":"A language-agnostic visualization web application for generating data visualizations from Python and R scripts in isolated Docker containers.","archived":false,"fork":false,"pushed_at":"2025-04-12T14:41:56.000Z","size":55580,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-07T08:57:56.071Z","etag":null,"topics":["docker","fastapi","python","r","react","shadcn","tailwindcss","typescript","vite"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Aamir-Hullur.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-12T00:02:25.000Z","updated_at":"2025-04-15T20:02:47.000Z","dependencies_parsed_at":"2025-04-12T01:22:39.720Z","dependency_job_id":"5400b62d-10b4-437e-8c24-75f86e8fec0e","html_url":"https://github.com/Aamir-Hullur/script-visualizer","commit_stats":null,"previous_names":["aamir-hullur/script-visualizer"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aamir-Hullur%2Fscript-visualizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aamir-Hullur%2Fscript-visualizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aamir-Hullur%2Fscript-visualizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Aamir-Hullur%2Fscript-visualizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Aamir-Hullur","download_url":"https://codeload.github.com/Aamir-Hullur/script-visualizer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252847504,"owners_count":21813450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","fastapi","python","r","react","shadcn","tailwindcss","typescript","vite"],"created_at":"2025-05-07T08:57:59.893Z","updated_at":"2025-05-07T08:58:00.457Z","avatar_url":"https://github.com/Aamir-Hullur.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Script Visualizer\n\nA web application that allows users to generate data visualizations from Python and R scripts. This project emphasizes performance, and user experience by executing code in isolated docker containers and providing a rich development environment. \n\n## Demo\n\n\u003cdiv align=\"center\"\u003e\n   \u003cimg src=\"https://raw.githubusercontent.com/Aamir-Hullur/script-visualizer/refs/heads/main/assets/Demo.gif\" width=\"800\" alt=\"Demo Video\"\u003e\n\u003c/div\u003e\n\n## Tech Stack\n\n### Frontend\n- **React with Vite**: For lightning-fast development and optimal production builds\n- **TypeScript**: For type safety and better developer experience\n- **Monaco Editor**: VS Code-like script editing experience\n- **ShadCn UI**: Modern, accessible UI components\n- **Tailwind CSS**: Utility-first styling\n\n### Backend\n- **FastAPI**: High-performance Python web framework\n- **Docker**: For secure script execution in isolated containers\n- **Python Libraries**: matplotlib, plotly\n- **R Libraries**: ggplot2\n\n## Key Features\n\n1. **Secure Code Execution**:\n   - Scripts run in isolated Docker containers\n   - Resource limits (CPU, memory, timeout)\n   - Network access restrictions\n   - Automated container cleanup after execution\n\n2. **Rich Development Experience**:\n   - Syntax highlighting for Python and R\n   - Code examples for different visualization types\n   - Real-time preview\n\n3. **Multiple Visualization Types**:\n   - Static plots (PNG output)\n   - Interactive visualizations (HTML output)\n   - 3D visualizations (via plotly)\n\n4. **Modern UI/UX**:\n   - Dark/light theme support\n   - Resizable panels\n   - Copy-to-clipboard functionality\n   - Error handling with user-friendly messages\n\n## Development Challenges \u0026 Solutions\n\n#### 1. **Secure Execution of Dynamic Code**\n**Issue:**  \nExecuting arbitrary Python and R scripts from user input posed a significant security risk, including the potential for code injection or resource exhaustion.\n\n**Resolution:**  \nTo mitigate this:\n- A sandboxed environment (Docker containers) was implemented to isolate script execution.\n- Resource limits (CPU/memory/timeouts) were set on containers to prevent abuse.\n- Only specific packages required for visualization (e.g., `matplotlib`, `plotly`, `ggplot2`) were pre-installed to limit what scripts could access.\n\n\n#### 2. **Rendering and Serving Visualizations**\n**Issue:**  \nReturning visualizations from backend execution to the frontend required supporting multiple formats (e.g., static PNG, interactive HTML).\n\n**Resolution:**  \n- For **static charts**, image files (PNG) were saved and served via a public endpoint.\n- For **interactive charts**, the backend captured and returned full HTML snippets (e.g., from Plotly or R’s htmlwidgets), which were embedded in an `\u003ciframe\u003e` on the frontend.\n\n\n#### 3. **Error Handling and Debugging**\n**Issue:**  \nErrors in user-submitted code were initially hard to trace and led to silent failures, making it difficult for users to understand what went wrong.\n\n**Resolution:**  \n- Integrated **Monaco Editor** on the frontend to enhance the code editing experience with syntax highlighting and error hints.\n- Improved backend logging to capture and return detailed error messages (e.g., syntax errors, missing libraries).\n\n\n#### 4. **Docker Image Build Time and Size**\n**Issue:**  \nThe initial Docker images — particularly for the Python environment — were large and slow to build, which negatively impacted development speed and deployment readiness.\n\n**Resolution:**  \n- Switched to the `python:3.11-slim` base image to significantly reduce the Python container size.\n- Installed only the essential visualization libraries (`matplotlib`, `plotly`) to avoid unnecessary dependencies.\n- Applied Docker layer caching and cleanup (e.g., using `--no-cache-dir` with pip) to speed up rebuilds and reduce final image size.\n- Used a similar minimal setup for the R container to keep both environments lean and efficient.\n\n\n## Environment Configuration\n\n### Backend (.env)\n```\nAPI_V1_STR=/api/v1\nDOCKER_CONTAINER_TIMEOUT=300\nDOCKER_MEMORY_LIMIT=512m\nDOCKER_CPU_LIMIT=0.5\nDOCKER_NETWORK_ACCESS=false\n```\n\n### Frontend (.env)\n```\nVITE_BACKEND_URL=http://localhost:8000\n```\n\n## Getting Started\n\n1. **Clone the repository**\n```bash\ngit clone https://github.com/Aamir-Hullur/script-visualizer.git\n```\n\n2. **Backend Setup**\n```bash\ncd backend\npython -m venv venv\nsource venv/bin/activate  # or `venv\\Scripts\\activate` on Windows\npip install -r requirements.txt\n\n# Build Docker images for Python and R visualization environments\ncd docker\nchmod +x build_images.sh\n./build_images.sh\n```\n\n3. **Frontend Setup**\n```bash\ncd frontend\nnpm install\n```\n\n4. **Start Development Servers**\n```bash\n# Terminal 1 - Backend\ncd backend\nuvicorn app.main:app --reload\n\n# Terminal 2 - Frontend\ncd frontend\nnpm run dev\n```\n\n5. **Access the application**\n```\nFrontend: http://localhost:5173\nBackend API: http://localhost:8000/docs\n```\n\n## Example Visualizations\n\n### Python with Matplotlib (Static)\n```python\nimport matplotlib.pyplot as plt\nimport numpy as np\n\n# Generate data\nx = np.linspace(0, 10, 100)\ny = np.sin(x)\n\n# Create visualization\nplt.figure(figsize=(8, 6))\nplt.plot(x, y, 'b-', linewidth=2)\nplt.title('Sine Wave')\nplt.xlabel('X axis')\nplt.ylabel('Y axis')\nplt.grid(True)\nplt.savefig('/output/plot.png')  # Save to mounted output directory\n```\n\n### Python with Plotly (Interactive)\n```python\nimport plotly.express as px\nimport pandas as pd\n\n# Create sample data\ndata = {\n    'Category': ['A', 'B', 'C', 'D'],\n    'Values': [10, 40, 30, 20]\n}\ndf = pd.DataFrame(data)\n\n# Create interactive visualization\nfig = px.bar(df, x='Category', y='Values', title='Interactive Bar Chart')\nfig.write_html('/output/plot.html')  # Save as interactive HTML\n```\n\n### R with ggplot2 (Static)\n```r\nlibrary(ggplot2)\n\n# Create sample data\ndata \u003c- data.frame(\n  x = 1:10,\n  y = 1:10\n)\n\n# Create visualization\np \u003c- ggplot(data, aes(x, y)) +\n  geom_point() +\n  geom_smooth(method = \"lm\") +\n  theme_minimal() +\n  labs(title = \"Linear Relationship\")\n\n# Save to output directory\nggsave(\"/output/plot.png\", p)\n```\n\n## System Architecture\n\n```mermaid\narchitecture-beta\n    group frontend(cloud)[Frontend]\n        service reactApp(server)[React App] in frontend\n        service monacoEditor(database)[Monaco Editor] in frontend\n        service ui(internet)[ShadCn UI] in frontend\n\n    group backend(cloud)[Backend]\n        service fastapi(server)[FastAPI Server] in backend\n        service docker(server)[Docker Container] in backend\n        service storage(disk)[Storage] in backend\n\n    reactApp:R -- L:monacoEditor\n    monacoEditor:R -- L:ui\n    reactApp:B -- T:fastapi\n    fastapi:R -- L:docker\n    docker:B -- T:storage\n```\n\n## Visualization Flow\n\n```mermaid\nflowchart TD\n    A[Code Editor Input] --\u003e B{Language Selection}\n    B --\u003e|Python| C[Python Container]\n    B --\u003e|R| D[R Container]\n    \n    C --\u003e E[Generate Visualization]\n    D --\u003e E\n    \n    E --\u003e F{Visualization Type}\n    F --\u003e|Static| G[Save as PNG]\n    F --\u003e|Interactive| H[Generate HTML]\n    \n    G --\u003e I[Display Output]\n    H --\u003e I\n```\n\n## Future Implementations\n\n1. Support for more programming languages\n2. Additional visualization libraries\n3. Collaborative features\n4. User accounts and saved visualizations\n5. Adding a download button","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faamir-hullur%2Fscript-visualizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faamir-hullur%2Fscript-visualizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faamir-hullur%2Fscript-visualizer/lists"}