{"id":31946742,"url":"https://github.com/deadflowers/bitcurrents-log-analyzer","last_synced_at":"2026-02-19T01:31:56.261Z","repository":{"id":317487155,"uuid":"1067612827","full_name":"deadflowers/bitcurrents-log-analyzer","owner":"deadflowers","description":"Web traffic log analyzer featuring IP address insights provided by IP2Location API. Concept is created as a entry for their 2025 Hackathon programming contest.","archived":false,"fork":false,"pushed_at":"2025-10-10T04:07:13.000Z","size":8015,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-23T11:59:48.141Z","etag":null,"topics":["analytics","geolocation-api","ip2location","ip2locationcontest","nginx","programmingcontest","python","security-tools","traffic-analysis"],"latest_commit_sha":null,"homepage":"https://bitcurrents.com/log-analyzer/","language":"HTML","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deadflowers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"security_patterns.json","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-01T05:54:33.000Z","updated_at":"2025-10-20T19:33:39.000Z","dependencies_parsed_at":"2025-10-01T08:33:48.760Z","dependency_job_id":"83f63fe1-4719-4afe-81b3-4440e5f980b1","html_url":"https://github.com/deadflowers/bitcurrents-log-analyzer","commit_stats":null,"previous_names":["deadflowers/bitcurrents-log-analyzer"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/deadflowers/bitcurrents-log-analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deadflowers%2Fbitcurrents-log-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deadflowers%2Fbitcurrents-log-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deadflowers%2Fbitcurrents-log-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deadflowers%2Fbitcurrents-log-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deadflowers","download_url":"https://codeload.github.com/deadflowers/bitcurrents-log-analyzer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deadflowers%2Fbitcurrents-log-analyzer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29600752,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-19T00:59:38.239Z","status":"ssl_error","status_checked_at":"2026-02-19T00:59:36.936Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","geolocation-api","ip2location","ip2locationcontest","nginx","programmingcontest","python","security-tools","traffic-analysis"],"created_at":"2025-10-14T11:22:51.720Z","updated_at":"2026-02-19T01:31:56.220Z","avatar_url":"https://github.com/deadflowers.png","language":"HTML","readme":"# BitCurrents Log Analyzer\n\n**A powerful, real-time Nginx log analyzer with a focus on security intelligence, performance monitoring, and beautiful data visualization. Powered by the IP2Location API.**\n\n[![Watch the demo](https://img.youtube.com/vi/_bsR1jk-Mas/0.jpg)](https://youtu.be/_bsR1jk-Mas?si=I0kenWTKE6MPDkoP)\n\nBitCurrents Analyzer turns raw Nginx logs into actionable insights. It moves beyond simple IP lookups to provide a rich, interactive dashboard that helps you understand your traffic, identify security threats, and diagnose server errors instantly. It's extremely lightweight as single-page web application, built with a Python/Flask backend and dynamic JavaScript frontend, private, nimble, customizable.\n\n---\n\n**[Log Analyzer LIVE demo](https://bitcurrents.com/log-analyzer/index.html)**\n\nFollow link, List Button, select file, Analyze Button, wait..., peruse all the things ;) \n\n---\n\n## 🔥 Key Features\n\n### 1. Multi-Mode Analysis Dashboard\nSwitch between three distinct analysis modes, each with a purpose-built dashboard:\n\n#### ✨ General Mode\nYour command center for at-a-glance traffic overview.\n-   **Rich KPI Cards:** Total Hits, Unique IPs, and critical counts of Impersonator and Suspicious traffic.\n-   **Comprehensive Charting:**\n    -   Traffic Breakdown (Human, Bots, Crawlers)\n    -   Device Types (Desktop, Mobile)\n    -   OS \u0026 Browser Distribution\n    -   HTTP Status Code Distribution\n    -   Top Countries \u0026 Top Successful Pages\n-   **Powerful Filtering:** Instantly hide noisy IPs to see the real signal, and switch between analyzing Unique IPs vs. All Hits.\n\n*![Main Dash](https://github.com/user-attachments/assets/947d71b1-dd58-4d06-9ef9-b813255c7cfa)*\n\n\n#### 🛡️ Security Mode\nProactively hunt for threats and analyze attack patterns.\n-   **Dynamic Threat Categories:** Filter logs for specific attack vectors like Config Exposure, Webshells, and WordPress probes using a configurable `security_patterns.json`.\n-   **Targeted Visualizations:** See charts for Top Attack Categories and Top Attacking IPs.\n-   **Bot Impersonator Detection:** Automatically flags traffic pretending to be a legitimate crawler from an unverified IP address.\n\n*![Security Dash](https://github.com/user-attachments/assets/89bb7272-8155-4888-a338-e7a2716f812c)*\n\n#### 🚨 Error Mode\nQuickly diagnose and understand server problems.\n-   **Smart Error Parsing:** Intelligently categorizes common Nginx `error.log` messages (e.g., \"Backend Unreachable,\" \"Index Forbidden\").\n-   **Insightful Charts:** Visualize error types and severity levels to prioritize fixes.\n-   **In-App Quick Guide:** A helpful \"Error Log Tips\" modal provides context and troubleshooting advice for common Nginx errors.\n\n*![IP Panel](https://github.com/user-attachments/assets/181f9579-e045-4437-995f-e57df7dddf46)*\n\n---\n\n### 👨‍💻 Hackathon Submission built for the IP2Location Programming Contest\n\nThis project leverages the power and precision of the **[IP2Location.io](https://www.ip2location.io/)** API to deliver advanced security features, including:\n-   **High-Fidelity Geolocation:** Pinpointing the source of traffic with country, city, and ISP data.\n-   **Intelligent Bot Detection:** Moving beyond simple user-agent strings.\n-   **Bot Impersonator Flagging:** A critical security feature that verifies if traffic claiming to be from major crawlers (like Googlebot) is actually coming from their published IP ranges.\n\n\n### Set Up: \n\nClone this repository.\n\n*   **Install pyenv:** If you don't have it, follow the installation guide. For macOS/Linux with Homebrew, it looks like this:\n    \n        `brew install pyenv`\n    \nMake sure to follow the post-install instructions to add pyenv init to your shell's startup script (.zshrc, .profile, .bashrc, etc.).\n\nThen add these lines to your shell config (~/.bashrc or ~/.zshrc):\n\n    ```\n    export PATH=\"$HOME/.pyenv/bin:$PATH\"\n    eval \"$(pyenv init -)\"\n    eval \"$(pyenv virtualenv-init -)\"\n    ```\n\nRestart your shell so changes take effect.\n    \n*   **Install a Python Version:** Let's use a recent, stable version.\n    \n        `pyenv install 3.13.0`\n      \n*   **Create Your Project:**\n    \n        `git clone https://github.com/deadflowers/bitcurrents-log-analyzer`\n    \n        `cd bitcurrents-log-analyzer`\n      \n*   **Set the Local Python Version:** This command creates a .python-version file in your directory, so pyenv automatically uses this version whenever you're in this folder.\n    \n        `pyenv local 3.13.0`\n      \n*   **Create \u0026 Activate a Virtual Environment:** This is a best practice within your pyenv version to keep packages project-specific.\n    \n     Create the virtual environment folder named 'venv'\n    `python -m venv .venv`\n    \n     Activate it\n    \n    `source .venv/bin/activate`\n\n     Your terminal prompt should have in it `(.venv)`\n      \n*   **Install Dependencies:**\n    \n        `pip install -r requirements.txt`\n      \n*   **Set Up API Key:**\n    \n    *   Sign up for a free account at [IP2Location.io](https://ip2location.io) to get an API key. The free plan is quite capable.\n        \n    *   Create a file named .env in your project directory and add your key:\n           \n         `IP2LOCATION_API_KEY=your-key`\n        \n        or\n        \n         `export IP2LOCATION_API_KEY=your-key`\n\nMake sure your .venv is activated.\n    \n*   **Run it:** In your terminal, inside the bitcurrents-log-analyzer directory, start the backend, and the front end\n  \n       `source .venv/bin/activate`\n\n### Start Backend: Run the Python server:\n\n```bash\npython app.py\n```\n\n### Launch Frontend: \nOpen `index.html` in your web browser as `file:///` path\n\n### Analyze:\nThe dashboard will load with the new dark theme. \n\n- Enter the directory where your logs are stored or copied to. Default is `.`\n- Hit \"List\" button to see log files in that directory\n- Optionally, enter a path fragment to track in the \"Track Custom Path\"\n- Click the \"Analyze Logs\" button and wait a moment.\n\nYou can use sample logs provided in './sample_logs' in the project folder, where you will find `combined.log`. Just entr `.` as directory to search logs then choose in the next area specific file `combined.log`\n\nThe charts will populate, and the detailed IP table will appear below. You can now distinguish OS/Bot and any suspicious paths hit directly in the table.\n- Add new paths to your watchlist on-the-fly using the \"Add to Suspicious List\" feature.\n\nBeing conservative with credits we keep an ip cache file as well automatically.\n\n### NGINX\n\n`combined.log` is a quick dump of nginx access logs. It is provided in the app as data to test. If you are not having luck when running your own, check your nginx config:\n\nTo display the referrer in Nginx access logs, ensure your `log_format` configuration includes the `$http_referer` variable.\n\nI have in my `nginx.conf` the following:\n\n```\n# Logging\n    log_format custom_format '$remote_addr - $remote_user [$time_local] '\n                          '\"$request\" $status $body_bytes_sent '\n                          '\"$http_referer\" \"$http_user_agent\"';\n    access_log\t\t      /var/log/nginx/access.log custom_format;\n    error_log              /var/log/nginx/error.log warn;\n```\n\nThen in my domain conf:\n\n```\n# logging\n    access_log              /var/log/nginx/site.com.access.log custom_format buffer=512k flush=1m;\n    error_log               /var/log/nginx/site.com.error.log warn;\n```\n\nReload\n\n```\nsudo systemctl reload nginx\n```\n\nThanks for checking it out, let me know what you think. We look forward to future iterations and new features.\n\n#IP2LocationContest #ProgrammingContest\n\nLog Analyzer concept by Ray Kooyenga\n\n@deadflowers @rkooyenga\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeadflowers%2Fbitcurrents-log-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeadflowers%2Fbitcurrents-log-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeadflowers%2Fbitcurrents-log-analyzer/lists"}