{"id":24842435,"url":"https://github.com/coulisse/spiderweb","last_synced_at":"2025-10-14T15:30:19.489Z","repository":{"id":40955904,"uuid":"235598863","full_name":"coulisse/spiderweb","owner":"coulisse","description":"Web Interface for DXSpider hamradio cluster","archived":false,"fork":false,"pushed_at":"2025-06-29T12:00:29.000Z","size":5552,"stargazers_count":33,"open_issues_count":15,"forks_count":14,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-09-04T21:48:15.648Z","etag":null,"topics":["bootstrap","cluster","dx","dx-cluster","dxcluster","echarts","flask","ham","hamradio","mysql","plots","python","qrz","radio","spiderweb"],"latest_commit_sha":null,"homepage":"https://www.iu1bow.it","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/coulisse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-01-22T15:09:22.000Z","updated_at":"2025-08-21T12:14:33.000Z","dependencies_parsed_at":"2024-03-10T08:25:39.118Z","dependency_job_id":"10362991-ece6-4f2a-9500-158bef7dbb04","html_url":"https://github.com/coulisse/spiderweb","commit_stats":null,"previous_names":[],"tags_count":31,"template":false,"template_full_name":null,"purl":"pkg:github/coulisse/spiderweb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coulisse%2Fspiderweb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coulisse%2Fspiderweb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coulisse%2Fspiderweb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coulisse%2Fspiderweb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/coulisse","download_url":"https://codeload.github.com/coulisse/spiderweb/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/coulisse%2Fspiderweb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279019321,"owners_count":26086711,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bootstrap","cluster","dx","dx-cluster","dxcluster","echarts","flask","ham","hamradio","mysql","plots","python","qrz","radio","spiderweb"],"created_at":"2025-01-31T08:01:55.662Z","updated_at":"2025-10-14T15:30:19.483Z","avatar_url":"https://github.com/coulisse.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003cimg  align=\"center\"  src=\"static/images/icons/icon-72x72.png\"/\u003e SPIDERWEB\n===\n\n### Ham radio cluster web viewer for DxSpider\n[![GPLv3 license](https://img.shields.io/badge/License-GPLv3-blue.svg)](http://perso.crans.org/besson/LICENSE.html)\n[![made-with-python](https://img.shields.io/badge/B.e.-Python-1f425f.svg)](https://www.python.org/)\n[![made-with-javascript](https://img.shields.io/badge/F.e.-JavaScript-yellow)](https://www.javascript.com)\n[![bootstrap](https://img.shields.io/badge/CSS%20Fwk-Bootstrap-purple)](https://getbootstrap.com/)\n[![CodeFactor](https://www.codefactor.io/repository/github/coulisse/spiderweb/badge/development)](https://www.codefactor.io/repository/github/coulisse/spiderweb/overview/development)\n\n\n- **Release:** v2.5.8\n- **Author:** Corrado Gerbaldo - [IU1BOW](https://www.qrz.com/db/IU1BOW)\n- **Mail:** \u003ccorrado.gerbaldo@gmail.com\u003e\n- **Licensing:** Gpl V3.0 see [LICENSE](LICENSE) file.\n- **Languages:** This application is written in Python 3.13/flask,Javascript and HTML\n\n___\n**DXSpider** is a great DX Cluster software that has useful telnet interface. \nI wrote this application in order to add a web user interface to DXSpider and show the spots collected.\nThe user could see 50 spots at time and filter them by band, spotter continent and spotted continent.\n\n**NOTE:**   starting from this release it will works only from Dxspider version V1.57 build 560 or greater.\n\nFor this application, I've used:\n- **Bootstrap** for stylesheet CSS\n- **Apache ECharts** for managing charts\n- **qrz.com** For each callsing, found you can click on lens and you'll see him on qrz.com\n- **flag-icon-css** [https://github.com/lipis/flag-icon-css](https://github.com/lipis/flag-icon-css) I used it to show the country flags \n- **ng3k.com** [ng3k.com](http://ng3k.com/misc/adxo.html) I used to get information about \"Announced Dx Operations\". Thanks to Bill/NG3K !!!\n- **silso** [sidc.be/silso](https://sidc.be/silso/) used to show propagation trend in \"Chart \u0026 stats\" section\n- **hamqsl** [www.hamqsl.com/solar.html](https://www.hamqsl.com/solar.html/) used to show solar and band conditions\n- **kc2g** [prop.kc2g.com](https://prop.kc2g.com/) used to show MUF map\n\n\nYou can find my web site at [https://www.iu1bow.it](https://www.i1bow.it)\n\n### Changelog\nsee file [\"CHANGELOG.md\"](docs/CHANGELOG.md)\n\n### Install            \n\n**1) DXSpider**\nFirst of all you have to installed [DXspider](http://www.dxcluster.org/) and connected with some other cluster nodes.\n\n**2) MariaDB / MySQL**\nThen you have to install MariaDB on your web server, on the same server where DXSpider is running and configure DXSpider to use it: in your spider folder edit `local/DXVars.pm` adding these lines:\n```DXWars.pm\n# the SQL database DBI dsn\n$dsn = \"dbi:mysql:dxcluster:localhost:3306\";\n$dbuser = \"your-user\";\n$dbpass = \"your-password\"; \n```\nIf you would change some MariaDB parameters, then you can find them in  `/etc/mysql/my.cnf` or `/etc/my.cnf`, depending on your distro.\nIf the database will not be created automatically, please see [\"DB_ISSUES.md\"](docs/DB_ISSUES.md) \n\nIs it possible that have also to install mariadb libraries. For Ubuntu:\n\n```console\n foo@bar:~$ sudo apt-get install -y libmariadb-dev             \n```\n\n\n**3) Python modules**  \nYou could install python modules using automatic or manual way.\n\n*3.1) Automatic modules install*  \nafter downloaded spiderweb move in the main folder and install using requirements.txt file\n```console\n foo@bar:~$ cd spiderweb                                \n foo@bar:~$ pip install -r requirements.txt\n```\n*3.2) Manual modules install*  \nFirst of all you have to install the python3 pip installer\n```console\nfoo@bar:~$ sudo apt install python3-pip\n```\nThis application is based on Flask \nTo install **Flask**:\n```console\nfoo@bar:~$ pip install flask \nfoo@bar:~$ pip install Flask-minify\nfoo@bar:~$ pip install flask_wtf\nfoo@bar:~$ pip install pandas\n```\n\n### Configuration  \n\nIn the path `spiderweb/cfg/` rename `config.json.template` in `config.json`:\n```console\nfoo@bar:~$ mv config.json.template config.json\n```\nthen edit it and set the user and password of your database, the menu items, and other stuffs (callsign, mail address...).\nThere is also a specific parameter, named \"enable_cq_filter\" used to enable the CQ Zone filtering.\n\nOthewhise, if you prefer, you could use a utility to edit your configuration and menu. Go in \"script\" folder and run ./config.sh\n\n```console\nfoo@bar:~$ cd scripts\nfoo@bar:~$ ./config.sh\n\n*** DxSpider configuration ***\nConfiguration file loaded from: ../cfg/config.json\n\n   h:  help\n   vc: view config.\n   ec: edit config.\n   vm: view menu\n   em: edit menu\n   s:  save\n   t:  load config. from template\n\n   x:  exit\n\nMake your choice: \n\n```\n\n**Telnet** (for the list of connected users/nodes)\n\nWith other necessary settings set up your spiderweb *telnet_user* Callsign and *telnet_password* in /cfg/config.json\n```console\n\"telnet_host\": \"mysite\",  \n\"telnet_port\": \"7300\",  \n\"telnet_user\": \"myuser_call\",  \n\"telnet_password\": \"myuser_call_password\"  \n```   \nRestart of the spiderweb application is needed for the changes to take effect.  \n\n**For a user to be assigned a password**, the SYSOP needs to use the command bellow **on the DX Spider Cluster**.   \nPassword is not propagated across the network for security reasons. Use the command without brackets with a strong password e.g. SET/PASSWORD AB1CD 1234  \n```console\nSET/PASSWORD \u003cmyuser_call\u003e \u003cmyuser_call_password\u003e\n```  \nIf we isolate our spiderweb user it will be able to connect to Node but it will not broadcast anything to the network, neither PC92 nor spots or ann. Everything will stay on our node and it will show up as a connected user. Use the command without brackets e.g. SET/ISOLATE AB1CD  \n\n```console\nSET/ISOLATE \u003cmyuser_call\u003e\n```\n\nMaybe on this occasion we set up some basic user data with SPOOF command. This command allows SYSOP to pretend that you are doing the command as the user you specify. Use your own information for \"myuser_call\" e.g. SPOOF AB1CD SET/NAME Spiderweb  \n```console\nSPOOF \u003cmyuser_call\u003e SET/NAME Spiderweb  \nSPOOF \u003cmyuser_call\u003e SET/QTH Cerkno  \nSPOOF \u003cmyuser_call\u003e SET/QRA JN66XD  \nSPOOF \u003cmyuser_call\u003e SET/HOME S50CLX  \n```\n\nIf for some reason the password for the spiderweb user on the DX Cluster Node needs to be changed, do so with the command:  \n```console\nSPOOF \u003cmyuser_call\u003e SET/PASSWORD  \n```\nTo change the password you must enter old and a new password.  \n```console\nEnter old password: \nEnter new password:  \nRe-enter new password:  \nPassword changed  \n```\n\n### Crontab\nStarting from version 2.4, since all activities are managed by the application, you don't need to schedule anything.\n\n\n### Run test\nNow, you can run your web application with the following command:\n```console\nfoo@bar:~$ python3 webapp.py\n```\nThe flask default port is 5000, so you can see your web app, typing `http://localhost:5000` in your web browser.\nKeep in mind that the flask web server, usually is used as a test server.\n\n### Production\nThere are some ways to use it in production. \n\nMy configuration is:\nCloudflare + Nginx + Bjoern\n\n**Bjoern is a lightweight WSGI for python.**\n\nfor installing it: \n```console\nfoo@bar:~$ sudo apt install libev-dev libevdev2\nfoo@bar:~$ pip3 install bjoern\n```\n\nIf you want you can make it as a _daemon service_.\nCreate and edit a file named for example spiderweb.service (in the systemd folder)\n\n```console\nfoo@bar:~$ sudo nano /etc/systemd/system/spiderweb.service\n```\nBelow an example of configuration, keep in mind that it is just an example (you have to put your data for your application [here a little guide](https://linuxhandbook.com/create-systemd-services/)   ):\n```console\n[Unit]\nDescription=bjoern instance spiderweb\nAfter=network.target\nAfter=multi-user.target\n\n[Service]\nUser=web\nGroup=www-data\nType=simple\nWorkingDirectory=/home/web/spiderweb\nEnvironment=\"PATH=/home/web/spiderweb\"\nExecStart=/usr/bin/python3 /home/web/spiderweb/wsgi.py\n\n[Install]\nWantedBy=multi-user.target\n```\n\nThen you can install and start the daemon:\n```console\nfoo@bar:~$ sudo systemctl enable spiderweb.service\nfoo@bar:~$ sudo systemctl start spiderweb.service\nfoo@bar:~$ sudo systemctl status spiderweb.service\n\n● spiderweb.service - bjoern instance spiderweb\n   Loaded: loaded (/etc/systemd/system/spiderweb.service; enabled; vendor preset: enabled)\n   Active: active (running) since Sun 2020-10-25 09:56:35 UTC; 8h ago\n Main PID: 6518 (python3)\n    Tasks: 1 (limit: 420)\n   CGroup: /system.slice/spiderweb.service\n           └─6518 /usr/bin/python3 /home/web/spiderweb/wsgi.py\n\nOct 25 09:56:35 dxcluster01 systemd[1]: Started bjoern instance spiderweb.\n```\nNow you can install and configure **NGINX**\n\nInstall with:  \n```console\nfoo@bar:~$ sudo apt install nginx\n```\n\nConfigure:\n```console\nsudo nano /etc/nginx/sites-available/myapp\n```\n\n```console\nserver {\n    listen 80;\n    server_name iu1bow.it www.iu1bow.it;\n    location ^~ /.well-known/ {\n      alias /home/web/verify/.well-known/;\n    }\n\n    location / {\n        ssi off;\n        include proxy_params;\n        proxy_pass http://localhost:8080/;\n        proxy_set_header Host $host;\n    }\n}\n\n```\n\nFor SSL, I'm using [Cloudflare](https://www.cloudflare.com/). This is a free service that allow you to use https and a proxy cache. \n\n\n**Search engine indexing:** when you are on-line, if you would to index your website on search engines, you have to generate a file named sitemap.xml and put it in /static/ folder. There are many tools to generate sitemap.xml, for example [www.xml-sitemaps.com](https://www.xml-sitemaps.com/)\n\n**Index on MySQL:** if you would to increase speed on callsign search, you could define some index on the table 'spot'. You can see more details on [\"create_mysql_index.sql\"](scripts/create_mysql_index.sql)\n\n\n### Mobile\nThis application is designed for desktop and mobile phone. It is a [PWA](https://en.wikipedia.org/wiki/Progressive_web_app), so it could be installed and used like an app on mobile.\n\n### API\n**Spot list**\n\nYou can retrieve last spots calling \"**/spotlist**\"; For example  [www.iu1bow.it/spotlist](https://www.iu1bow.it/spotlist)\n\n**Country of a callsign**\n\nYou can retrive some information about a callsign with **callsign**; for example:  [www.iu1bow.it/callsign?c=IU1BOW](https://www.iu1bow.it/callsign?c=IU1BOW)\n\n\n### Development\n**Directory structure**\n```\n/                 . main application files\n├── cfg           . configuration files (put here your config.json with your setting)\n|── data          . data folder where all running information will be put (ie cty_wt_mod.dat, or visit.json)\n├── docs          . documentation\n├── lib           . python libs used for the application\n├── log           . application log\n├── scripts       . utility scripts for testing, build etc.\n├── static        . static files css, js, data, html, images etc.\n│   ├── css       .\n│   │   ├── dev   . development css not minifyed/uglifyed\n│   │   └── rel   . release css minifyed/uglifyed  (do not change these files)\n│   ├── data      . application data (world.json)\n│   ├── html      .\n│   │   └── dev   . html templates used for build release static html (for offline)\n│   │   └── rel   . release static html (for offline)\n│   ├── images    . static images\n│   │   └── icons . static icons\n│   └── js        .\n│       ├── dev   . development js not minifyed/uglifyed\n│       └── rel   . release js minifyed/uglifyed  (do not change these files)\n└── templates     . html templates used by python flask for serve the web pages\n```\n**Application description**\n\nThe main **server** application ```webapp.py``` is in the root folder. In this application there are routing to html dynamic templates and serves also back-end API. This is wrapped by ```wsgi.py``` for using with **bjoern** server.\n\nStatic files (css, js...) are in ```static``` directory: here there are subdirectories: \n- ```dev```  where you can edit and modify sources\n- ```rel```  here there are release files created with the building process and used in production\n\n**Lint**\n\nFor lint javascript I use **ESLint**. You can install with ```npm init @eslint/config```\n pylint        ```pip install pylint```\n\n**Building process**\n\nPrerequisites:\n| **Component** | **Description**                                    | **Install command**             |\n|---------------|----------------------------------------------------|---------------------------------|\n| npm           | a packet manager for javascript                                  |  depend on your os. See [official page](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm) |\n| uglify-js     | npm component used to minify and uglify javascript | ```npm install uglify-js -g```  |\n| css-minify    | npm component used to minify css                   | ```npm install css-minify -g``` |\n| staticjinja   | python module used to create static page starting from a html template                | ```pip install staticjinja```   |\n\n\nYou can build the software for test (dev), or for production (release) environments.\nIn ```scripts``` directory launch:\n- ```./build.sh -d``` for dev environment\n\n- ```./build.sh -r``` for release environment\n\n\n### Screenshots\n----------\n\n**desktop**\n\n\u003cimg src=\"docs/images/01_desktop_main.png\" width=\"400\"/\u003e\n\u003cimg src=\"docs/images/02_desktop_plot.png\" width=\"400\"/\u003e\n\n\n**mobile**\n\n\u003cp float=\"left\"\u003e\n\u003cimg src=\"docs/images/m01_mobile.png\" width=\"200\"/\u003e\n\u003cimg src=\"docs/images/m02_mobile.png\" width=\"200\"/\u003e\n\u003cimg src=\"docs/images/m03_mobile.png\" width=\"200\"/\u003e\n\u003cimg src=\"docs/images/m04_mobile.png\" width=\"200\"/\u003e\n\u003c/p\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoulisse%2Fspiderweb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcoulisse%2Fspiderweb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcoulisse%2Fspiderweb/lists"}