{"id":51550306,"url":"https://github.com/gesiscss/gesis_surf_backend","last_synced_at":"2026-07-09T23:01:51.435Z","repository":{"id":341445559,"uuid":"713798566","full_name":"gesiscss/gesis_surf_backend","owner":"gesiscss","description":"Backend service GESIS Surf","archived":false,"fork":false,"pushed_at":"2026-06-21T12:15:57.000Z","size":3428,"stargazers_count":1,"open_issues_count":5,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2026-06-21T14:33:12.717Z","etag":null,"topics":["backend","django-rest-framework","python","tracking"],"latest_commit_sha":null,"homepage":"https://www.gesis.org/info/surf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gesiscss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"security_scripts/manage_patterns.py","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-11-03T09:01:23.000Z","updated_at":"2026-05-09T13:00:19.000Z","dependencies_parsed_at":"2026-03-01T21:07:08.089Z","dependency_job_id":null,"html_url":"https://github.com/gesiscss/gesis_surf_backend","commit_stats":null,"previous_names":["gesiscss/gesis_surf_backend"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/gesiscss/gesis_surf_backend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gesiscss%2Fgesis_surf_backend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gesiscss%2Fgesis_surf_backend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gesiscss%2Fgesis_surf_backend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gesiscss%2Fgesis_surf_backend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gesiscss","download_url":"https://codeload.github.com/gesiscss/gesis_surf_backend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gesiscss%2Fgesis_surf_backend/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35314872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","django-rest-framework","python","tracking"],"created_at":"2026-07-09T23:01:50.300Z","updated_at":"2026-07-09T23:01:51.426Z","avatar_url":"https://github.com/gesiscss.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\u003ctable\u003e\u003ctr\u003e\u003ctd bgcolor=\"white\" style=\"padding: 20px;\"\u003e\n\u003cimg src=\"images/gesis.png\" alt=\"GESIS\" height=\"60\"\u003e\n\u003c/td\u003e\u003c/tr\u003e\u003c/table\u003e\n\n# GESIS Surf Backend\n\n**A Django-based backend for GESIS Surf research services**\n\n[![Python](https://img.shields.io/badge/python-3.10+-3670A0?style=for-the-badge\u0026logo=python\u0026logoColor=ffdd54)](https://www.python.org/)\n[![Django](https://img.shields.io/badge/django-4.2-092E20?style=for-the-badge\u0026logo=django\u0026logoColor=white)](https://www.djangoproject.com/)\n[![PostgreSQL](https://img.shields.io/badge/postgresql-13-316192?style=for-the-badge\u0026logo=postgresql\u0026logoColor=white)](https://www.postgresql.org/)\n[![Docker](https://img.shields.io/badge/docker-ready-2496ED?style=for-the-badge\u0026logo=docker\u0026logoColor=white)](https://www.docker.com/)\n[![License](https://img.shields.io/badge/license-MIT-green?style=for-the-badge)](LICENSE)\n\n[Features](#-features) •\n[Installation](#-installation) •\n[Usage](#-usage) •\n[API Documentation](#-api-documentation) •\n[Contributing](#-contributing) •\n[License](#-license)\n\n\u003c/div\u003e\n\n---\n\n## 📖 Overview\n\nGESIS Surf Backend is the server-side component of the GESIS Surf project, designed to collect and manage browsing behavior data for social science research. Built with Django REST Framework, it provides a robust API for browser extension integration.\n\n\u003e 🔗 **Looking for the browser extension?** Check out [GESIS Surf Extension](https://github.com/gesiscss/gesis_surf_extension)\n\n## ✨ Features\n\n- 🔐 **Secure Authentication** - Token-based user authentication\n- 📊 **Data Collection** - Track windows, tabs, domains, clicks, scrolls and HTMLs\n- 🔄 **Real-time Processing** - Celery-powered async task queue\n- 🔍 **Elasticsearch Integration** - Fast and scalable search capabilities\n- 📈 **Monitoring** - ELK stack for logging and visualization\n- 🐳 **Docker Ready** - Full containerized deployment\n\n## 🏗️ Architecture\n\n```\n┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐\n│  Browser Ext.   │────▶│   Django API    │────▶│   PostgreSQL    │\n└─────────────────┘     └─────────────────┘     └─────────────────┘\n                               │\n                               ▼\n                        ┌─────────────────┐\n                        │  Celery/Redis   │\n                        └─────────────────┘\n                               │\n                               ▼\n                        ┌─────────────────┐\n                        │  Elasticsearch  │\n                        └─────────────────┘\n```\n\n## 📋 Requirements\n\n- Python 3.10+\n- Docker \u0026 Docker Compose\n- PostgreSQL 13+\n- Redis 7+\n- Elasticsearch 7.x\n\n## 🚀 Installation\n\n### Using Docker (Recommended)\n\n1. **Clone the repository**\n\n   ```bash\n   git clone https://github.com/geomario/gesis_surf_backend.git\n   cd gesis_surf_backend\n   ```\n\n2. **Create environment file**\n\n   ```bash\n   cp .env.example .env\n   # Edit .env with your configuration\n   ```\n\n3. **Build and start services**\n\n   ```bash\n   docker-compose up -d --build\n   ```\n\n4. **Run migrations**\n\n   ```bash\n   docker-compose run --rm app sh -c \"python manage.py migrate\"\n   ```\n\n5. **Create superuser**\n   ```bash\n   docker-compose run --rm app sh -c \"python manage.py createsuperuser\"\n   ```\n\n### Local Development\n\n1. **Clone the repository**\n\n   ```bash\n   git clone https://github.com/geomario/gesis_surf_backend.git\n   cd gesis_surf_backend\n   ```\n\n2. **Install Poetry** (if not already installed)\n\n   ```bash\n   curl -sSL https://install.python-poetry.org | python3 -\n   ```\n\n3. **Install dependencies**\n\n   ```bash\n   poetry install\n   ```\n\n4. **Activate virtual environment**\n\n   ```bash\n   poetry shell\n   ```\n\n5. **Set up environment variables**\n   ```bash\n   cp .env.example .env\n   # Edit .env with your configuration\n   ```\n\n### Useful Poetry Commands\n\n| Command                            | Description                        |\n| ---------------------------------- | ---------------------------------- |\n| `poetry install`                   | Install all dependencies           |\n| `poetry install --with dev`        | Install with dev dependencies      |\n| `poetry add \u003cpackage\u003e`             | Add a new dependency               |\n| `poetry add --group dev \u003cpackage\u003e` | Add a dev dependency               |\n| `poetry update`                    | Update all dependencies            |\n| `poetry shell`                     | Activate virtual environment       |\n| `poetry run \u003ccommand\u003e`             | Run command in virtual environment |\n\n## 💻 Usage\n\n### Running the Application\n\n```bash\n# Start all services\ndocker-compose up -d\n\n# View logs\ndocker-compose logs -f app\n\n# Stop services\ndocker-compose down\n```\n\n### Common Commands\n\n| Command                                                               | Description            |\n| --------------------------------------------------------------------- | ---------------------- |\n| `docker-compose run --rm app sh -c \"python manage.py test\"`           | Run all tests          |\n| `docker-compose run --rm app sh -c \"python manage.py test APP_NAME\"`  | Run specific app tests |\n| `docker-compose run --rm app sh -c \"python manage.py makemigrations\"` | Create migrations      |\n| `docker-compose run --rm app sh -c \"python manage.py migrate\"`        | Apply migrations       |\n| `docker-compose run --rm app sh -c \"flake8\"`                          | Run linting            |\n\n### Access Points\n\n| Service         | URL                             |\n| --------------- | ------------------------------- |\n| API             | http://localhost:8000/api/      |\n| Admin           | http://localhost:8000/admin/    |\n| API Docs        | http://localhost:8000/api/docs/ |\n| Flower (Celery) | http://localhost:5555/          |\n| Kibana          | http://localhost:5601/          |\n| PgAdmin         | http://localhost:5050/          |\n\n## 📚 API Documentation\n\nInteractive API documentation is available via **drf-spectacular**:\n\n- **Swagger UI**: `http://localhost:8000/api/docs/`\n- **OpenAPI Schema**: `http://localhost:8000/api/schema/`\n\n### Key Endpoints\n\n| Endpoint           | Method    | Description        |\n| ------------------ | --------- | ------------------ |\n| `/api/user/`       | POST      | User registration  |\n| `/api/user/token/` | POST      | Obtain auth token  |\n| `/api/window/`     | GET, POST | Window management  |\n| `/api/tab/`        | GET, POST | Tab tracking       |\n| `/api/domain/`     | GET       | Domain information |\n| `/api/clicks/`     | POST      | Click events       |\n| `/api/scrolls/`    | POST      | Scroll events      |\n\n## 🗃️ Database Design\n\n### Entity Relationship Diagram\n\n![ERD GESIS](./images/entity_relationship_diagram.png)\n\n### Key Relationships\n\n| Relationship  | Type         | Description                         |\n| ------------- | ------------ | ----------------------------------- |\n| User ↔ Wave   | Many-to-Many | Users participate in research waves |\n| User → Window | One-to-Many  | Users have multiple browser windows |\n| Window → Tab  | One-to-Many  | Windows contain multiple tabs       |\n| Tab → Domain  | Many-to-One  | Tabs belong to domains              |\n| Tab → Click   | One-to-Many  | Tabs record click events            |\n| Tab → Scroll  | One-to-Many  | Tabs record scroll events           |\n\n## 🧪 Testing\n\n```bash\n# Run all tests\ndocker-compose run --rm app sh -c \"python manage.py test\"\n\n# Run with coverage\ndocker-compose run --rm app sh -c \"coverage run manage.py test \u0026\u0026 coverage report\"\n\n# Run specific test\ndocker-compose run --rm app sh -c \"python manage.py test core.tests.test_models\"\n```\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see our **[Contributing Guide](CONTRIBUTING.md)** for detailed information on:\n\n- 🌿 **Branching Strategy** - `dev` → `main` → `prod` workflow\n- 📝 **Commit Conventions** - Using Commitizen with Conventional Commits\n- 🔍 **Code Quality** - Pre-commit hooks, linting, and formatting\n- 🔀 **Pull Request Process** - Guidelines and review workflow\n\n### Quick Start\n\n1. **Fork the repository**\n2. **Create a feature branch** from `dev`\n   ```bash\n   git checkout dev \u0026\u0026 git pull origin dev\n   git checkout -b feature/amazing-feature\n   ```\n3. **Install pre-commit hooks**\n   ```bash\n   poetry install\n   poetry run pre-commit install\n   ```\n4. **Commit using Commitizen**\n   ```bash\n   git add .\n   poetry run cz commit\n   ```\n5. **Push and open a Pull Request** targeting `dev`\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 📝 Citing\n\nIf you use this software in your research, please cite:\n\n```bibtex\n@article{ramirez2025gesis,\n  title = {GESIS Surf },\n  author = {Ramirez, Mario and },\n  journal = {SoftwareX},\n  volume = {XX},\n  pages = {XXXXXX},\n  year = {2026},\n  publisher = {Elsevier},\n  doi = {10.1016/j.softx.2025.xxxxxx}\n}\n```\n\nSee [CITATION.cff](CITATION.cff) for more citation formats.\n\n## 👥 Authors\n\n- **Mario Ramirez** - _Lead Research Software Engineer_ - [@geomario](https://github.com/geomario) [@MarioGesis](https://www.gesis.org/en/institute/about-us/staff/person/mario.ramirez)\n- **Fernando Guzman** - _Software Architect Consultant_ - [@Fernando](https://www.linkedin.com/in/fernando-guzman-9262801b/)\n- **Prof. Dr. Sebastian Stier** - _Department Director CSS_ [@Seb](https://www.gesis.org/en/institute/about-us/staff/person/sebastian.stier)\n- **Dr. Frank Mangold** - _Kommissarischer Teamleiter DDD_ [@Frank](https://www.gesis.org/institut/ueber-uns/mitarbeitendenverzeichnis/person/Frank.Mangold)\n\n## 🙏 Acknowledgments\n\n- [GESIS - Leibniz Institute for the Social Sciences](https://www.gesis.org/)\n- [Computational Social Science Department](https://www.gesis.org/en/institute/about-us/departments/computational-social-science)\n\n## 📧 Contact\n\nQuestions or feedback? Reach out!\n\n- **Email**: mario.ramirez@gesis.org\n- **GitHub Issues**: [Create an issue](https://github.com/geomario/gesis_surf_backend/issues)\n\n---\n\n\u003cdiv align=\"center\"\u003e\nMade with ❤️ at GESIS\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgesiscss%2Fgesis_surf_backend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgesiscss%2Fgesis_surf_backend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgesiscss%2Fgesis_surf_backend/lists"}