{"id":31004598,"url":"https://github.com/ecopque/ozmap-sync","last_synced_at":"2026-04-17T09:32:50.127Z","repository":{"id":313828593,"uuid":"1052824987","full_name":"ecopque/ozmap-sync","owner":"ecopque","description":"OZmap Sync is a service built with Node.js + TypeScript for data synchronization between an ISP system (mocked using json-server) and an external management system, simulating OZmap.","archived":false,"fork":false,"pushed_at":"2025-09-09T00:29:58.000Z","size":5727,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-09-09T02:46:27.493Z","etag":null,"topics":["node","nodejs","typescript"],"latest_commit_sha":null,"homepage":"https://linktr.ee/edsoncopque","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ecopque.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-08T15:46:17.000Z","updated_at":"2025-09-09T00:30:01.000Z","dependencies_parsed_at":"2025-09-09T02:59:38.990Z","dependency_job_id":null,"html_url":"https://github.com/ecopque/ozmap-sync","commit_stats":null,"previous_names":["ecopque/ozmap-sync"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/ecopque/ozmap-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecopque%2Fozmap-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecopque%2Fozmap-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecopque%2Fozmap-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecopque%2Fozmap-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ecopque","download_url":"https://codeload.github.com/ecopque/ozmap-sync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ecopque%2Fozmap-sync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31923170,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T09:10:15.403Z","status":"ssl_error","status_checked_at":"2026-04-17T09:10:14.455Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["node","nodejs","typescript"],"created_at":"2025-09-13T01:58:24.574Z","updated_at":"2026-04-17T09:32:50.105Z","avatar_url":"https://github.com/ecopque.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OZmap Sync\n\n## Overview\n\n**OZmap Sync** is a service built with **Node.js + TypeScript** for **data synchronization** between an ISP system (mocked using `json-server`) and an external management system, simulating **OZmap**.\n\nThe system:\n- Periodically fetches data from the ISP system.\n- Transforms the data into the required format.\n- Sends processed data to the target system.\n- Manages failures and respects request rate limits.\n- Logs all operations and events.\n- Stores synchronized records in **MongoDB**.\n\n---\n\n## Technologies Used\n\n- **Node.js** + **TypeScript** → Core of the application  \n- **json-server** → Mock ISP system  \n- **Mongoose** → MongoDB integration  \n- **Axios** → HTTP requests  \n- **Winston** → Advanced logging  \n- **Node-cron** → Scheduling synchronization jobs  \n- **dotenv** → Environment configuration\n\n---\n\n## Project Structure\n```bash\nozmap-sync/\n├── src/\n│   ├── config/\n│   │   ├── database.ts          # MongoDB connection configuration\n│   │   └── env.ts               # Environment variables loader (dotenv)\n│   │\n│   ├── database/\n│   │   ├── models/\n│   │   │   └── syncRecord.ts    # Mongoose schema for sync records\n│   │   └── index.ts             # Database connection initializer\n│   │\n│   ├── integrations/\n│   │   ├── ispClient.ts         # Fetch data from the ISP (json-server)\n│   │   └── ozmapClient.ts       # Simulated OZmap SDK client\n│   │\n│   ├── jobs/\n│   │   └── scheduler.ts         # Cron job to trigger synchronization\n│   │\n│   ├── services/\n│   │   ├── syncService.ts       # Main sync process orchestration\n│   │   └── transformService.ts  # Transforms ISP data to OZmap format\n│   │\n│   ├── utils/\n│   │   ├── logger.ts            # Centralized logging (Winston)\n│   │   └── rateLimiter.ts       # Handles request throttling per minute\n│   │\n│   ├── app.ts                   # Initializes app, scheduler, and configs\n│   └── index.ts                 # Main entry point\n│\n├── logs/\n│   ├── combined.log             # All application logs\n│   ├── error.log                # Only error logs\n│   ├── exceptions.log           # Exceptions captured automatically\n│   └── rejections.log           # Promise rejections logs\n│\n├── docs/\n│   └── architecture-diagram.png # Architecture diagram for README\n│\n├── db.json                      # Mock ISP data (json-server)\n├── .env.example                # Example environment variables\n├── .gitignore                  # Git ignored files and folders\n├── package.json\n├── tsconfig.json               # TypeScript configuration\n└── README.md\n\n```\n\n---\n\n## Project Setup\n\n### **1. Clone the repository**\n```bash\n$ git clone https://github.com/your-username/ozmap-sync.git\n$ cd ozmap-sync\n```\n\n### **2. Install dependencies**\n```bash\n$ npm install\n```\n\n### **3. Configure environment variables**\n```bash\nCreate a .env file in the project root based on .env.example:\nISP_BASE_URL=http://localhost:4000\nRATE_LIMIT_PER_MIN=50\nSYNC_CRON=*/1 * * * *\nMONGODB_URI=mongodb://localhost:27017/ozmap_sync\nLOG_LEVEL=debug\n```\n\n### **4. ISP Mock Server**\n```bash\nThis project uses json-server to simulate the ISP data source.\n$ npx json-server --watch db.json --port 4000\n```\n\n### **5. Running the Application**\n```bash\nDevelopment mode\n$ npm run dev\n\n. The app will:\n. Connect to MongoDB.\n. Fetch data from json-server.\n. Transform and send data to the target system (mocked OZmap).\n. Store processed records in MongoDB.\n. Generate detailed logs in the console and the logs/ folder.\n```\n\n### **6. Logs \u0026 Data Storage**\n```bash\nLogs are stored in:\n\nlogs/\n├── combined.log\n├── error.log\n├── exceptions.log\n└── rejections.log\n```\n\n### **7. To check synchronized records in MongoDB**\n```bash\n$ mongosh\n$ use ozmap_sync\n$ db.syncrecords.find().pretty()\n```\n\n### **8. Architecture**\n```bash\nThe architecture is designed to be modular and scalable, with clear separation of concerns:\n\n+--------------------+\n| json-server (ISP)  |\n+--------------------+\n          ↓\n    fetchISPData()\n          ↓\n+--------------------+\n| Transform Service  |\n+--------------------+\n          ↓\n+--------------------+\n| OZmap Client (SDK) |\n+--------------------+\n          ↓\n+--------------------+\n| MongoDB Storage    |\n+--------------------+\n```\n\n---\n\n## About the Author\n    . Developer: Edson Copque\n    . Website: https://linktr.ee/edsoncopque\n    . GitHub: https://github.com/ecopque\n    . Signal Messenger: ecop.01\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecopque%2Fozmap-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fecopque%2Fozmap-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fecopque%2Fozmap-sync/lists"}