{"id":23823304,"url":"https://github.com/mimi-netizen/python-scripts","last_synced_at":"2026-04-17T13:31:43.901Z","repository":{"id":263273533,"uuid":"889856924","full_name":"mimi-netizen/Python-Scripts","owner":"mimi-netizen","description":"This repository provides a versatile collection of Python scripts to streamline system administration tasks. It includes tools for browser automation, web scraping, and cryptographic operations, with detailed documentation and planned enhancements to expand functionality","archived":false,"fork":false,"pushed_at":"2025-01-23T13:23:34.000Z","size":27188,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-11-12T13:23:45.370Z","etag":null,"topics":["cryptography","selenium-webdriver","webscraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mimi-netizen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-17T12:26:21.000Z","updated_at":"2025-01-23T13:23:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"6cd0eafb-cf84-470e-a395-fc60eae37576","html_url":"https://github.com/mimi-netizen/Python-Scripts","commit_stats":null,"previous_names":["mimi-netizen/python-scripts-for-system-administrators","mimi-netizen/web-scrapers"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mimi-netizen/Python-Scripts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimi-netizen%2FPython-Scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimi-netizen%2FPython-Scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimi-netizen%2FPython-Scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimi-netizen%2FPython-Scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mimi-netizen","download_url":"https://codeload.github.com/mimi-netizen/Python-Scripts/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mimi-netizen%2FPython-Scripts/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31931248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-17T12:37:54.787Z","status":"ssl_error","status_checked_at":"2026-04-17T12:37:25.095Z","response_time":62,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cryptography","selenium-webdriver","webscraping"],"created_at":"2025-01-02T10:16:33.358Z","updated_at":"2026-04-17T13:31:43.882Z","avatar_url":"https://github.com/mimi-netizen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python Scripts Collection\n\nA comprehensive collection of Python scripts focusing on Web Automation, Web Scraping, and Cryptography implementations. This repository serves as a resource for developers interested in automation, data extraction, and security.\n\n## 🌟 Projects Overview\n\n### 1. Selenium Web Automation\n\nAdvanced web automation scripts using Selenium WebDriver for various platforms and applications.\n\n#### Key Features\n\n- **Email Automation**: Automated email tasks and inbox management\n- **Social Media Automation**:\n  - Facebook interactions\n  - WhatsApp messaging\n  - Telegram bot functionality\n  - YouTube automation\n- **Task Management**:\n  - Trello board automation\n  - Project workflow automation\n- **Locator Strategies**:\n  - XPath implementations\n  - CSS Selector examples\n  - Practical automation templates\n\n### 2. Web Scrapers\n\nCollection of Python web scraping scripts for automated data extraction from various websites.\n\n#### Key Features\n\n- **Specialized Scrapers**:\n  - Books information extractor\n  - Carrefour website product scraper\n  - IMDB movie/TV show data collector\n  - Indeed job listings extractor\n  - General HTML scraper\n  - API data extractor\n- **Data Extraction Capabilities**:\n  - Product information\n  - Pricing details\n  - Description extraction\n  - Rating systems\n  - User reviews\n\n### 3. Cryptography\n\nImplementation of various cryptographic algorithms and security concepts.\n\n#### Current Implementations\n\n- Types of Ciphers\n- Public Key Cryptography\n- Hash Functions\n- Message Authentication Codes (MAC, CMAC, HMAC)\n- Digital Certificates\n- Authentication Systems\n- TCP/IP Security\n- Internet Security Protocols\n- Firewall Implementation\n- Intrusion Detection System\n- Network Security Concepts\n\n## 🛠️ Technical Requirements\n\n### General Requirements\n\n- Python 3.x\n- Git\n\n### Project-Specific Dependencies\n\n#### Selenium Web Automation\n\n```bash\npip install selenium webdriver_manager\n```\n\n#### Web Scrapers\n\n```bash\npip install requests beautifulsoup4\n```\n\n#### Cryptography\n\n```bash\npip install pycryptodome\n```\n\n## 📦 Installation\n\n1. Clone the repository\n\n```bash\ngit clone https://github.com/mimi-netizen/Python-Scripts.git\n```\n\n2. Navigate to the project directory\n\n```bash\ncd Python-Scripts\n```\n\n3. Install required dependencies\n\n```bash\npip install -r requirements.txt\n```\n\n## 🚀 Usage\n\n### Selenium Web Automation\n\n```python\nfrom selenium import webdriver\nfrom selenium.webdriver.chrome.service import Service\nfrom selenium.webdriver.chrome.options import Options\nfrom webdriver_manager.chrome import ChromeDriverManager\n\n# Setup Chrome WebDriver\ndriver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))\n```\n\n### Web Scrapers\n\n```python\nimport requests\nfrom bs4 import BeautifulSoup\n\n# Basic scraping template\nresponse = requests.get('https://example.com')\nsoup = BeautifulSoup(response.text, 'html.parser')\n```\n\n### Cryptography\n\n```python\n# Example cipher implementation\ndef caesar_cipher(text, shift, encrypt=True):\n    result = \"\"\n    for char in text:\n        if char.isalpha():\n            ascii_offset = ord('A') if char.isupper() else ord('a')\n            shift_value = shift if encrypt else -shift\n            result += chr((ord(char) - ascii_offset + shift_value) % 26 + ascii_offset)\n        else:\n            result += char\n    return result\n```\n\n## 🔍 Monitoring and Logging\n\n### Selenium Projects\n\nAll Selenium projects create logs in `logs/selenium/`:\n\n```\ntail -f logs/selenium/automation.log\n```\n\n### Web Scrapers\n\nMonitor scraping progress:\n\n```bash\ntail -f logs/scrapers/scraping_progress.log\n```\n\nRate limiting and error logs:\n\n```bash\ncat logs/scrapers/rate_limits.log\ncat logs/scrapers/errors.log\n```\n\n### Cryptography\n\nOperation logs and performance metrics:\n\n```bash\ncat logs/crypto/operations.log\ncat logs/crypto/performance_metrics.log\n```\n\n## 🐛 Debugging Tips\n\n### Selenium Debugging\n\n- Enable verbose logging:\n\n```python\nimport logging\nlogging.basicConfig(level=logging.DEBUG)\n```\n\n- Use Chrome DevTools:\n\n```python\nchrome_options.add_argument('--auto-open-devtools-for-tabs')\n```\n\n### Web Scraper Debugging\n\n- Test selectors in browser console\n- Use `scrapy shell` for interactive debugging:\n\n```bash\nscrapy shell \"http://example.com\"\n```\n\n### Cryptography Debugging\n\n- Enable verbose mode:\n\n```bash\npython crypto_script.py --verbose\n```\n\n- Use test vectors in `tests/vectors/` directory\n\n## 🐛 Advanced Debugging Guide\n\n### 1. Selenium Debugging Techniques\n\n#### Basic Logging Setup\n\n```python\nimport logging\n\n# Configure detailed logging\nlogging.basicConfig(\n    level=logging.DEBUG,\n    format='%(asctime)s - %(levelname)s - %(message)s',\n    handlers=[\n        logging.FileHandler('debug.log'),\n        logging.StreamHandler()\n    ]\n)\n\n# Create a custom logger\nlogger = logging.getLogger('selenium_automation')\n```\n\n#### Screenshot Capture\n\n```python\ndef capture_error_screenshot(driver, error_name):\n    timestamp = datetime.now().strftime('%Y%m%d_%H%M%S')\n    screenshot_path = f\"debug/screenshots/{error_name}_{timestamp}.png\"\n    driver.save_screenshot(screenshot_path)\n    logger.debug(f\"Screenshot saved: {screenshot_path}\")\n\n# Usage in try-except block\ntry:\n    element = driver.find_element(By.ID, \"submit-button\")\n    element.click()\nexcept Exception as e:\n    capture_error_screenshot(driver, \"click_error\")\n    logger.error(f\"Click failed: {str(e)}\")\n```\n\n#### Element State Debugging\n\n```python\ndef debug_element(driver, locator, by=By.CSS_SELECTOR):\n    try:\n        element = driver.find_element(by, locator)\n        debug_info = {\n            \"is_displayed\": element.is_displayed(),\n            \"is_enabled\": element.is_enabled(),\n            \"is_selected\": element.is_selected(),\n            \"location\": element.location,\n            \"size\": element.size,\n            \"class\": element.get_attribute(\"class\"),\n            \"computed_style\": driver.execute_script(\n                \"return window.getComputedStyle(arguments[0]);\",\n                element\n            )\n        }\n        logger.debug(f\"Element debug info: {debug_info}\")\n        return debug_info\n    except Exception as e:\n        logger.error(f\"Element debugging failed: {str(e)}\")\n```\n\n#### Network Traffic Monitoring\n\n```python\nfrom selenium.webdriver.chrome.options import Options\n\ndef setup_network_debugging():\n    chrome_options = Options()\n    chrome_options.set_capability(\n        \"goog:loggingPrefs\",\n        {\"performance\": \"ALL\", \"browser\": \"ALL\"}\n    )\n    return chrome_options\n\ndef analyze_network_logs(driver):\n    logs = driver.get_log(\"performance\")\n    for log in logs:\n        network_log = json.loads(log[\"message\"])[\"message\"]\n        if \"Network.responseReceived\" in network_log[\"method\"]:\n            logger.debug(f\"Network response: {network_log}\")\n```\n\n### 2. Web Scraper Debugging\n\n#### Request Inspection\n\n```python\nimport requests\nfrom urllib3.util.retry import Retry\nfrom requests.adapters import HTTPAdapter\n\ndef debug_request(url, headers=None):\n    session = requests.Session()\n    retries = Retry(\n        total=3,\n        backoff_factor=0.5,\n        status_forcelist=[500, 502, 503, 504]\n    )\n    session.mount('http://', HTTPAdapter(max_retries=retries))\n    session.mount('https://', HTTPAdapter(max_retries=retries))\n\n    try:\n        response = session.get(url, headers=headers)\n        debug_info = {\n            \"status_code\": response.status_code,\n            \"headers\": dict(response.headers),\n            \"cookies\": dict(response.cookies),\n            \"encoding\": response.encoding,\n            \"redirect_history\": [\n                {\"url\": r.url, \"status_code\": r.status_code}\n                for r in response.history\n            ]\n        }\n        logger.debug(f\"Request debug info: {debug_info}\")\n        return debug_info\n    except Exception as e:\n        logger.error(f\"Request failed: {str(e)}\")\n```\n\n#### HTML Parser Debugging\n\n```python\nfrom bs4 import BeautifulSoup\nimport lxml.html\n\ndef debug_parsing(html_content):\n    # BeautifulSoup parsing\n    soup = BeautifulSoup(html_content, 'html.parser')\n\n    # Compare with lxml parsing\n    lxml_doc = lxml.html.fromstring(html_content)\n\n    debug_info = {\n        \"soup_title\": soup.title.string if soup.title else None,\n        \"lxml_title\": lxml_doc.find(\".//title\").text if lxml_doc.find(\".//title\") is not None else None,\n        \"soup_encoding\": soup.original_encoding,\n        \"tags_count\": len(soup.find_all()),\n        \"broken_tags\": soup.find_all(lambda tag: tag.name is None)\n    }\n\n    logger.debug(f\"Parsing debug info: {debug_info}\")\n    return debug_info\n```\n\n### 3. Cryptography Debugging\n\n#### Key Generation Debugging\n\n```python\nfrom cryptography.hazmat.primitives import serialization\nfrom cryptography.hazmat.primitives.asymmetric import rsa\n\ndef debug_key_generation(key_size=2048):\n    try:\n        start_time = time.time()\n        private_key = rsa.generate_private_key(\n            public_exponent=65537,\n            key_size=key_size\n        )\n        generation_time = time.time() - start_time\n\n        # Analyze key properties\n        debug_info = {\n            \"key_size\": key_size,\n            \"generation_time\": generation_time,\n            \"public_numbers\": {\n                \"n\": private_key.public_key().public_numbers().n,\n                \"e\": private_key.public_key().public_numbers().e\n            },\n            \"private_numbers\": {\n                \"p\": private_key.private_numbers().p,\n                \"q\": private_key.private_numbers().q,\n                \"d\": private_key.private_numbers().d\n            }\n        }\n\n        logger.debug(f\"Key generation debug info: {debug_info}\")\n        return debug_info\n    except Exception as e:\n        logger.error(f\"Key generation failed: {str(e)}\")\n```\n\n#### Encryption Process Debugging\n\n```python\ndef debug_encryption_process(data, key):\n    try:\n        debug_info = {\n            \"input_data\": {\n                \"length\": len(data),\n                \"encoding\": chardet.detect(data)\n            },\n            \"key_info\": {\n                \"algorithm\": key.algorithm.name,\n                \"key_size\": key.key_size\n            }\n        }\n\n        # Track encryption steps\n        padded_data = pad(data)\n        debug_info[\"padding\"] = {\n            \"original_length\": len(data),\n            \"padded_length\": len(padded_data)\n        }\n\n        # Monitor encryption performance\n        start_time = time.time()\n        encrypted_data = encrypt(padded_data, key)\n        debug_info[\"encryption_time\"] = time.time() - start_time\n\n        logger.debug(f\"Encryption debug info: {debug_info}\")\n        return encrypted_data, debug_info\n    except Exception as e:\n        logger.error(f\"Encryption failed: {str(e)}\")\n```\n\n#### Memory Usage Monitoring\n\n```python\nimport psutil\nimport os\n\ndef monitor_memory_usage():\n    process = psutil.Process(os.getpid())\n    memory_info = {\n        \"rss\": process.memory_info().rss / 1024 / 1024,  # MB\n        \"vms\": process.memory_info().vms / 1024 / 1024,  # MB\n        \"percent\": process.memory_percent(),\n        \"cpu_percent\": process.cpu_percent()\n    }\n    logger.debug(f\"Memory usage: {memory_info}\")\n    return memory_info\n\n# Usage example\ndef process_large_file(file_path):\n    initial_memory = monitor_memory_usage()\n\n    try:\n        with open(file_path, 'rb') as f:\n            for chunk in iter(lambda: f.read(4096), b''):\n                process_chunk(chunk)\n                current_memory = monitor_memory_usage()\n\n                if current_memory[\"percent\"] \u003e 90:\n                    logger.warning(\"High memory usage detected!\")\n\n    except Exception as e:\n        logger.error(f\"File processing failed: {str(e)}\")\n\n    final_memory = monitor_memory_usage()\n    logger.debug(f\"Memory change: {final_memory['rss'] - initial_memory['rss']} MB\")\n```\n\n## 🔄 Common Issues and Solutions\n\n### Selenium\n\n- ChromeDriver version mismatch: Update using `webdriver_manager`\n- Element not found: Implement explicit waits\n- Stale elements: Refresh page or relocate element\n\n### Web Scrapers\n\n- Rate limiting: Implement delays between requests\n- IP blocks: Rotate proxies using `proxy_rotator.py`\n- Dynamic content: Use Selenium for JavaScript-heavy sites\n\n### Cryptography\n\n- Key generation errors: Check system entropy\n- Performance issues: Use appropriate key sizes\n- Memory errors: Implement streaming for large files\n\n## 📚 Documentation\n\nEach project contains its own detailed documentation:\n\n- [Selenium Web Automation Documentation](./Selenium%20Web%20Automation/README.md)\n- [Web Scrapers Documentation](./Web%20Scrapers/README.md)\n- [Cryptography Documentation](./cryptography/README.md)\n\n## 🤝 Contributing\n\n1. Fork the repository\n2. Create your feature branch (`git checkout -b feature/AmazingFeature`)\n3. Commit your changes (`git commit -m 'Add some AmazingFeature'`)\n4. Push to the branch (`git push origin feature/AmazingFeature`)\n5. Open a Pull Request\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 📞 Contact\n\nCelyne Kydd - celynekydd@gmail.com\n\nProject Link: [https://github.com/mimi-netizen/Python-Scripts](https://github.com/mimi-netizen/Python-Scripts)\n\n## 🙏 Acknowledgments\n\n- Selenium Documentation\n- BeautifulSoup Documentation\n- Cryptography Libraries and Resources\n- Python Community\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmimi-netizen%2Fpython-scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmimi-netizen%2Fpython-scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmimi-netizen%2Fpython-scripts/lists"}