{"id":26256744,"url":"https://github.com/victorlcastro-dsa/sharepoint-migration-app","last_synced_at":"2025-06-13T21:05:55.429Z","repository":{"id":276039258,"uuid":"921883570","full_name":"victorlcastro-dsa/sharepoint-migration-app","owner":"victorlcastro-dsa","description":"The \"sharepoint-migration-app\" repository is a Python project designed to facilitate the migration of data to SharePoint sites. The application aims to streamline the process, ensuring efficient and seamless data transfer.","archived":false,"fork":false,"pushed_at":"2025-02-27T18:20:01.000Z","size":36,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-13T20:19:02.552Z","etag":null,"topics":["data-transfer","python","sharepoint-online"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/victorlcastro-dsa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-24T19:53:00.000Z","updated_at":"2025-02-27T18:20:04.000Z","dependencies_parsed_at":"2025-02-06T00:18:57.801Z","dependency_job_id":"98343766-b494-49d8-b613-ae547e03a7c9","html_url":"https://github.com/victorlcastro-dsa/sharepoint-migration-app","commit_stats":null,"previous_names":["victorlcastro-dsa/sharepoint-migration-app"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/victorlcastro-dsa/sharepoint-migration-app","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorlcastro-dsa%2Fsharepoint-migration-app","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorlcastro-dsa%2Fsharepoint-migration-app/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorlcastro-dsa%2Fsharepoint-migration-app/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorlcastro-dsa%2Fsharepoint-migration-app/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/victorlcastro-dsa","download_url":"https://codeload.github.com/victorlcastro-dsa/sharepoint-migration-app/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/victorlcastro-dsa%2Fsharepoint-migration-app/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":259719716,"owners_count":22901239,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-transfer","python","sharepoint-online"],"created_at":"2025-03-13T20:19:04.993Z","updated_at":"2025-06-13T21:05:55.404Z","avatar_url":"https://github.com/victorlcastro-dsa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SharePoint Migration App\n\n## Overview\n\nThe SharePoint Migration App is a Python-based application designed to facilitate the migration of folder structures and files from one SharePoint site to another. The application fetches the folder structure from the source SharePoint site, saves it to an Excel file, and creates copy jobs to transfer the files to the destination SharePoint site.\n\n## Features\n\n- **Authentication**: Uses Microsoft Authentication Library (MSAL) to acquire access tokens for SharePoint API.\n- **Logging**: Configurable logging settings to monitor the application's activities.\n- **Configuration**: Loads configuration settings from environment variables.\n- **Fetch Structure**: Fetches the folder structure from the source SharePoint site using REST API.\n- **Export to Excel**: Saves the fetched folder structure to an Excel file.\n- **Create Copy Jobs**: Creates copy jobs in SharePoint to transfer files from the source to the destination site.\n\n## TODOs\n```\n# TODO: Implement monitoring of copy jobs\n# TODO: Implement filtering of items by name or other criteria\n# TODO: Add support for files, not just folders\n# TODO: Allow customization of the migration process via the configuration file and command-line arguments\n# TODO: Implement size validation to prevent exceeding SharePoint limits\n# TODO: Implement count validation to prevent exceeding SharePoint limits\n# TODO: Ensure the copy operation preserves the full folder hierarchy, even when copying a specific file or subfolder.\n```\n\n## Project Structure\n```\n.\n├── .env\n├── .gitignore\n├── app/\n│   ├── auth/\n│   │   ├── __init__.py\n│   │   ├── authenticator.py\n│   │   └── certificate_loader.py\n│   ├── config/\n│   │   ├── __init__.py\n│   │   ├── log_settings.py\n│   │   └── settings.py\n│   ├── data/\n│   │   ├── __init__.py\n│   │   └── sharepoint_folder_structure.xlsx\n│   ├── exceptions/\n│   │   ├── __init__.py\n│   │   ├── asyncio_exceptions.py\n│   │   ├── authentication_exceptions.py\n│   │   ├── certificate_exceptions.py\n│   │   ├── configuration_exceptions.py\n│   │   ├── excel_exceptions.py\n│   │   ├── job_exceptions.py\n│   │   ├── main_exceptions.py\n│   │   └── sharepoint_exceptions.py\n│   ├── services/\n│   │   ├── __init__.py\n│   │   ├── create_copy_jobs.py\n│   │   ├── create_excel.py\n│   │   ├── fetch_structure.py\n│   │   └── monitor_jobs.py\n│   └── utils/\n│       └── __init__.py\n├── certificate.pem\n├── main.py\n├── README.md\n├── requirements.txt\n└── .gitignore\n```\n\n## Installation\n\n1. Clone the repository:\n    ```sh\n    git clone \u003crepository-url\u003e\n    cd sharepoint-migration-app\n    ```\n\n2. Create and activate a virtual environment:\n    ```sh\n    python -m venv venv\n    source venv/bin/activate  # On Windows, use `venv\\Scripts\\activate`\n    ```\n\n3. Install the required dependencies:\n    ```sh\n    pip install -r requirements.txt\n    ```\n\n4. Create a `.env` file in the root directory with the following content:\n    ```ini\n    # Tenant Identification\n    TENANT_ID=\"your-tenant-id\"  # Azure AD tenant ID\n    TENANT_NAME=\"your-tenant-name\"  # Azure AD tenant name\n\n    # Client Credentials\n    CLIENT_ID=\"your-client-id\"  # Client ID of the registered application in Azure AD\n    CLIENT_SECRET=\"your-client-secret\"  # Client secret of the registered application in Azure AD\n\n    # Source and Destination URLs\n    ORIGIN_URL=\"your-origin-url\"  # Source SharePoint site URL\n    PARTIAL_ORIGIN_URL=\"your-partial-origin-url\"  # Partial URL of the source SharePoint site\n    DESTINATION_URL=\"your-destination-url\"  # Destination SharePoint site URL\n    BASE_URL=\"your-base-url\"  # Base URL of the SharePoint site\n\n    # API Configurations\n    API_SCOPE=\"your-api-scope\"  # API scope for authentication\n    CERTIFICATE_PATH=\"./certificado_completo.pem\"  # Path to the certificate file\n    THUMBPRINT=\"your-thumbprint\"  # Thumbprint of the certificate\n\n    # Log Configurations\n    LOG_LEVEL=\"DEBUG\"  # Log level\n    LOG_FORMAT=\"%(asctime)s - %(name)s - %(levelname)s - %(message)s\"  # Log format\n\n    # Migration Configurations\n    IGNORE_VERSION_HISTORY=False  # Ignore version history\n    ALLOW_SCHEMA_MISMATCH=True  # Allow schema mismatch\n    ALLOW_SMALLER_VERSION_LIMIT_ON_DESTINATION=True  # Allow smaller version limit on destination\n    INCLUDE_ITEM_PERMISSIONS=False  # Include item permissions\n    BYPASS_SHARED_LOCK=True  # Bypass shared lock\n    MOVE_BUT_KEEP_SOURCE=False  # Move but keep source\n    EXCLUDE_CHILDREN=False  # Exclude children\n    IS_MOVE_MODE=False  # Move mode\n    LEVEL=0  # Level of items to create copy jobs\n\n    # Data File Configurations\n    FETCH_FILENAME=\"sharepoint_folder_structure.xlsx\"  # Filename for the SharePoint folder structure\n\n    # aiohttp Configurations\n    AIOHTTP_LIMIT=10  # Connection limit for aiohttp\n    ```\n\n## Usage\n\nRun the main script:\n```sh\npython main.py\n```\n\nThe application will:\n1. Load configuration settings.\n2. Configure logging.\n3. Acquire an access token.\n4. Fetch the SharePoint folder structure.\n5. Save the structure to an Excel file.\n6. Create copy jobs to transfer files to the destination site.\n\n## Modules\n\n- **Authenticator**: Located in `app/auth/authenticator.py`, this module handles the acquisition and management of access tokens using MSAL.\n- **CertificateLoader**: Located in `app/auth/certificate_loader.py`, this module handles loading of certificates from a file.\n- **LogSettings**: Located in `app/config/log_settings.py`, this module configures logging settings for the application.\n- **Settings**: Located in `app/config/settings.py`, this module loads and stores configuration settings from environment variables.\n- **SharePointStructureFetcher**: Located in `app/services/fetch_structure.py`, this module fetches the folder structure from the SharePoint site using REST API.\n- **ExcelExporter**: Located in `app/services/create_excel.py`, this module handles exporting the SharePoint folder structure to an Excel file.\n- **CopyJobsCreator**: Located in `app/services/create_copy_jobs.py`, this module creates copy jobs in SharePoint for items with the specified level.\n- **MonitorJobs**: Located in `app/services/monitor_jobs.py`, this module is intended to monitor the status of copy jobs (currently empty).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorlcastro-dsa%2Fsharepoint-migration-app","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvictorlcastro-dsa%2Fsharepoint-migration-app","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvictorlcastro-dsa%2Fsharepoint-migration-app/lists"}