{"id":23331289,"url":"https://github.com/sassoftware/ci360-download-client-python","last_synced_at":"2025-07-30T05:03:43.391Z","repository":{"id":44732286,"uuid":"234328551","full_name":"sassoftware/ci360-download-client-python","owner":"sassoftware","description":"ci360 data download client program using Python","archived":false,"fork":false,"pushed_at":"2025-06-03T20:44:20.000Z","size":104,"stargazers_count":15,"open_issues_count":3,"forks_count":11,"subscribers_count":22,"default_branch":"master","last_synced_at":"2025-06-04T00:43:03.788Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sassoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"SUPPORT.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-16T13:40:10.000Z","updated_at":"2025-06-03T14:09:39.000Z","dependencies_parsed_at":"2024-12-20T22:33:29.926Z","dependency_job_id":"0a116ace-0601-499e-b988-7e18b87ab15c","html_url":"https://github.com/sassoftware/ci360-download-client-python","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sassoftware/ci360-download-client-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sassoftware%2Fci360-download-client-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sassoftware%2Fci360-download-client-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sassoftware%2Fci360-download-client-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sassoftware%2Fci360-download-client-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sassoftware","download_url":"https://codeload.github.com/sassoftware/ci360-download-client-python/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sassoftware%2Fci360-download-client-python/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267814282,"owners_count":24148326,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-20T22:33:19.943Z","updated_at":"2025-07-30T05:03:43.365Z","avatar_url":"https://github.com/sassoftware.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"#  SAS Customer Intelligence 360 Download Client: Python\n\n## Overview\nThis Python script enables you to download cloud-hosted data tables from SAS Customer Intelligence 360.\n \nThe script can perform the following tasks:\n * Download the following data marts: `detail`, `dbtReport`, and `snapshot (previously identity)`.\n * Specify a time range to be downloaded.\n * Automatically unzip the download packages and create csv files with header rows and field delimiters.\n * Keep track of all initiated downloads. This lets you download a delta from the last complete download and append it to one file per table.\n\nThis topic contains the following sections:\n* [Configuration](#configuration)\n* [Using the Download Script](#using-the-download-script)\n    * [Considerations](#considerations)\n    * [Running the script](#running-the-script)\n    * [Examples](#examples)\n* [Contributing](#contributing)\n* [License](#license)\n* [Additional Resources](#additional-resources)\n\n\n\n## Configuration\n1. Install Python (version 3 or later) from https://www.python.org/.\n\n   **Tip:** Select the option to add Python to your PATH variable. If you choose the advanced installation option, make sure to install the pip utility.\n   \n2. Make sure the following modules are installed for Python: `argparse`, `backoff`, `base64`, `codecs`, `csv`, `gzip`, `json`, `os`, \n`pandas` (version 1.3.0 or later), `PyJWT`, `requests`, `sys`, `time`, and `tqdm`.\n\n     In most cases, many of the modules are installed by default. To list all packages that are installed with Python \n     (through pip or by default), use this command:  \n     ```python -c help('modules')```\n     \n     **Tip:** In most situations, you can install the non-default packages with this command:  \n     ```pip install backoff pandas PyJWT requests tqdm```\n  \n\n3. Create an access point in SAS Customer Intelligence 360.\n    1. From the user interface, navigate to **General Settings** \u003e **External Access** \u003e **Access Points**.\n    2. Create a new access point if one does not exist.\n    3. Get the following information from the access point:  \n       ```\n        External gateway address: e.g. https://extapigwservice-\u003cserver\u003e/marketingGateway  \n        Name: ci360_agent  \n        Tenant ID: abc123-ci360-tenant-id-xyz  \n        Client secret: ABC123ci360clientSecretXYZ  \n       ```\n4. Download the Python script from this repository and save it to your local machine.\n\n5. In the `./dsccnfg/config.txt` file, set the following variables for your tenant:\n   ```\n     agentName = ci360_agent\n     tenantId  = abc123-ci360-tenant-id-xyz\n     secret    = ABC123ci360clientSecretXYZ\n     baseUrl   = https://extapigwservice-\u003cserver\u003e/marketingGateway/discoverService/dataDownload/eventData/\n   ```\n\n6. Verify the installation by running the following command from command prompt:  \n```py discover.py –h```\n\n\n## Using the Download Script\n\n### Considerations\nBefore starting a download, make a note of the following things:\n* When you use the option to create a CSV, choose a delimiter that is not present in the source data.\n* If data resets were processed and you download data in append mode, the old data is not deleted.  \n  The new reset data for the same time period will be appended to the file.\n* If you download data using schema 1 and then use append mode to download data using schema 6, the data is appended based on schema 6. However, the header rows in the existing file will not be updated.\n\n### Running the Script\n\n1. Open a command prompt.\n2. Run the discover.py script with \u003ca href=\"#script-parameters\"\u003eparameter values\u003c/a\u003e that are based on the tables that you want to download. For example, to download the detail tables with a start and end date range, you can run the following command:\n```\n  py discover.py -m detail -st 2019-12-01T00 -et 2019-12-01T12\n```\n\n---\n**Note:** On Unix-like environments and Macs, the default `py` or `python` command might default to Python 2 if that version is installed. Uninstall earlier versions of Python, or explicitly call Python 3 when you run script like this example:\n```\npython3 discover.py -m detail -st 2019-12-01T00 -et 2019-12-01T12\n```\n\nYou can verify which version runs by default with the following command: `python --version`\n\n---\n\n\u003ca name=\"script-parameters\"\u003e \u003c/a\u003e\n\nThese are the parameters to use when you run the discover.py script:\n\n| Parameter   | Description       |\n| :---------- | :-----------------|\n| -h          | Displays the help |\n| -m          | The table set to download. Use one of these values:\u003cbr\u003e\u003cul\u003e\u003cli\u003edetail (This value downloads Detail mart tables and the partitioned CDM tables - cdm_contact_history and cdm_response_history.)\u003c/li\u003e\u003cli\u003edbtReport\u003c/li\u003e\u003cli\u003esnapshot (for CDM tables that are not partitioned, identity tables, and metadata tables)\u003c/li\u003e\u003c/ul\u003e  |\n| -svn        | Specify a specific schema of tables to download. |\n| -st         | The start value in this datetime format: `yyyy-mm-ddThh` |\n| -et         | The end value in this datetime format: `yyyy-mm-ddThh`   |\n| -ct         | The category of tables to download. When the parameter is not specified, you download tables for all the categories that you have a license to access.\u003cbr\u003e\u003cbr\u003eTo download tables from a specific category, you can use one of these values:\u003cul\u003e\u003cli\u003ecdm\u003c/li\u003e\u003cli\u003ediscover\u003c/li\u003e\u003cli\u003eengagedigital\u003c/li\u003e\u003cli\u003eengagedirect\u003c/li\u003e\u003cli\u003eengagemetadata\u003c/li\u003e\u003cli\u003eengagemobile\u003c/li\u003e\u003cli\u003eengageweb\u003c/li\u003e\u003cli\u003eengageemail\u003c/li\u003e\u003cli\u003eoptoutdata\u003c/li\u003e\u003cli\u003eplan\u003c/li\u003e\u003c/ul\u003e\u003cbr\u003eFor more information, see [Schemas and Categories](https://go.documentation.sas.com/?cdcId=cintcdc\u0026cdcVersion=production.a\u0026docsetId=cintag\u0026docsetTarget=dat-export-api-sch.htm).| \n| -d          | Download only the changes (the delta) from the previous download. Set the value to `yes` or `no`. |\n| -l          | For partitioned tables, specify a limit of partitions to download. For example, `-l 150` downloads only the first 150 partitions of a specific set.|\n| -a          | Append the download to the existing files. Set the value to `yes` or `no`.  |\n| -cf         | Create a CSV file from the download tables. Set the value to `yes` or `no`. |\n| -cd         | Specify a delimiter other than the default |\n| -ch         | Include a column header in the first row. Set the value to `yes` or `no`. |\n| -cl         | Clean the download .zip files. By default, the files are deleted, but you can set this parameter to `no` to keep them. |\n\n**Note:** The start and end ranges are only used for the script's first run. After the first run, the download history is stored in the dsccnfg directory. To force the script to use the variables for start date and end date, delete or move the history information.\n\n   In addition, the values in the dataRangeStartTimeStamp column and dataRangeEndTimeStamp column in the download history tables are in the UTC time zone. The values in the download_dttm column are in the local time zone.\n\n### Examples\n\n* Download the detail tables:  \n```py discover.py –m detail```\n\n* Download the discover Base tables:  \n```py discover.py –m dbtReport```\n\n* Download the snapshot tables:  \n```py discover.py –m snapshot```\n\n* Download the complete set of the CDM tables (both partitioned tables and non-partitioned tables):  \n```py discover.py –m snapshot -ct cdm```  \n```py discover.py –m detail -ct cdm```\n\n* Download the detail tables (with only the delta from the last download), create a CSV file, and append to the existing files:  \n```py discover.py –m detail –d yes –cf yes –a yes```\n\n* Download the detail tables for the specific time range from start hour (`-st`) to end hour (`-et`):  \n```py discover.py -m detail -st 2019-12-01T00 -et 2019-12-01T12```\n\n* Download the discover base tables, create a CSV file, use the \";\" (semicolon) delimiter, and include a column header in \nthe first row:  \n```py discover.py -m dbtReport -cf yes -cd \";\" -ch yes```\n\n* This example is similar to the previous example, but the option `-cl no` keeps the downloaded zip files in the download \nfolder:  \n```py discover.py -m dbtReport -cf yes -cd \";\" -ch yes -cl no```\n\n* Download the detail tables with a specific schema (`-svn`), and specify a limit (`-l`) to download only the most recent \n150 partitions:  \n```py discover.py -m detail -svn 3 -l 150 -cf yes -cd \",\" -ch yes```\n\n* Download the Plan data tables, create a CSV file, use the \";\" (semicolon) delimiter, and include a column header in \nthe first row:  \n```py discover.py -m snapshot -ct plan -svn 5 -cf yes -cd \";\" -ch yes```\n\n\n\n## Contributing\n\nWe welcome your contributions! Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on how to submit contributions to this project.\n\n\n\n## License\n\nThis project is licensed under the [Apache 2.0 License](LICENSE).\n\n\n\n## Additional Resources\nFor more information, see [Downloading Data Tables with the REST API](https://go.documentation.sas.com/?softwareId=ONEMKTMID\u0026softwareVersion=production.a\u0026softwareContextId=DownloadDataTables) in the Help Center for SAS Customer Intelligence 360.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsassoftware%2Fci360-download-client-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsassoftware%2Fci360-download-client-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsassoftware%2Fci360-download-client-python/lists"}