{"id":18664328,"url":"https://github.com/dzsquared/sqlbindings-python-datatransfer","last_synced_at":"2025-11-06T09:30:34.119Z","repository":{"id":95258150,"uuid":"606939820","full_name":"dzsquared/sqlbindings-python-datatransfer","owner":"dzsquared","description":"Sample code for Python Azure Functions connecting to Azure SQL","archived":false,"fork":false,"pushed_at":"2023-02-27T01:04:58.000Z","size":6,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-12-27T17:20:24.541Z","etag":null,"topics":["azure-functions","azure-sql","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dzsquared.png","metadata":{"files":{"readme":"README.MD","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-27T01:02:53.000Z","updated_at":"2023-06-12T17:41:08.000Z","dependencies_parsed_at":"2023-03-13T16:52:36.122Z","dependency_job_id":null,"html_url":"https://github.com/dzsquared/sqlbindings-python-datatransfer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzsquared%2Fsqlbindings-python-datatransfer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzsquared%2Fsqlbindings-python-datatransfer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzsquared%2Fsqlbindings-python-datatransfer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dzsquared%2Fsqlbindings-python-datatransfer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dzsquared","download_url":"https://codeload.github.com/dzsquared/sqlbindings-python-datatransfer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239491352,"owners_count":19647811,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure-functions","azure-sql","python"],"created_at":"2024-11-07T08:23:06.111Z","updated_at":"2025-02-18T14:45:40.286Z","avatar_url":"https://github.com/dzsquared.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Sample: Load data from SQL using Python and Azure Functions\n\n**I'd prefer to use Python, how do I transfer data from Azure SQL Database every day?**\n\n* [Scenario 1](#scenario-1-write-to-an-ftp-server): Generate .txt files from data currently stored in Azure SQL Database and send the files to an FTP server\n* [Scenario 2](#scenario-2-send-data-to-an-api-endpoint): Take data from the Azure SQL Database and send the data to an API endpoint\n\n## Get data from Azure SQL Database in Azure Functions\n\nWith [Azure SQL bindings for Azure Functions](https://aka.ms/sqlbindings), we can easily retrieve data from an Azure SQL Database in an Azure Function.\n\nWe retrieve data from SQL using an **input binding** for Azure Functions by adding the following to the `function.json` file:\n\n```json\n{\n    \"name\": \"products\",\n    \"type\": \"sql\",\n    \"direction\": \"in\",\n    \"commandText\": \"SELECT [ProductID],[Name],[ProductModel],[Description] FROM [SalesLT].[vProductAndDescription]\",\n    \"commandType\": \"Text\",\n    \"connectionStringSetting\": \"SqlConnectionString\"\n}\n```\n\nIn this input binding, we are specifying a **query** to run against the database in the `commandText` property. We also specify the `connectionStringSetting` which is the name of the connection string in the `local.settings.json` file and Azure Functions app settings.\n\nThe output of that query is passed to the Azure Function as the parameter `products`, specified by the json property `name`.\n\n\u003e **Warning**\n\u003e SQL bindings for Azure Functions are currently in public preview. Include the preview bundle in your host.json file to use them:\n\u003e ```json\n\u003e \"extensionBundle\": {\n\u003e    \"id\": \"Microsoft.Azure.Functions.ExtensionBundle.Preview\",\n\u003e    \"version\": \"[4.*, 5.0.0)\"\n\u003e  }\n\u003e ```\n\n## Azure Functions timer trigger\n\nFor tasks that need to run on a schedule, we can use the [timer trigger](https://learn.microsoft.com/azure/azure-functions/functions-bindings-timer?tabs=in-process\u0026pivots=programming-language-python) for Azure Functions.\n\nA few notes:\n- The timer trigger assumes UTC time, and the `WEBSITE_TIME_ZONE` app setting is available only for certain hosts.\n- Timer triggers use NCRONTAB expressions to set the schedule, which is similar to CRON but with an additional field for seconds (`{second} {minute} {hour} {day} {month} {day-of-week}`).\n\n```json\n{\n    \"name\": \"everyDayAt5AM\",\n    \"type\": \"timerTrigger\",\n    \"direction\": \"in\",\n    \"schedule\": \"0 0 5 * * *\"\n}\n```\n\n## Scenario 1: write to an FTP server\n\nTo write data to an FTP server, we can use the built-in library `ftplib` in Python.\n\n- More about ftplib: https://docs.python.org/3/library/ftplib.html\n- More about pandas to_csv: https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_csv.html\n- Test FTP server: https://dlptest.com/ftp-test/\n- Sample function code: [SendDataToFTP](SendDataToFTP)\n\n```python\ndef main(everyDayAt5AM: func.TimerRequest, products: func.SqlRowList) -\u003e func.HttpResponse:\n    logging.info('Python HTTP trigger function processed a request.')\n    filename = \"products.txt\"\n    filesize = 0\n\n    # convert the SQL data to comma separated text\n    product_list = pandas.DataFrame(products)\n    product_csv = product_list.to_csv(index=False)\n    datatosend = io.BytesIO(product_csv.encode('utf-8'))\n\n    # get FTP connection details from app settings\n    FTP_HOST = os.environ['FTP_HOST']\n    FTP_USER = os.environ['FTP_USER']\n    FTP_PASS = os.environ['FTP_PASS']\n\n    # connect to the FTP server\n    try:\n        with ftplib.FTP(FTP_HOST, FTP_USER, FTP_PASS, encoding=\"utf-8\") as ftp:\n            logging.info(ftp.getwelcome())\n            # use FTP's STOR command to upload the data\n            ftp.storbinary(f\"STOR {filename}\", datatosend)\n            filesize = ftp.size(filename)\n            ftp.quit()\n    except Exception as e:\n        logging.error(e)\n\n    logging.info(f\"File {filename} uploaded to FTP server. Size: {filesize} bytes\")\n```\n\n## Scenario 2: send data to an API endpoint\n\nTo send the data from the SQL input binding to an API endpoint from the Azure Function, we can import the `requests` library and use it to make a POST request to the API endpoint.\n\n- More about requests: https://requests.readthedocs.io/en/latest/user/quickstart/\n- Sample function code: [SendDataToAPI](SendDataToAPI)\n\n```python\ndef main(everyDayAt5AM: func.TimerRequest, products: func.SqlRowList) -\u003e None:\n    logging.info('Python timer trigger function started')\n    # convert the SQL data to JSON in memory\n    rows = list(map(lambda r: json.loads(r.to_json()), products))\n\n    # get the API endpoint from app settings\n    api_url = os.environ['API_URL']\n\n    # send the data to the API\n    response = requests.post(api_url, json=rows)\n    # check for 2xx status code\n    if response.status_code // 100 != 2:\n        logging.error(f\"API response: {response.status_code} {response.reason}\")\n    else:\n        logging.info(f\"API response: {response.status_code} {response.reason}\")\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzsquared%2Fsqlbindings-python-datatransfer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdzsquared%2Fsqlbindings-python-datatransfer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdzsquared%2Fsqlbindings-python-datatransfer/lists"}