{"id":19620607,"url":"https://github.com/titan550/bcpy","last_synced_at":"2025-04-28T03:32:07.585Z","repository":{"id":55432120,"uuid":"172234359","full_name":"titan550/bcpy","owner":"titan550","description":"Python Wrapper for Microsoft SQL Server bcp Utility","archived":false,"fork":false,"pushed_at":"2023-09-11T12:39:05.000Z","size":157,"stargazers_count":34,"open_issues_count":13,"forks_count":16,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-03T04:48:04.177Z","etag":null,"topics":["bcp","python","sqlserver"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/titan550.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-02-23T16:08:46.000Z","updated_at":"2024-11-19T21:36:06.000Z","dependencies_parsed_at":"2024-06-19T10:01:52.714Z","dependency_job_id":"8882ccc7-e404-421f-aefc-e49924dac7b8","html_url":"https://github.com/titan550/bcpy","commit_stats":{"total_commits":249,"total_committers":4,"mean_commits":62.25,"dds":"0.016064257028112428","last_synced_commit":"5eeb9de1fb1a30d23e963da6db985ff89600f3a8"},"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titan550%2Fbcpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titan550%2Fbcpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titan550%2Fbcpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/titan550%2Fbcpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/titan550","download_url":"https://codeload.github.com/titan550/bcpy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251246155,"owners_count":21558759,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bcp","python","sqlserver"],"created_at":"2024-11-11T11:19:24.212Z","updated_at":"2025-04-28T03:32:07.295Z","avatar_url":"https://github.com/titan550.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bcpy\n\n\u003ctable\u003e\n\u003ctr\u003e\n  \u003ctd\u003eLatest Release\u003c/td\u003e\n  \u003ctd\u003e\n    \u003ca href=\"https://pypi.org/project/bcpy/\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/v/bcpy.svg\" alt=\"latest release\" /\u003e\n    \u003c/a\u003e\n  \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003ctd\u003eLicense\u003c/td\u003e\n  \u003ctd\u003e\n    \u003ca href=\"https://github.com/titan550/bcpy/blob/master/LICENSE\"\u003e\n    \u003cimg src=\"https://img.shields.io/pypi/l/bcpy.svg\" alt=\"license\" /\u003e\n    \u003c/a\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n  \u003ctd\u003eBuild Status (master)\u003c/td\u003e\n  \u003ctd\u003e\n    \u003ca href=\"https://travis-ci.org/titan550/bcpy\"\u003e\n    \u003cimg src=\"https://api.travis-ci.org/titan550/bcpy.svg?branch=master\" alt=\"travis build status\" /\u003e\n    \u003c/a\u003e\n  \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n## What is it?\n\nThis package is a wrapper for Microsoft's SQL Server bcp utility. Current database drivers available in Python are not fast enough for transferring millions of records (yes, I have tried [pyodbc fast_execute_many](https://github.com/mkleehammer/pyodbc/wiki/Features-beyond-the-DB-API#fast_executemany)). Despite the IO hits, the fastest option by far is saving the data to a CSV file in file system (preferably /dev/shm tmpfs) and using the bcp utility to transfer the CSV file to SQL Server.\n\n## How Can I Install It?\n\n1. Make sure your computeer has the [requirements](#requirements).\n1. You can download and install this package from PyPI repository by running the command below.\n\n```bash\npip install bcpy\n```\n\n## Examples\n\nFollowing examples show you how to load (1) flat files and (2) DataFrame objects to SQL Server using this package.\n\n### Flat File\n\nFollowing example assumes that you have a comma separated file with no qualifier in path 'tests/data1.csv'. The code below sends the the file to SQL Server.\n\n```python\nimport bcpy\n\n\nsql_config = {\n    'server': 'sql_server_hostname',\n    'database': 'database_name',\n    'username': 'test_user',\n    'password': 'test_user_password1234'\n}\nsql_table_name = 'test_data1'\ncsv_file_path = 'tests/data1.csv'\nflat_file = bcpy.FlatFile(qualifier='', path=csv_file_path)\nsql_table = bcpy.SqlTable(sql_config, table=sql_table_name)\nflat_file.to_sql(sql_table)\n```\n\n### DataFrame\n\nThe following example creates a DataFrame with 100 rows and 4 columns populated with random data and then it sends it to SQL Server.\n\n```python\nimport bcpy\nimport numpy as np\nimport pandas as pd\n\n\nsql_config = {\n    'server': 'sql_server_hostname',\n    'database': 'database_name',\n    'username': 'test_user',\n    'password': 'test_user_password1234'\n}\ntable_name = 'test_dataframe'\ndf = pd.DataFrame(np.random.randint(-100, 100, size=(100, 4)),\n                  columns=list('ABCD'))\nbdf = bcpy.DataFrame(df)\nsql_table = bcpy.SqlTable(sql_config, table=table_name)\nbdf.to_sql(sql_table)\n```\n\n## Requirements\n\nYou need a working version of Microsoft bcp installed in your system. Your PATH environment variable should contain the directory of the bcp utility. Following are the installation tutorials for different operating systems.\n\n- [Dockerfile (Ubuntu 18.04)](./bcp.Dockerfile)\n- [Linux](https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-tools)\n- [Mac](https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-setup-tools?view=sql-server-2017#macos)\n- [Windows](https://docs.microsoft.com/en-us/sql/tools/bcp-utility)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftitan550%2Fbcpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftitan550%2Fbcpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftitan550%2Fbcpy/lists"}