{"id":13502621,"url":"https://github.com/catherinedevlin/ipython-sql","last_synced_at":"2025-05-14T02:04:48.702Z","repository":{"id":7555893,"uuid":"8909193","full_name":"catherinedevlin/ipython-sql","owner":"catherinedevlin","description":"%%sql magic for IPython, hopefully evolving into full SQL client","archived":false,"fork":false,"pushed_at":"2024-07-12T16:16:57.000Z","size":581,"stargazers_count":1795,"open_issues_count":115,"forks_count":373,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-05-06T10:13:45.705Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/catherinedevlin.png","metadata":{"files":{"readme":"README.rst","changelog":"NEWS.rst","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2013-03-20T16:41:10.000Z","updated_at":"2025-05-05T10:52:19.000Z","dependencies_parsed_at":"2024-01-07T12:52:17.976Z","dependency_job_id":"1734ce8a-6c80-445a-942d-e70ac624ce13","html_url":"https://github.com/catherinedevlin/ipython-sql","commit_stats":{"total_commits":205,"total_committers":56,"mean_commits":"3.6607142857142856","dds":0.673170731707317,"last_synced_commit":"eb274844b4a619463149e0d57df705e1bba47635"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catherinedevlin%2Fipython-sql","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catherinedevlin%2Fipython-sql/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catherinedevlin%2Fipython-sql/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catherinedevlin%2Fipython-sql/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/catherinedevlin","download_url":"https://codeload.github.com/catherinedevlin/ipython-sql/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253528970,"owners_count":21922635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T22:02:20.038Z","updated_at":"2025-05-14T02:04:48.666Z","avatar_url":"https://github.com/catherinedevlin.png","language":"Python","funding_links":[],"categories":["Python","Jupyter-magic拓展","CLI"],"sub_categories":[],"readme":"===========\nipython-sql\n===========\n\n:Author: Catherine Devlin, http://catherinedevlin.blogspot.com\n\nIntroduces a %sql (or %%sql) magic.\n\nLegacy project\n--------------\n\nIPython-SQL's functionality and maintenance have been eclipsed by JupySQL_, a fork maintained and developed by the Ploomber team.  Future work will be directed into JupySQL - please file issues there, as well!\n\nDescription\n-----------\n\nConnect to a database, using `SQLAlchemy URL`_ connect strings, then issue SQL\ncommands within IPython or IPython Notebook.\n\n.. image:: https://raw.github.com/catherinedevlin/ipython-sql/master/examples/writers.png\n   :width: 600px\n   :alt: screenshot of ipython-sql in the Notebook\n\nExamples\n--------\n\n.. code-block:: python\n\n    In [1]: %load_ext sql\n\n    In [2]: %%sql postgresql://will:longliveliz@localhost/shakes\n       ...: select * from character\n       ...: where abbrev = 'ALICE'\n       ...:\n    Out[2]: [(u'Alice', u'Alice', u'ALICE', u'a lady attending on Princess Katherine', 22)]\n\n    In [3]: result = _\n\n    In [4]: print(result)\n    charid   charname   abbrev                description                 speechcount\n    =================================================================================\n    Alice    Alice      ALICE    a lady attending on Princess Katherine   22\n\n    In [4]: result.keys\n    Out[5]: [u'charid', u'charname', u'abbrev', u'description', u'speechcount']\n\n    In [6]: result[0][0]\n    Out[6]: u'Alice'\n\n    In [7]: result[0].description\n    Out[7]: u'a lady attending on Princess Katherine'\n\nAfter the first connection, connect info can be omitted::\n\n    In [8]: %sql select count(*) from work\n    Out[8]: [(43L,)]\n\nConnections to multiple databases can be maintained.  You can refer to\nan existing connection by username@database\n\n.. code-block:: python\n\n    In [9]: %%sql will@shakes\n       ...: select charname, speechcount from character\n       ...: where  speechcount = (select max(speechcount)\n       ...:                       from character);\n       ...:\n    Out[9]: [(u'Poet', 733)]\n\n    In [10]: print(_)\n    charname   speechcount\n    ======================\n    Poet       733\n\nIf no connect string is supplied, ``%sql`` will provide a list of existing connections;\nhowever, if no connections have yet been made and the environment variable ``DATABASE_URL``\nis available, that will be used.\n\nFor secure access, you may dynamically access your credentials (e.g. from your system environment or `getpass.getpass`) to avoid storing your password in the notebook itself. Use the `$` before any variable to access it in your `%sql` command.\n\n.. code-block:: python\n\n    In [11]: user = os.getenv('SOME_USER')\n       ....: password = os.getenv('SOME_PASSWORD')\n       ....: connection_string = \"postgresql://{user}:{password}@localhost/some_database\".format(user=user, password=password)\n       ....: %sql $connection_string\n    Out[11]: u'Connected: some_user@some_database'\n\nYou may use multiple SQL statements inside a single cell, but you will\nonly see any query results from the last of them, so this really only\nmakes sense for statements with no output\n\n.. code-block:: python\n\n    In [11]: %%sql sqlite://\n       ....: CREATE TABLE writer (first_name, last_name, year_of_death);\n       ....: INSERT INTO writer VALUES ('William', 'Shakespeare', 1616);\n       ....: INSERT INTO writer VALUES ('Bertold', 'Brecht', 1956);\n       ....:\n    Out[11]: []\n\n\nAs a convenience, dict-style access for result sets is supported, with the\nleftmost column serving as key, for unique values.\n\n.. code-block:: python\n\n    In [12]: result = %sql select * from work\n    43 rows affected.\n\n    In [13]: result['richard2']\n    Out[14]: (u'richard2', u'Richard II', u'History of Richard II', 1595, u'h', None, u'Moby', 22411, 628)\n\nResults can also be retrieved as an iterator of dictionaries (``result.dicts()``)\nor a single dictionary with a tuple of scalar values per key (``result.dict()``)\n\nVariable substitution \n---------------------\n\nBind variables (bind parameters) can be used in the \"named\" (:x) style.\nThe variable names used should be defined in the local namespace.\n\n.. code-block:: python\n\n    In [15]: name = 'Countess'\n\n    In [16]: %sql select description from character where charname = :name\n    Out[16]: [(u'mother to Bertram',)]\n\n    In [17]: %sql select description from character where charname = '{name}' \n    Out[17]: [(u'mother to Bertram',)]\n\nAlternately, ``$variable_name`` or ``{variable_name}`` can be \nused to inject variables from the local namespace into the SQL \nstatement before it is formed and passed to the SQL engine.\n(Using ``$`` and ``{}`` together, as in ``${variable_name}``, \nis not supported.)\n\nBind variables are passed through to the SQL engine and can only \nbe used to replace strings passed to SQL.  ``$`` and ``{}`` are \nsubstituted before passing to SQL and can be used to form SQL \nstatements dynamically.\n\nAssignment\n----------\n\nOrdinary IPython assignment works for single-line `%sql` queries:\n\n.. code-block:: python\n\n    In [18]: works = %sql SELECT title, year FROM work\n    43 rows affected.\n\nThe `\u003c\u003c` operator captures query results in a local variable, and\ncan be used in multi-line ``%%sql``:\n\n.. code-block:: python\n\n    In [19]: %%sql works \u003c\u003c SELECT title, year\n        ...: FROM work\n        ...:\n    43 rows affected.\n    Returning data to local variable works\n\nConnecting\n----------\n\nConnection strings are `SQLAlchemy URL`_ standard.\n\nSome example connection strings::\n\n    mysql+pymysql://scott:tiger@localhost/foo\n    oracle://scott:tiger@127.0.0.1:1521/sidname\n    sqlite://\n    sqlite:///foo.db\n    mssql+pyodbc://username:password@host/database?driver=SQL+Server+Native+Client+11.0\n\n.. _`SQLAlchemy URL`: http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls\n\nNote that ``mysql`` and ``mysql+pymysql`` connections (and perhaps others)\ndon't read your client character set information from .my.cnf.  You need\nto specify it in the connection string::\n\n    mysql+pymysql://scott:tiger@localhost/foo?charset=utf8\n\nNote that an ``impala`` connection with `impyla`_  for HiveServer2 requires disabling autocommit::\n\n    %config SqlMagic.autocommit=False\n    %sql impala://hserverhost:port/default?kerberos_service_name=hive\u0026auth_mechanism=GSSAPI\n\n.. _impyla: https://github.com/cloudera/impyla\n\nConnection arguments not whitelisted by SQLALchemy can be provided as\na flag with (-a|--connection_arguments)the connection string as a JSON string.\nSee `SQLAlchemy Args`_.\n\n    | %sql --connection_arguments {\"timeout\":10,\"mode\":\"ro\"} sqlite:// SELECT * FROM work;\n    | %sql -a '{\"timeout\":10, \"mode\":\"ro\"}' sqlite:// SELECT * from work;\n\n.. _`SQLAlchemy Args`: https://docs.sqlalchemy.org/en/13/core/engines.html#custom-dbapi-args\n\nDSN connections\n~~~~~~~~~~~~~~~\n\nAlternately, you can store connection info in a \nconfiguration file, under a section name chosen to \nrefer to your database.\n\nFor example, if dsn.ini contains \n\n    | [DB_CONFIG_1] \n    | drivername=postgres \n    | host=my.remote.host \n    | port=5433 \n    | database=mydatabase \n    | username=myuser \n    | password=1234\n\nthen you can  \n\n    | %config SqlMagic.dsn_filename='./dsn.ini'\n    | %sql --section DB_CONFIG_1 \n\nConfiguration\n-------------\n\nQuery results are loaded as lists, so very large result sets may use up\nyour system's memory and/or hang your browser.  There is no autolimit\nby default.  However, `autolimit` (if set) limits the size of the result\nset (usually with a `LIMIT` clause in the SQL).  `displaylimit` is similar,\nbut the entire result set is still pulled into memory (for later analysis);\nonly the screen display is truncated.\n\n.. code-block:: python\n\n   In [2]: %config SqlMagic\n   SqlMagic options\n   --------------\n   SqlMagic.autocommit=\u003cBool\u003e\n       Current: True\n       Set autocommit mode\n   SqlMagic.autolimit=\u003cInt\u003e\n       Current: 0\n       Automatically limit the size of the returned result sets\n   SqlMagic.autopandas=\u003cBool\u003e\n       Current: False\n       Return Pandas DataFrames instead of regular result sets\n   SqlMagic.column_local_vars=\u003cBool\u003e\n       Current: False\n       Return data into local variables from column names\n   SqlMagic.displaycon=\u003cBool\u003e\n       Current: False\n       Show connection string after execute\n   SqlMagic.displaylimit=\u003cInt\u003e\n       Current: None\n       Automatically limit the number of rows displayed (full result set is still\n       stored)\n   SqlMagic.dsn_filename=\u003cUnicode\u003e\n       Current: 'odbc.ini'\n       Path to DSN file. When the first argument is of the form [section], a\n       sqlalchemy connection string is formed from the matching section in the DSN\n       file.\n   SqlMagic.feedback=\u003cBool\u003e\n       Current: False\n       Print number of rows affected by DML\n   SqlMagic.short_errors=\u003cBool\u003e\n       Current: True\n       Don't display the full traceback on SQL Programming Error\n   SqlMagic.style=\u003cUnicode\u003e\n       Current: 'DEFAULT'\n       Set the table printing style to any of prettytable's defined styles\n       (currently DEFAULT, MSWORD_FRIENDLY, PLAIN_COLUMNS, RANDOM)\n\n   In[3]: %config SqlMagic.feedback = False\n\nPlease note: if you have autopandas set to true, the displaylimit option will not apply. You can set the pandas display limit by using the pandas ``max_rows`` option as described in the `pandas documentation \u003chttp://pandas.pydata.org/pandas-docs/version/0.18.1/options.html#frequently-used-options\u003e`_.\n\nPandas\n------\n\nIf you have installed ``pandas``, you can use a result set's\n``.DataFrame()`` method\n\n.. code-block:: python\n\n    In [3]: result = %sql SELECT * FROM character WHERE speechcount \u003e 25\n\n    In [4]: dataframe = result.DataFrame()\n\n\nThe ``--persist`` argument, with the name of a \nDataFrame object in memory, \nwill create a table name\nin the database from the named DataFrame.  \nOr use ``--append`` to add rows to an existing \ntable by that name.\n\n.. code-block:: python\n\n    In [5]: %sql --persist dataframe\n\n    In [6]: %sql SELECT * FROM dataframe;\n\n.. _Pandas: http://pandas.pydata.org/\n\nGraphing\n--------\n\nIf you have installed ``matplotlib``, you can use a result set's\n``.plot()``, ``.pie()``, and ``.bar()`` methods for quick plotting\n\n.. code-block:: python\n\n    In[5]: result = %sql SELECT title, totalwords FROM work WHERE genretype = 'c'\n\n    In[6]: %matplotlib inline\n\n    In[7]: result.pie()\n\n.. image:: https://raw.github.com/catherinedevlin/ipython-sql/master/examples/wordcount.png\n   :alt: pie chart of word count of Shakespeare's comedies\n\nDumping\n-------\n\nResult sets come with a ``.csv(filename=None)`` method.  This generates\ncomma-separated text either as a return value (if ``filename`` is not\nspecified) or in a file of the given name.\n\n.. code-block:: python\n\n    In[8]: result = %sql SELECT title, totalwords FROM work WHERE genretype = 'c'\n\n    In[9]: result.csv(filename='work.csv')\n\nPostgreSQL features\n-------------------\n\n``psql``-style \"backslash\" `meta-commands`_ commands (``\\d``, ``\\dt``, etc.)\nare provided by `PGSpecial`_.  Example:\n\n.. code-block:: python\n\n    In[9]: %sql \\d\n\n.. _PGSpecial: https://pypi.python.org/pypi/pgspecial\n\n.. _meta-commands: https://www.postgresql.org/docs/9.6/static/app-psql.html#APP-PSQL-META-COMMANDS\n\n\nOptions\n-------\n\n``-l`` / ``--connections``\n    List all active connections\n\n``-x`` / ``--close \u003csession-name\u003e`` \n    Close named connection \n\n``-c`` / ``--creator \u003ccreator-function\u003e``\n    Specify creator function for new connection\n\n``-s`` / ``--section \u003csection-name\u003e``\n    Section of dsn_file to be used for generating a connection string\n\n``-p`` / ``--persist``\n    Create a table name in the database from the named DataFrame\n\n``--append``\n    Like ``--persist``, but appends to the table if it already exists \n\n``-a`` / ``--connection_arguments \u003c\"{connection arguments}\"\u003e``\n    Specify dictionary of connection arguments to pass to SQL driver\n\n``-f`` / ``--file \u003cpath\u003e``\n    Run SQL from file at this path\n\nCaution \n-------\n\nComments\n~~~~~~~~\n\nBecause ipyton-sql accepts ``--``-delimited options like ``--persist``, but ``--`` \nis also the syntax to denote a SQL comment, the parser needs to make some assumptions.\n\n- If you try to pass an unsupported argument, like ``--lutefisk``, it will \n  be interpreted as a SQL comment and will not throw an unsupported argument \n  exception.\n- If the SQL statement begins with a first-line comment that looks like one \n  of the accepted arguments - like ``%sql --persist is great!`` - it will be \n  parsed like an argument, not a comment.  Moving the comment to the second \n  line or later will avoid this.\n\nInstalling\n----------\n\nInstall the latest release with::\n\n    pip install ipython-sql\n\nor download from https://github.com/catherinedevlin/ipython-sql and::\n\n    cd ipython-sql\n    sudo python setup.py install\n\nDevelopment\n-----------\n\nhttps://github.com/catherinedevlin/ipython-sql\n\nCredits\n-------\n\n- Matthias Bussonnier for help with configuration\n- Olivier Le Thanh Duong for ``%config`` fixes and improvements\n- Distribute_\n- Buildout_\n- modern-package-template_\n- Mike Wilson for bind variable code\n- Thomas Kluyver and Steve Holden for debugging help\n- Berton Earnshaw for DSN connection syntax\n- Bruno Harbulot for DSN example \n- Andrés Celis for SQL Server bugfix\n- Michael Erasmus for DataFrame truth bugfix\n- Noam Finkelstein for README clarification\n- Xiaochuan Yu for `\u003c\u003c` operator, syntax colorization\n- Amjith Ramanujam for PGSpecial and incorporating it here\n- Alexander Maznev for better arg parsing, connections accepting specified creator\n- Jonathan Larkin for configurable displaycon \n- Jared Moore for ``connection-arguments`` support\n- Gilbert Brault for ``--append`` \n- Lucas Zeer for multi-line bugfixes for var substitution, ``\u003c\u003c`` \n- vkk800 for ``--file``\n- Jens Albrecht for MySQL DatabaseError bugfix\n- meihkv for connection-closing bugfix\n- Abhinav C for SQLAlchemy 2.0 compatibility\n\n.. _Distribute: http://pypi.python.org/pypi/distribute\n.. _Buildout: http://www.buildout.org/\n.. _modern-package-template: http://pypi.python.org/pypi/modern-package-template\n.. _JupySQL: https://github.com/ploomber/jupysql\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatherinedevlin%2Fipython-sql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcatherinedevlin%2Fipython-sql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatherinedevlin%2Fipython-sql/lists"}