{"id":25961877,"url":"https://github.com/nameko/nameko-sqlalchemy","last_synced_at":"2025-03-04T19:50:05.969Z","repository":{"id":27400664,"uuid":"30877119","full_name":"nameko/nameko-sqlalchemy","owner":"nameko","description":"SQLAlchemy dependency for nameko services","archived":false,"fork":false,"pushed_at":"2024-05-21T19:57:34.000Z","size":123,"stargazers_count":71,"open_issues_count":7,"forks_count":21,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-02-25T21:56:20.423Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nameko.png","metadata":{"files":{"readme":"README.rst","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-02-16T16:33:57.000Z","updated_at":"2025-02-20T15:51:23.000Z","dependencies_parsed_at":"2024-06-18T22:51:28.666Z","dependency_job_id":"f795e57b-7460-49cb-b7b3-87962ce1e348","html_url":"https://github.com/nameko/nameko-sqlalchemy","commit_stats":null,"previous_names":["onefinestay/nameko-sqlalchemy"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nameko%2Fnameko-sqlalchemy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nameko%2Fnameko-sqlalchemy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nameko%2Fnameko-sqlalchemy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nameko%2Fnameko-sqlalchemy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nameko","download_url":"https://codeload.github.com/nameko/nameko-sqlalchemy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241913762,"owners_count":20041461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-04T19:50:04.026Z","updated_at":"2025-03-04T19:50:05.925Z","avatar_url":"https://github.com/nameko.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"nameko-sqlalchemy\n=================\n\nDependencyProviders and utilities for `nameko \u003chttp://http://nameko.readthedocs.org\u003e`_ services to interface with a relational database using SQLAlchemy.\n\n\nUsage\n-----\n\n.. code-block:: python\n\n    from nameko_sqlalchemy import Database\n\n    from .models import Model, DeclarativeBase\n\n    class Service(object):\n        name = \"service\"\n\n        db = Database(DeclarativeBase)\n\n        @entrypoint\n        def write_to_db(self):\n            model = Model(...)\n            with self.db.get_session() as session:\n                session.add(model)\n\n        @entrypoint\n        def query_db(self):\n            queryset = self.db.session.query(Model).filter(...)\n            ...\n\n\nThe ``nameko_sqlalchemy.Database`` DependencyProvider can be used in three ways:\n\nAs a context manager that issues a commit or rollback on exit:\n\n.. code-block:: python\n\n    @entrypoint\n    def method(self):\n        with self.db.get_session() as session:\n            session.add(model)\n\nTo explicitly retrieve sessions that can be manually manipulated:\n\n.. code-block:: python\n\n    @entrypoint\n    def method(self):\n        session1 = self.db.get_session()\n        session1.add(...)\n        session2 = self.db.get_session()\n        session2.add(...)\n        session1.commit()\n        session1.close()\n        session2.commit()\n        session2.close()\n\n\nTo manage a session that is lazily opened on first use and closed when the Nameko entrypoint exits:\n\n\n.. code-block:: python\n\n    @entrypoint\n    def method(self):\n        self.db.session.add(...)\n        self.db.session.commit()\n\n\nThe ``nameko_sqlalchemy.DatabaseSession`` DependencyProvider maintains the original interface from the early versions of the library. It behaves similarly to the third example above, except that the session is opened before the entrypoint fires, rather than lazily.\n\n\n.. code-block:: python\n\n    class Service:\n        name = \"legacy\"\n\n        session = DatabaseSession()\n\n        @entrypoint\n        def method(self):\n            self.session.add(...)\n            self.session.commit()\n\n\n\nDatabase drivers\n----------------\n\nYou may use any database `driver compatible with SQLAlchemy \u003chttp://docs.sqlalchemy.org/en/rel_0_9/dialects/index.html\u003e`_ provided it is safe to use with `eventlet \u003chttp://eventlet.net\u003e`_. This will include all pure-python drivers. Known safe drivers are:\n\n* `pysqlite \u003chttp://docs.sqlalchemy.org/en/rel_0_9/dialects/sqlite.html#module-sqlalchemy.dialects.sqlite.pysqlite\u003e`_\n* `pymysql \u003chttp://docs.sqlalchemy.org/en/rel_0_9/dialects/mysql.html#module-sqlalchemy.dialects.mysql.pymysql\u003e`_\n\n\nDecorators\n----------\n\ntransaction_retry\n^^^^^^^^^^^^^^^^^\nThis decorator automatically retries the wrapped function when a database connection error occurs.\nIf the optional ``session`` argument is passed it will issue a rollback on it before retrying so the transaction can be processed again.\nThe ``session`` argument can either be the ``sqlalchemy.orm.session.Session`` or an ``operator.attrgetter`` object if the session is a class attribute.\n\n\nUsage\n\"\"\"\"\"\n\n.. code-block:: python\n\n    from sqlalchemy import create_engine\n    from sqlalchemy.orm import sessionmaker\n\n    from nameko_sqlalchemy import transaction_retry\n\n\n    engine = create_engine('postgresql://username:password@localhost/test')\n    Session = sessionmaker(bind=engine)\n    db_session = Session()\n\n    @transaction_retry()\n    def get_example_data():\n        db_session.query(ExampleModel).all()\n\n    example_data = get_example_data()\n\n\nor using with the ``Database`` dependency provider\n\n.. code-block:: python\n\n    from sqlalchemy.ext.declarative import declarative_base\n    from nameko_sqlalchemy import Database, transaction_retry\n\n\n    DeclBase = declarative_base(name='examplebase')\n\n\n    class ExampleService:\n        name = 'exampleservice'\n\n        db = Database(DeclBase)\n\n        @entrypoint\n        @transaction_retry\n        def get_examples(self):\n            with self.db.get_session() as session:\n                return session.query(ExampleModel).all()\n\n        @entrypoint\n        def get_examples_with_retry_inside(self):\n            with self.db.get_session() as session:\n                @transaction_retry(session=session)\n                def foo():\n                    return session.query(ExampleModel).all()\n\n                return foo()\n\n        @entrypoint\n        @transaction_retry\n        def create_example_without_using_context_manager(self):\n            session = self.db.get_session()\n            session.add(ExampleModel(data='hello'))\n            session.commit()\n\n        @entrypoint\n        @transaction_retry(session=operator.attrgetter('db.session'))\n        def create_example_with_worker_scoped_session(self):\n            self.db.session.add(ExampleModel(data='hello'))\n            self.db.session.commit()\n\nOptionally, the transaction may be retried multiple times with an exponential backoff delay.\nIf given, the backoff factor is applied between attempts after the second try\n(most errors are resolved immediately by a second try without a delay).\nThe applied delay will then be::\n\n    {backoff_factor} * (2 ** ({number of total retries} - 1))\n\nseconds.\nThe delay will never be longer than `backoff_max` seconds.\nBy default, backoff is disabled (`backoff_factor` set to 0).\nFinally, if the connection has still not recovered after `total` tries, the error is reraised.\nThe following code will thus wait for 0.0s, 0.2s, 0.4s, 0.8s, 1.0s before raising an error:\n\n.. code-block:: python\n\n    @transaction_retry(total=5, backoff_factor=0.1, backoff_max=1.0)\n    def get_example_data():\n        db_session.query(ExampleModel).all()\n\n    example_data = get_example_data()\n\n\n.. caution::\n\n    Using the decorator may cause unanticipated consequences when the decorated function uses more than one transaction.\n\nIt should only be used around single transactions because all transactions inside the decorator will be re-executed if there is a connection error during any of them. Take a look at the following example:\n\n.. code-block:: python\n\n    class ExampleService:\n\n        db = Database(DeclBase)\n\n        @entrypoint\n        @transaction_retry\n        def method(self):\n            with self.db.get_session() as session:\n                session.add(something)\n\n            do_something()  # during this a network error occurs\n\n            with self.db.get_session() as session:\n                session.add(something_else)  # throws error because the db connection is gone, method will be executed again\n\n\nSince the method is retried all of the statements are executed twice, including the ones that didn't fail. As a result of that ``something`` will be added twice.\nIn order to avoid that one may want to do something like this:\n\n.. code-block:: python\n\n    class ExampleService:\n\n        db = Database(DeclBase)\n\n        @entrypoint\n        def method(self):\n            with self.db.get_session() as session:\n                @transaction_retry(session=session)\n                def add_two_things():\n                    session.add(something)\n                    do_something()\n                    session.add(something_else)\n\n                add_two_things()\n\nIn this case the failed transaction will be rolled back (because the session is passed to the decorator) and records will not be duplicated.\n\nPytest fixtures\n---------------\n\nPytest fixtures to allow for easy testing are available.\n\n* ``db_session`` fixture (which depends on ``db_connection`` fixture) will instantiate test database and tear it down at the end of each test.\n* ``model_base`` fixture can be overridden to provide custom ``declarative_base``.\n* ``db_engine_options`` fixture can be overriden to provide additional keyword arguments to ``sqlalchemy.create_engine``.\n* ``database`` fixture which is similar to ``db_session`` but can be passed as ``Database`` dependency replacement\n  when using ``worker_factory`` or ``replace_dependencies``.\n\n\n.. code-block:: python\n\n    import pytest\n\n    from sqlalchemy import Column, Integer, String\n    from sqlalchemy.ext.declarative import declarative_base\n\n\n    class Base(object):\n        pass\n\n\n    DeclarativeBase = declarative_base(cls=Base)\n\n\n    class User(DeclarativeBase):\n        __tablename__ = \"users\"\n\n        id = Column(Integer, primary_key=True)\n        name = Column(String)\n\n\n    @pytest.fixture(scope='session')\n    def model_base():\n        return DeclarativeBase\n\n\n    @pytest.fixture(scope='session')\n    def db_engine_options():\n        return dict(client_encoding='utf8')\n\n\n    def test_users(db_session):\n        user = User(id=1, name='Joe')\n        db_session.add(user)\n        db_session.commit()\n        saved_user = db_session.query(User).get(user.id)\n        assert saved_user.id \u003e 0\n        assert saved_user.name == 'Joe'\n\nWhen running tests you can pass database test url with ``--test-db-url`` parameter or override ``db_url`` fixture.\nBy default SQLite memory database will be used.\n\n.. code-block:: shell\n\n    pytest test --test-db-url=sqlite:///test_db.sql\n    pytest test --test-db-url=mysql+mysqlconnector://root:password@localhost:3306/nameko_sqlalchemy_test\n\n\nRunning the tests\n-----------------\n\nPrerequisites\n^^^^^^^^^^^^^\n\nSome of the tests use `toxiproxy \u003chttps://github.com/Shopify/toxiproxy\u003e`_ to simulate network errors. In order to be able to run those tests you need a toxiproxy server to be in place. You may install it manually or by running the following command (docker is required):\n\n.. code-block:: shell\n\n    make test-deps\n\nThis will setup a mysql and a toxiproxy server with a proxy set up to the database.\n\n\nRunning tests by using docker\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nOnce the containers have been set up the tests can be run by running the following command:\n\n.. code-block:: shell\n\n    make test\n\n\nRunning tests by using pytest command\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThree extra parameters may be passed to `pytest`:\n\n* ``test-db-url``: The database URL\n* ``toxiproxy-api-url``: The url of the Toxiproxy HTTP API\n* ``toxiproxy-db-url``: The url of the database through Toxiproxy\n\nIf ``toxiproxy-api-url`` and ``toxiproxy-db-url`` parameters are provided the tests assume that the toxiproxy endpoint is already set up to a database upstream and this proxy can be disabled and enabled via the HTTP API of toxiproxy.\n\n.. code-block:: shell\n\n    pytest test \\\n        --test-db-url=\"mysql+pymysql://test_user:password@database_host:3306/nameko_sqlalchemy_test\" \\\n        --toxiproxy-api-url=\"http://toxiproxy_server:8474\"\n        --toxiproxy-db-url=\"http://toxiproxy_server:3306\"\n\nif no ``toxiproxy-api-url`` and ``toxiproxy-db-url`` parameter was provided the tests that require toxiproxy will be skipped.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnameko%2Fnameko-sqlalchemy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnameko%2Fnameko-sqlalchemy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnameko%2Fnameko-sqlalchemy/lists"}