{"id":19766225,"url":"https://github.com/akaliutau/python-recipes","last_synced_at":"2026-05-12T08:32:21.131Z","repository":{"id":49406911,"uuid":"354209374","full_name":"akaliutau/python-recipes","owner":"akaliutau","description":"Examples of Python code to solve common problems in web development, data processing, etc","archived":false,"fork":false,"pushed_at":"2021-04-16T11:33:59.000Z","size":485,"stargazers_count":1,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-28T10:55:24.572Z","etag":null,"topics":["django","oauth2-authorization","python3","rest-api","rpa","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/akaliutau.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-04-03T05:36:49.000Z","updated_at":"2022-01-13T07:50:37.000Z","dependencies_parsed_at":"2022-09-17T00:02:13.536Z","dependency_job_id":null,"html_url":"https://github.com/akaliutau/python-recipes","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/akaliutau/python-recipes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akaliutau%2Fpython-recipes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akaliutau%2Fpython-recipes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akaliutau%2Fpython-recipes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akaliutau%2Fpython-recipes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/akaliutau","download_url":"https://codeload.github.com/akaliutau/python-recipes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/akaliutau%2Fpython-recipes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32930494,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-11T17:09:15.040Z","status":"online","status_checked_at":"2026-05-12T02:00:06.338Z","response_time":102,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","oauth2-authorization","python3","rest-api","rpa","web-scraping"],"created_at":"2024-11-12T04:23:27.312Z","updated_at":"2026-05-12T08:32:21.116Z","avatar_url":"https://github.com/akaliutau.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"About\n======\nThis is a collection of recipes to create different web apps components on the basis of Python 3.9\n\nTopics covered so far:\n\n* RPA (web scraping)\n* OAuth2 authorization\n* Consuming public web APIs\n* Microservices in Python\n* Asynchronous messaging in Python\n* Serverless apps in Python (TBA)\n\n\nOverview\n=========\n\n# Preliminary steps\n\n1) First check the version - it should be 3.9.x (3.9.1 in our case)\n\n```\npython --version\n\nPython 3.9.1\n```\n\n2) Create a virtual environment\n\n```\npython -m venv env\n```\nAs a result a folder env with python libs will be created\n\n3) Activate virtual env:\n\n```\n.\\env\\Scripts\\activate\n```\nThe promt will be changed to (env)\n\n4) Install necessary dependencies into virtual environment:\n\n```\npip install -r requirements.txt\n```\nAs a result all extra dependencies will be added\n\n\n# RPA (web scraping): Weather app\n\nThis is a simple web-scraping console application\n\n\n1) One can perform testing in two ways - using standard browser with Chrome driver or the headless one. As for the latter, one can use PhantomJS (https://phantomjs.org/download.html).\n\nIn this project we are using PhantomJS. Download and install it into some directory, f.e. phantom\n\n6) Execute the following command in console:\n\n```\npython -m weatherapp -p WeatherComParser -u Celsius -a ae8230efd4bc57fdf721a02c7eb2b88c56aa6e71d73666328e33af3ea2039032132e24ae91b6a07862c5091a9d95a4b8 -td\n```\n\n\nThe output should look like:\n\n```\nusing sub type: _today_forecast\nhttp://weather.com/weather/today/l/ae8230efd4bc57fdf721a02c7eb2b88c56aa6e71d73666328e33af3ea2039032132e24ae91b6a07862c5091a9d95a4b8\nretrieved raw data 899273 bytes\n{'location': ['London, England, United Kingdom Weather'], 'unit': ['°F'], 'temperature': ['51°'], 'phase': ['Partly Cloudy']}\n```\nArea codes for locations can be found at https://weather.com\n\n# OAuth2 authorization\n\nThis is a simple app demonstrating the work with real OAuth2-secured services on the basis of Spotify\nNOTE: it will need curses library to be installed, for Windows install it manually:\n\n```\npip install windows-curses\n```\n\n1) Create an account on Spotify:\nhttps://developer.spotify.com/dashboard\n   \nSee screenshots 1 and 2\n   \n2) In a file called config.yaml in the spotify directory add client_id and client_secret are the keys that were created for us when we created the\nSpotify application. These keys will be used to get an access token that we will have to\nacquire every time we need to send a new request to Spotify's REST API.\n\n```\nclient_id: '\u003cyour client ID\u003e'\nclient_secret: '\u003cyour client secret\u003e'\naccess_token_url: 'https://accounts.spotify.com/api/token'\nauth_url: 'http://accounts.spotify.com/authorize'\napi_version: 'v1'\napi_url: 'https://api.spotify.com'\nauth_method: 'AUTHORIZATION_CODE'\n```\n3) Run auth code to get a token (note way the module is executed - the FQN path must be specified)\n\n```\npython -m spotify.spotify_auth\n```\nafter successful authorization the server will catch the token and persist it into the file .jukebox in the current directory\n\n4) Run main application to search information about artists and albums:\n\n```\npython -m spotify.app\n```\n# Consuming public API\n\n# Exchange Rates, Currency Conversion Tool\n\nThis is a super simple project which is using foxit.io service to get currency exchange rates and MongoDB as \nthe intermediate history persistent layer - basically it just plays the role of cache\n\n1) This project requires the MongoDB instance, so the first step will be to create it using docker-compose file in root directory:\n\n```\ndocker-compose -f docker-compose-min.yml up --build\n```\n\n2) Open account on fixer.io\nThis project is yet another example of consuming API. \nFor this one the fixer API is used (https://github.com/fixerAPI/fixer#readme)\nThere is an option to get a free api token, see the details on https://fixer.io/\n\nThe screenshot on fixer_1.png shows how the page to open new account looks like\n\n3) Run the app:\n\n```\npython -m forexapp --from EUR --to RUB --value 200 --token \u003cyour token\u003e\n\nconverting 200.0 units from EUR-\u003eRUB\nFetching exchange rates from fixer.io [base currency: EUR]\n200.0 EUR = 17866.24 RUB\n```\n\n4) One can query DB against collected rates to see the cached values:\n\n```\ndb.rates.find({})\n   .projection({})\n   .sort({_id:-1})\n   .limit(100)\n```\n\nThe result will be similar to this one (rates field was truncated for brevity):\n\n```\n{\n\t\"_id\" : ObjectId(\"6065bae4716e1877db2e3c39\"),\n\t\"base\" : \"EUR\",\n\t\"date\" : \"2021-04-01\",\n\t\"rates\" : {\n\t\t\"AED\" : 4.310488,\n\t\t\"AFN\" : 91.902225,\n\t\t\"ALL\" : 123.004027,\n\t\t\"USD\" : 1.173573,\n\t\t\"UYU\" : 52.083039,\n\t\t\"UZS\" : 12307.312892,\n\t\t\"VEF\" : 217051133237.23477,\n\t\t\"ZMW\" : 25.924749,\n\t\t\"ZWL\" : 377.891029\n\t},\n\t\"success\" : true,\n\t\"timestamp\" : 1617277386\n}\n\n```\nCache eviction rule is very simple:\n\u003cb\u003e if the date of record in cache is too old (previous day or earlier) fresh data will be requested \u003c/b\u003e\n\n# Microservices\n\n## A classical microservice with lightweight db layer on the basis of SQLight\n\nThe module soa.microservices contains the simple implementation of Order microservice with the following endpoints:\n\n```\n\n```\nOfficial Django documentation on microservices can the found here https://docs.djangoproject.com/en/3.2/intro/tutorial01/\n\n\nNote the typical Django structure for project:\n```\norder/\n\n    main/\n       migrations/\n           __init__\n           \u003cmicroservice implementation\u003e\n      \n    order/\n       __init__\n       settings.py\n       urls.py\n       wsgi.py\n      \n    manage.py\n\n```\n\nIn order to create all tables and add the initial auth records to internal db use the step 0:\n\n0) A preliminary step: generate all necessary code using the commands:\n\ncreate a package migrations inside your app directory (main in this case)\n\ncreate a migration tasks\n```\npython -m soa.microservices.order.manage makemigrations\n```\nperform migrations\n```\npython -m soa.microservices.order.manage migrate\n```\nThe result should look like this one:\n\n```\ncurrent directory D:\\repos-research\\python-recipes\nsystem paths ['D:\\\\repos-research\\\\python-recipes', 'C:\\\\ProgramData\\\\python391\\\\python39.zip', 'C:\\\\ProgramData\\\\python391\\\\DLLs', 'C:\\\\ProgramData\\\\python391\\\\lib', 'C:\\\\ProgramData\\\\python391', 'C:\\\\ProgramData\\\\python391\\\\lib\\\\site\n-packages', 'D:\\\\repos-research\\\\python-recipes', 'D:\\\\repos-research\\\\python-recipes\\\\soa\\\\microservices\\\\order']\nOperations to perform:\n  Apply all migrations: admin, auth, authtoken, contenttypes, main, sessions\nRunning migrations:\n  Applying contenttypes.0001_initial... OK\n  Applying auth.0001_initial... OK\n  Applying admin.0001_initial... OK\n  Applying admin.0002_logentry_remove_auto_add... OK\n  Applying contenttypes.0002_remove_content_type_name... OK\n  Applying auth.0002_alter_permission_name_max_length... OK\n  Applying auth.0003_alter_user_email_max_length... OK\n  Applying auth.0004_alter_user_username_opts... OK\n  Applying auth.0005_alter_user_last_login_null... OK\n  Applying auth.0006_require_contenttypes_0002... OK\n  Applying auth.0007_alter_validators_add_error_messages... OK\n  Applying auth.0008_alter_user_username_max_length... OK\n  Applying auth.0009_alter_user_last_name_max_length... OK\n  Applying authtoken.0001_initial... OK\n  Applying authtoken.0002_auto_20160226_1747... OK\n  Applying main.0001_initial... OK\n  Applying sessions.0001_initial... OK\n\n```\n1) Create an initial user (admin):\n\n\n```\npython -m soa.microservices.order.manage createsuperuser\n\nUsername (leave blank to use 'akaliutau'): admin\nEmail address: test@test.org\nPassword:\nPassword (again):\n\nSuperuser created successfully.\n\n```\n\n2) To start the service with the following command (by some reason it does not start as a module):\n\n```\npython .\\soa\\microservices\\order\\manage.py runserver\n\ncur directory D:\\repos-research\\python-recipes\nPerforming system checks...\n\nSystem check identified no issues (0 silenced).\n\nApril 09, 2021 - 10:01:19\nDjango version 3.2, using settings 'soa.microservices.order.order.settings'\nStarting development server at http://127.0.0.1:8000/\n\n```\nThe first run will create a db.sqlite3 database with necessary tables.\n\nIf the system log contains the output like this one:\n```\nYou have 17 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, authtoken, contenttypes, main, sessions.\nRun 'python manage.py migrate' to apply them.\n```\nthen perform migration in accordance with step 0 and step 1\n\n\n\n3) Point browser to https://localhost:8000/admin, login.\n\nThen click on Add and create a user with the username test_api. \n\nWhen the user is created, create an API token using AUTH TOKEN section.\nAssign this token to a newly created user: select the test_api in the drop-down menu and click SAVE. \n\n4) Finally one can test built microservice using the simple test app in test_order.py:\n\n```\npython -m soa.microservices.order.test_order   --token 83622fab3429404dccd8da65ba97468a0306551e\n\n```\nIn server's log one can observe that server has responded with 201 status:\n\n```\ncreated Order object (2)\n[09/Apr/2021 17:01:35] \"POST /api/order/add/ HTTP/1.1\" 201 14\n\n```\n\n## A microservice built with the help of Nameko framework\n\nThis is a simple echo microservice is built using a sophisticated (but simple in use) framework - Nameko, \nRPC connectivity and Redis Data Store as a persistence layer \n\n\n1) start Docker using configuration specified in Dockerfile\nNote the default user created for RabbitMQ - these creds must be duplicated in config.yaml\n   \n```\ndocker build -t rabbitmq:1.0 .\ndocker run -d -p 5672:5672 rabbitmq:1.0\n\ndocker run -d -p 6379:6379 --name redis redis\n```\n\nAlternatively one can use a docker-compose-asynch-msg.yml file:\n\n```\ndocker-compose -f docker-compose-asynch-msg.yml up --build\n```\n\n2) To run service, execute the following command:\n\n```\nnameko run messenger.core.service --config .\\messenger\\config.yaml\n```\nThis command will result in the following output:\n\n```\nnameko run messenger.core.service --config .\\messenger\\config.yaml\nstarting services: nameko_message_service, web_server\nConnected to amqp://user:**@127.0.0.1:5672//\nConnected to amqp://user:**@127.0.0.1:5672//\n\n```\n\n3) In order to make our own calls, we can launch a Python shell that has Nameko integrated to allow us to call our entrypoints. \n   To access it, open a new terminal window and execute the following command:\n   \n```\nnameko shell --broker pyamqp://user:user@localhost:5672\n```\n\nThis should give you access to a Python shell with the ability to make Remote Procedure Calls. \n\n```\n\u003e\u003e\u003e n.rpc.nameko_message_service.save_message(\"this is the first message\")\n'f5becb1e408e455cad3b479681e541ff'\n\u003e\u003e\u003e n.rpc.nameko_message_service.get_all_messages()\n[{'id': 'f5becb1e408e455cad3b479681e541ff', 'message': 'this is the first message', 'expires_in': 6246}]\n\u003e\u003e\u003e\n\n```\n\n\n\nRequirements\n=============\n\nIn order to execute the code the following toolkit is needed:\n\n- Virtualenv\n- Python 3.9\n- pgAdmin\n- Docker\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakaliutau%2Fpython-recipes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakaliutau%2Fpython-recipes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakaliutau%2Fpython-recipes/lists"}