{"id":13429378,"url":"https://github.com/python-ruia/ruia-motor","last_synced_at":"2026-03-27T04:35:47.468Z","repository":{"id":57463153,"uuid":"170631513","full_name":"python-ruia/ruia-motor","owner":"python-ruia","description":"A Ruia plugin that uses the motor to store data to MongoDB","archived":false,"fork":false,"pushed_at":"2022-06-30T08:05:15.000Z","size":12,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-18T06:03:23.121Z","etag":null,"topics":["motor","ruia"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/python-ruia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-02-14T05:15:32.000Z","updated_at":"2022-06-30T08:05:19.000Z","dependencies_parsed_at":"2022-09-12T13:40:46.779Z","dependency_job_id":null,"html_url":"https://github.com/python-ruia/ruia-motor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/python-ruia%2Fruia-motor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/python-ruia%2Fruia-motor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/python-ruia%2Fruia-motor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/python-ruia%2Fruia-motor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/python-ruia","download_url":"https://codeload.github.com/python-ruia/ruia-motor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243822312,"owners_count":20353496,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["motor","ruia"],"created_at":"2024-07-31T02:00:36.164Z","updated_at":"2025-12-14T15:05:53.747Z","avatar_url":"https://github.com/python-ruia.png","language":"Python","readme":"# ruia-motor\n\nA [Ruia](https://github.com/howie6879/ruia) plugin that uses the motor to store data\n\n\n```text\nNotice:  Works on ruia \u003e= 0.8.0\n```\n\n### Installation\n\n```shell\npip install -U ruia-motor\n```\n\n### Usage\n\n`ruia-motor` will be automatically store data to mongodb:\n\n```python\nfrom ruia import AttrField, Item, Response, Spider, TextField\n\nfrom ruia_motor import RuiaMotorInsert, RuiaMotorUpdate, init_spider\n\n\nclass HackerNewsItem(Item):\n    target_item = TextField(css_select=\"tr.athing\")\n    title = TextField(css_select=\"a.storylink\")\n    url = AttrField(css_select=\"a.storylink\", attr=\"href\")\n\n    async def clean_title(self, value):\n        return value.strip()\n\n\nclass HackerNewsSpider(Spider):\n    start_urls = [\"https://news.ycombinator.com/news?p=1\"]\n    aiohttp_kwargs = {\"proxy\": \"http://0.0.0.0:1087\"}\n\n    async def parse(self, response: Response):\n        async for item in HackerNewsItem.get_items(html=await response.text()):\n            # Update data\n            # https://motor.readthedocs.io/en/stable/api-asyncio/asyncio_motor_collection.html#motor.motor_asyncio.AsyncIOMotorCollection.update_one\n            yield RuiaMotorUpdate(\n                collection=\"hn_demo\",\n                filter={\"title\": item.title},\n                update={\"$set\": item.results},\n                upsert=True,\n            )\n            # Insert data\n            # https://motor.readthedocs.io/en/stable/api-asyncio/asyncio_motor_collection.html#motor.motor_asyncio.AsyncIOMotorCollection.insert_one\n            # yield RuiaMotorInsert(collection=\"hn_demo\", data=item.results)\n\n\nasync def init_plugins_after_start(spider_ins):\n    spider_ins.mongodb_config = {\"host\": \"127.0.0.1\", \"port\": 27017, \"db\": \"ruia_motor\"}\n    init_spider(spider_ins=spider_ins)\n\n\nif __name__ == \"__main__\":\n    HackerNewsSpider.start(after_start=init_plugins_after_start)\n```\n\nEnjoy it :)","funding_links":[],"categories":["Uncategorized"],"sub_categories":["Uncategorized"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpython-ruia%2Fruia-motor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpython-ruia%2Fruia-motor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpython-ruia%2Fruia-motor/lists"}