{"id":29182635,"url":"https://github.com/jessielw/tmdb-service","last_synced_at":"2025-07-01T20:07:03.624Z","repository":{"id":299853043,"uuid":"1004421185","full_name":"jessielw/TMDB-Service","owner":"jessielw","description":"Mirrors TMDB API (most of it) to cache locally for use with your own services. Setup to work with Postgres. Automatically keeps data 1:1 based on CRON tasks.","archived":false,"fork":false,"pushed_at":"2025-06-18T16:02:29.000Z","size":74,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-06-18T17:20:18.619Z","etag":null,"topics":["cache","database","local","postgres","tmdb","tmdb-api"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jessielw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-06-18T15:41:59.000Z","updated_at":"2025-06-18T16:37:12.000Z","dependencies_parsed_at":"2025-06-18T17:20:38.653Z","dependency_job_id":"a5cd1f43-81e4-4717-a01a-402e692f9399","html_url":"https://github.com/jessielw/TMDB-Service","commit_stats":null,"previous_names":["jessielw/tmdb-service"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/jessielw/TMDB-Service","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessielw%2FTMDB-Service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessielw%2FTMDB-Service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessielw%2FTMDB-Service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessielw%2FTMDB-Service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jessielw","download_url":"https://codeload.github.com/jessielw/TMDB-Service/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jessielw%2FTMDB-Service/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263029212,"owners_count":23402354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cache","database","local","postgres","tmdb","tmdb-api"],"created_at":"2025-07-01T20:07:02.091Z","updated_at":"2025-07-01T20:07:03.593Z","avatar_url":"https://github.com/jessielw.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TMDB-Service\n\nService that Mirrors the [TMDB API](https://developer.themoviedb.org/docs/getting-started) (most of it) to cache locally for use with your own services that need access with quick response times. Setup to work with Postgres. Automatically keeps data 1:1 based on CRON tasks.\n\n### Should You Use This\n\n- You are running a service or application that frequently accesses TMDB data and would benefit from a local cache to improve performance and reduce external API usage.\n- You want to minimize your reliance on TMDB’s rate limits or ensure availability during outages.\n- You need faster response times or more control over the data retrieval process.\n\n### Who Shouldn't Use This\n\n- If you are just occasionally looking up data you should use [TMDB-API](https://developer.themoviedb.org/docs/getting-started) directly.\n- You're only making occasional or infrequent requests to TMDB.\n- You're building a simple app, script, or tool that doesn’t need caching or local storage.\n- You don’t want to maintain an additional service or deal with syncing data.\n\n## .env File Example\n\n```\nDATABASE_URI='postgresql://tmdb:pw@tmdb_postgres:5432/tmdb'\nPOSTGRES_USER=tmdb\nPOSTGRES_PASSWORD='pw'\nPOSTGRES_DB=tmdb\nENABLE_UNACCENT=true\nCRON_FULL_SWEEP='0 3 15,30 * *'\nCRON_MISSING_ONLY='0 6 * * 1'\nCRON_PRUNE='0 3 19 * *'\nCRON_CHANGES_SYNC='0 18 * * *'\nLOG_TO_CONSOLE=true\nLOG_LVL=20\nTMDB_READ_ACCESS_TOKEN='access token'\nTMDB_RATE_LIMIT=45\nTMDB_MAX_CONNECTIONS=20\nTMDB_BATCH_INSERT=1000\n\n# notifications (maubot webhook)\nWEBHOOK_ENABLED=true\nWEBHOOK_BOT_USR=maubot\nWEBHOOK_BOT_PW=SOME_PASSWORD\nWEBHOOK_URL='https://matrix.SOME_URL.net/_matrix/maubot/plugin/BOT_URL/send'\n```\n\n### Disable CRON tasks\n\nIf you'd like to disable of the **CRON tasks** you can simply pass any of the following **\"\"**, **\"false\"**, **\"off\"**, **\"disable\"**, **\"disabled\"**, or **\"no\"** _(case insensitive)_.\n\n`CRON_PRUNE=''`\n\n`CRON_PRUNE='off'`\n\n#### CRON_CHANGES_SYNC\n\nThis task will automatically keep your database up to date with the most recent changes from TMDB API in the last 24 hours. You can set this task to when ever you want but it should be ran every **24 hours**. On the day that **CRON_FULL_SWEEP** has ran, this task will be skipped automatically.\n\n### Log Levels\n\nFor `LOG_LVL` provide an **integer** for the logging from below (defaults to 20).\n\n```\nCRITICAL = 50\nERROR = 40\nWARNING = 30\nINFO = 20\nDEBUG = 10\nNOTSET = 0\n```\n\n`LOG_LVL=20`\n\n### Webhook\n\nWebhooks will alert you of task completions as well as errors.\n\n_Currently only [maubot-webhook](https://github.com/jkhsjdhjs/maubot-webhook) is supported._\n\n### manage_jobs CLI\n\nWhile TMDB-Service is meant to be self maintained, there is a convenience CLI to run some basic commands. You must execute this from within running docker container/network.\n\n**Add Movie**:\n\n```\ndocker compose exec tmdb_service manage_jobs add_movie --id 603\n```\n\n**Usage**:\n\n```\nusage: manage_jobs [-h] [--id ID] [--force]\n                   {full_sweep,missing_ids,prune_deleted,changes_sync,create_tables,add_movie,add_series}\n\nEnqueue TMDB jobs\n\npositional arguments:\n  {full_sweep,missing_ids,prune_deleted,changes_sync,create_tables,add_movie,add_series}\n                        Type of job to enqueue\n\noptions:\n  -h, --help            show this help message and exit\n  --id ID               TMDB ID for add_movie/add_series\n  --force               Force full sweep regardless of row counts.\n```\n\n### Build Example\n\n```\ndocker buildx build --platform linux/amd64,linux/arm64 -t ghcr.io/jessielw/tmdb-service:1.0.0 -t ghcr.io/jessielw/tmdb-service:latest --push .\n```\n\n### Backup\n\n```\ndocker compose exec postgres pg_dump -U tmdb tmdb \u003e /some_path/dump.sql\n```\n\n### Restore\n\n```\n# copy dump\ndocker compose cp /some_path/dump.sql tmdb_postgres:/tmp/dump.sql\n\n# clean schema\ndocker compose exec tmdb_postgres psql -U tmdb tmdb -c \"DROP SCHEMA public CASCADE;\"\n\n# create schema\ndocker compose exec tmdb_postgres psql -U tmdb tmdb -c \"CREATE SCHEMA public;\"\n\n# update database from copied dump\ndocker compose exec tmdb_postgres psql -U tmdb tmdb -f /tmp/dump.sql\n\n# remove copied dump inside of the container\ndocker compose exec tmdb_postgres rm /tmp/dump.sql\n```\n\n## How To Use\n\nRefer to the [.env example](#env-file-example) for the required environment variables.\n\n### First Time Setup\n\n1. Setup a docker compose file _(You can use a .env file or directly supply environmental variables in your docker run/compose)_.\n\n   ```yaml\n   services:\n     tmdb_postgres:\n       image: postgres:16\n       container_name: tmdb_postgres\n       restart: unless-stopped\n       env_file:\n         - .env\n       volumes:\n         - SOME_PATH:/var/lib/postgresql/data\n\n     tmdb_service:\n       image: ghcr.io/jessielw/tmdb-service:latest\n       container_name: tmdb_service\n       restart: unless-stopped\n       depends_on:\n         - tmdb_postgres\n       env_file:\n         - .env\n       volumes:\n         - SOME_PATH:/temp_dir\n         - SOME_PATH:/logs\n       networks:\n         - proxynet\n       command: [\"python\", \"-m\", \"tmdb_service.worker\"]\n   ```\n\n2. Start the service.\n\n   `docker compose up` or `docker run ...`\n\n3. Utilize [Manage Jobs CLI](#manage_jobs-cli) to start initial ingestion in another terminal by triggering the **full_sweep**.\n\n   `docker compose exec tmdb_service manage_jobs full_sweep --force`\n\n   This will take some time depending on network conditions.\n\n### Required Volumes\n\n#### tmdb_postgres\n\n```\n/somewhere:/var/lib/postgresql/data\n```\n\n#### tmdb_service\n\n```\n/somewhere/temp:/temp_dir\n/somewhere/logs:/logs\n```\n\n### Maintaining The Service\n\nThis is handled automatically via the CRON tasks (refer to [.env example](#env-file-example)).\n\n_The [example CRON](#env-file-example) schedule should be adequate for most use cases._\n\n`CRON_FULL_SWEEP`: Performs a **complete** re-ingestion of TMDB data from the API, similar to the initial population.\n\n`CRON_MISSING_ONLY`: Ingests only missing IDs based on the most recent full dataset. _(If using `CRON_CHANGES_SYNC` you can disable this)_\n\n`CRON_PRUNE`: Removes any IDs currently in the local cache that no longer exist in the latest full dataset. _(If using `CRON_CHANGES_SYNC` you can disable this)_\n\n`CRON_CHANGES_SYNC`: Should be run approximately every **24 hours** to keep up with incremental changes from TMDB.\n\nYou can also utilize [Manage Jobs CLI](#manage_jobs-cli) to run numerous commands without utilizing the **CRON** schedules.\n\n## Using with Flask and Flask-SQLAlchemy\n\nFor convenience I've added **movies.py** and **series.py** models already converted for **Flask-SQLAlchemy** in `examples/flask_sqlalchemy_models/*.py`. You'll need to update `from your_app_service import db` import from your application. However, follow the guide below explaining how you can do this yourself if needed.\n\nTo integrate TMDB models into your Flask project using **Flask-SQLAlchemy**, the simplest approach is to copy `movies.py` and `series.py` into your project. You’ll need to make a few adjustments for **every model** and **every association table**:\n\n- Add `__bind_key__ = \"tmdb\"` to each model.\n- Add `bind_key=\"tmdb\"` to any `db.Table` association definitions.\n- Adjust model inheritance to use your project's `db.Model`.\n\n### Example\n\n```python\nfrom sqlalchemy import BigInteger, Column, DateTime, ForeignKey, SmallInteger, String\nfrom sqlalchemy.orm import Mapped, MappedAsDataclass, mapped_column, relationship\n\nfrom your_flask_project import db  # use your actual db instance\n\nclass MovieCollections(db.Model, MappedAsDataclass): # ➤ New: Change from (Base) to (db.Model, MappedAsDataclass)\n    __bind_key__ = \"tmdb\"  # ➤ New: use the TMDB database bind\n    __tablename__ = \"movie_collections\"\n\n    id: Mapped[int] = mapped_column(BigInteger, primary_key=True, autoincrement=False)\n    name: Mapped[str | None] = mapped_column(default=None)\n    poster_path: Mapped[str | None] = mapped_column(String(255), default=None)\n    backdrop_path: Mapped[str | None] = mapped_column(String(255), default=None)\n\n    movies: Mapped[list[\"Movie\"]] = relationship(\n        back_populates=\"belongs_to_collection\",\n        init=False,\n        cascade=\"all, delete-orphan\",\n        single_parent=True,\n        default_factory=list,\n        repr=False,\n    )\n\nmovie_genres_assoc = db.Table(\n    \"movie_genres_assoc\",\n    Column(\"movie_id\", ForeignKey(\"movie.id\"), primary_key=True),\n    Column(\"genre_id\", ForeignKey(\"movie_genres.id\"), primary_key=True),\n    bind_key=\"tmdb\",  # ➤ New: specify bind for the association table\n)\n```\n\n### Notes\n\n- Adding `__bind_key__ = \"tmdb\"` tells SQLAlchemy to use the TMDB-specific database connection.\n- The TMDB-Service is designed as a **read-only** cache, but nothing stops you from writing to it if needed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessielw%2Ftmdb-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjessielw%2Ftmdb-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjessielw%2Ftmdb-service/lists"}