https://github.com/louis030195/tfrecord-builder

API Google App Engine + PubSub to build tfrecord
https://github.com/louis030195/tfrecord-builder

google-appengine google-cloud-platform

Last synced: 6 months ago
JSON representation

API Google App Engine + PubSub to build tfrecord

• Host: GitHub
• URL: https://github.com/louis030195/tfrecord-builder
• Owner: louis030195
• Created: 2019-08-07T15:56:57.000Z (over 6 years ago)
• Default Branch: master
• Last Pushed: 2024-04-16T23:58:20.000Z (over 1 year ago)
• Last Synced: 2025-02-10T17:58:36.706Z (11 months ago)
• Topics: google-appengine, google-cloud-platform
• Language: Python
• Size: 20.5 KB
• Stars: 0
• Watchers: 3
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Python Google Cloud Pub/Sub for Google App Engine Flexible Environment

[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/louis030195/tfrecord-builder)

## Setup

Before you can run or deploy the sample, you will need to do the following:

1. Enable the Cloud Pub/Sub API in the [Google Developers Console](https://console.developers.google.com/project/_/apiui/apiview/pubsub/overview).

2. Create a topic and subscription.

        $ gcloud pubsub topics create topic_tfrecord

        $ gcloud pubsub subscriptions create sub_tfrecord \

            --topic topic_tfrecord \

            --push-endpoint \

                https://[your-app-id].appspot.com/pubsub/push?token=[your-token] \

            --ack-deadline 30

3. Update the environment variables in ``app.yaml``.

## Running locally

Install dependencies, preferably with a virtualenv:

    virtualenv env

    source env/bin/activate

    pip install -r requirements.txt

Then set environment variables before starting your application:

    export GOOGLE_APPLICATION_CREDENTIALS=[path-to-json-cred]

    export PUBSUB_VERIFICATION_TOKEN=[your-verification-token]

    export PUBSUB_TOPIC=[your-topic]

    export BUCKET_NAME=[your-bucket]

    export PROJECT_ID=[your-project-id]

    python main.py

### Simulating push notifications

Assuming you have Frame entities in Datastore with imageUrl property pointing to their storage (on GCS ?)

    export PUBSUB_VERIFICATION_TOKEN=[your-verification-token]

    python test/call.py

## Running on App Engine

    gcloud app deploy