{"id":13505611,"url":"https://github.com/sam-goodwin/punchcard","last_synced_at":"2025-04-04T13:04:01.495Z","repository":{"id":39644572,"uuid":"181975248","full_name":"sam-goodwin/punchcard","owner":"sam-goodwin","description":"Type-safe AWS infrastructure.","archived":false,"fork":false,"pushed_at":"2021-12-09T20:28:33.000Z","size":4062,"stargazers_count":511,"open_issues_count":41,"forks_count":19,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-03-29T04:12:34.593Z","etag":null,"topics":["aws","aws-cdk","infrastructure-as-code","javascript","nodejs","punchcard","serverless","typescript"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sam-goodwin.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-04-17T22:08:25.000Z","updated_at":"2025-03-07T14:45:52.000Z","dependencies_parsed_at":"2022-09-09T11:23:02.833Z","dependency_job_id":null,"html_url":"https://github.com/sam-goodwin/punchcard","commit_stats":null,"previous_names":["punchcard/punchcard"],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sam-goodwin%2Fpunchcard","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sam-goodwin%2Fpunchcard/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sam-goodwin%2Fpunchcard/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sam-goodwin%2Fpunchcard/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sam-goodwin","download_url":"https://codeload.github.com/sam-goodwin/punchcard/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246769973,"owners_count":20830771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-cdk","infrastructure-as-code","javascript","nodejs","punchcard","serverless","typescript"],"created_at":"2024-08-01T00:01:10.651Z","updated_at":"2025-04-04T13:04:01.460Z","avatar_url":"https://github.com/sam-goodwin.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"![CodeBuild badge](https://codebuild.us-west-2.amazonaws.com/badges?uuid=eyJlbmNyeXB0ZWREYXRhIjoiZFYyRk82d3plVWlNdW8xblNKZk1XOEpCSkwvQTZjeUFVMG1odkdDWFU3Zm1CVXZYcENWS0VCbW52QnNieml3NFUvTnlYbkIzVUJGU2U1a0hTanlRYitVPSIsIml2UGFyYW1ldGVyU3BlYyI6IlY2TGhPdjJ6ZjZZVVJxVDgiLCJtYXRlcmlhbFNldFNlcmlhbCI6MX0%3D\u0026branch=master)\n[![Gitter](https://badges.gitter.im/punchcard-cdk/community.svg)](https://gitter.im/punchcard-cdk/community?utm_source=badge\u0026utm_medium=badge\u0026utm_campaign=pr-badge)\n[![npm version](https://badge.fury.io/js/punchcard.svg)](https://badge.fury.io/js/punchcard)\n\n# Punchcard\n\nPunchcard is a TypeScript framework for building cloud applications with the [AWS CDK](https://github.com/aws/aws-cdk). It unifies **infrastructure** code with **runtime** code, meaning you can both declare resources and implement logic within the context of one node.js application. AWS resources are thought of as generic, type-safe objects — DynamoDB Tables are like a `Map\u003cK, V\u003e`; SNS Topics, SQS Queues, and Kinesis Streams feel like an `Array\u003cT\u003e`; and a Lambda Function is akin to a `Function\u003cA, B\u003e` – like the standard library of a programming language.\n\n# Blog Series\n\nIf you'd like to learn more about the philosophy behind this project, check out my blog series:\n* [Punchcard: Imagining the future of cloud programming](https://bit.ly/punchcard-cdk).\n* [Type-Safe Infrastructure, Part 1 — Data Types and Data Flows.](https://medium.com/swlh/punchcard-building-a-data-lake-5471cfadf66f?source=friends_link\u0026sk=56b4d7647a348d53c1542d9b4ab9da17)\n* [Type Safe Infrastructure, Part 2 — GraphQL APIs with AWS AppSync.](https://medium.com/@sam.goodwin1989/type-safe-infrastructure-part-2-graphql-apis-with-aws-appsync-d1225e4e21e3?source=friends_link\u0026sk=e27527002a553d5e3e2e6fc9c7d3ba1a)\n\n# Sample Repository\n\nhttps://github.com/punchcard/punchcard-example - fork to get started\n\n# Developer Guide\n\nTo understand the internals, there is the guide:\n\n1. [Getting Started](docs/1-getting-started.md)\n2. [Creating Functions](docs/2-creating-functions.md)\n3. [Runtime Dependencies](docs/3-runtime-dependencies.md)\n4. [Shapes: Type-Safe Schemas](docs/4-shapes.md)\n5. [Dynamic (and safe) DynamoDB DSL](docs/5-dynamodb-dsl.md)\n6. [Stream Processing](docs/6-stream-processing.md)\n\n# Tour\n\nInitialize an App and Stack:\n```ts\nconst app = new Core.App();\nconst stack = app.stack('hello-world');\n```\n\n## Runtime Code and Dependencies\n\nCreating a Lambda Function is super simple - just create it and implement `handle`:\n\n```ts\nnew Lambda.Function(stack, 'MyFunction', {}, async (event) =\u003e {\n  console.log('hello world');\n});\n```\n\nTo contact other services in your Function, data structures such as SNS Topics, SQS Queues, DynamoDB Tables, etc. are declared as a `Dependency`. \n\nThis will create the required IAM policies for your Function's IAM Role, add any environment variables for details such as the Topic's ARN, and automatically create a client for accessing the `Construct`. The result is that your `handle` function is now passed a `topic` instance which you can interact with:\n\n```ts\nnew Lambda.Function(stack, 'MyFunction', {\n  depends: topic,\n}, async (event, topic) =\u003e {\n  await topic.publish(new NotificationRecord({\n    key: 'some key',\n    count: 1,\n    timestamp: new Date()\n  }));\n});\n```\n\nFurthermore, its interface is higher-level than what would normally be expected when using the `aws-sdk`, and it's also type-safe: the argument to the `publish` method is not an opaque `string` or `Buffer`, it is an `object` with keys and rich types such as `Date`. This is because data structures in punchcard, such as `Topic`, `Queue`, `Stream`, etc. are generic with statically declared types (like an `Array\u003cT\u003e`):\n\n```ts\n/**\n * Message is a JSON Object with properties: `key`, `count` and `timestamp`.\n */\nclass NotificationRecord extends Type({\n  key: string,\n  count: integer,\n  timestamp\n}) {}\n\nconst topic = new SNS.Topic(stack, 'Topic', {\n  shape: NofiticationRecord\n});\n```\n\nThis `Topic` is now of type:\n```ts\nTopic\u003cNotificationRecord\u003e\n```\n\n## Type-Safe DynamoDB Expressions\n\nThis feature in punchcard becomes even more evident when using DynamoDB. To demonstrate, let's create a DynamoDB `Table` and use it in a `Function`:\n\n```ts\n// class describing the data in the DynamoDB Table\nclass TableRecord extends Type({\n  id: string,\n  count: integer\n    .apply(Minimum(0))\n}) {}\n\n// table of TableRecord, with a single hash-key: 'id'\nconst table = new DynamoDB.Table(stack, 'my-table', {\n  data: TableRecord\n  key: {\n    partition: 'id'\n  }\n});\n```\n\nNow, when getting an item from DynamoDB, there is no need to use `AttributeValues` such as `{ S: 'my string' }`, like you would when using the low-level `aws-sdk`. You simply use ordinary javascript types:\n\n```ts\nconst item = await table.get({\n  id: 'state'\n});\n```\n\nThe interface is statically typed and derived from the definition of the `Table` - we specified the `partitionKey` as the `id` field which has type `string`, and so the object passed to the `get` method must correspond.\n\n`PutItem` and `UpdateItem` have similarly high-level and statically checked interfaces. More interestingly, condition and update expressions are built with helpers derived (again) from the table definition:\n\n```ts\n// put an item if it doesn't exist\nawait table.put(new TableRecord({\n  id: 'state',\n  count: 1\n}), {\n  if: _ =\u003e _.id.notExists()\n});\n\n// increment the count property by 1 if it is less than 0\nawait table.update({\n  // value of the partition key\n  id: 'state'\n}, {\n  // use the DSL to construt an array of update actions\n  actions: _ =\u003e [\n    _.count.increment(1)\n  ],\n  // optional: use the DSL to construct a conditional expression for the update\n  if: _ =\u003e _.id.lessThan(0)\n});\n```\n\nTo also specify `sortKey`, use a tuple of `TableRecord's` keys:\n\n```ts\nconst table = new DynamoDB.Table(stack, 'my-table',{\n  data: TableRecord,\n  key: {\n    partition: 'id',\n    sort: 'count'\n  }\n});\n```\n\nNow, you can also build typesafe query expressions:\n\n```ts\nawait table.query({\n  // id is the partition key, so we must provide a literal value\n  id: 'id',\n  // count is the sort key, so use the DSL to construct the sort-key condition\n  count: _ =\u003e _.greaterThan(1)\n}, {\n  // optional: use the DSL to construct a filter expression\n  filter: _ =\u003e _.count.lessThan(0)\n})\n```\n## Stream Processing\n\nPunchcard has the concept of `Stream` data structures, which should feel similar to in-memory streams/arrays/lists because of its chainable API, including operations such as `map`, `flatMap`, `filter`, `collect` etc. Data structures that implement `Stream` are: `SNS.Topic`, `SQS.Queue`, `Kinesis.Stream`, `Firehose.DeliveryStream` and `Glue.Table`.\n\nFor example, given an SNS Topic:\n```ts\nconst topic = new SNS.Topic(stack, 'Topic', {\n  shape: NotificationRecord\n});\n```\n\nYou can attach a new Lambda Function to process each notification:\n```ts\ntopic.notifications().forEach(stack, 'ForEachNotification', {},\n  async (notification) =\u003e {\n    console.log(`notification delayed by ${new Date().getTime() - notification.timestamp.getTime()}ms`);\n  });\n```\n\nOr, create a new SQS Queue and subscribe notifications to it:\n\n*(Messages in the `Queue` are of the same type as the notifications in the `Topic`.)*\n\n```ts\nconst queue = topic.toSQSQueue(stack, 'MyNewQueue');\n```\n\nWe can then, perhaps, `map` over each message in the `Queue` and collect the results into a new AWS Kinesis `Stream`:\n\n```ts\nclass LogDataRecord extends Type({\n  key: string,\n  count: integer,\n  tags: array(string)\n  timestamp\n}) {}\n\nconst stream = queue.messages()\n  .map(async (message, e) =\u003e new LogDataRecord({\n    ...message,\n    tags: ['some', 'tags'],\n  }))\n  .toKinesisStream(stack, 'Stream', {\n    // partition values across shards by the 'key' field\n    partitionBy: value =\u003e value.key,\n\n    // type of the data in the stream\n    shape: LogData\n  });\n```\n\nWith data in a `Stream`, we might want to write out all records to a new S3 `Bucket` by attaching a new Firehose `DeliveryStream` to it:\n\n```ts\nconst s3DeliveryStream = stream.toFirehoseDeliveryStream(stack, 'ToS3');\n```\n\nWith data now flowing to S3, let's partition and catalog it in a `Glue.Table` (backed by a new `S3.Bucket`) so we can easily query it with AWS Athena, AWS EMR and AWS Glue:\n\n```ts\nimport glue = require('@aws-cdk/aws-glue');\nimport { Glue } from 'punchcard';\n\nconst database = stack.map(stack =\u003e new glue.Database(stack, 'Database', {\n  databaseName: 'my_database'\n}));\ns3DeliveryStream.objects().toGlueTable(stack, 'ToGlue', {\n  database,\n  tableName: 'my_table',\n  columns: LogDataRecord,\n  partition: {\n    // Glue Table partition keys: minutely using the timestamp field\n    keys: Glue.Partition.Minutely,\n    // define the mapping of a record to its Glue Table partition keys\n    get: record =\u003e Glue.Partition.byMinute(record.timestamp)\n  }\n});\n```\n\n## Example Stacks\n\n* [GraphQL API](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/straw-poll.ts) - Implements a GraphQL API with AWS AppSync for a real-time voting app, \"straw poll\".\n* [Stream Processing](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/stream-processing.ts) - respond to SNS notifications with a Lambda Function; subscribe notifications to a SQS Queue and process them with a Lambda Function; process and forward data from a SQS Queue to a Kinesis Stream; sink records from the Stream to S3 and catalog it in a Glue Table.\n* [Invoke a Function from another Function](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/invoke-function.ts) - call a Function from another Function\n* [Real-Time Data Lake](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/data-lake.ts) - collects data with Kinesis and persists to S3, exposed as a Glue Table in a Glue Database.\n* [Scheduled Lambda Function](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/scheduled-function.ts) - runs a Lambda Function every minute and stores data in a DynamoDB Table.\n* [Pet Store API Gateway](https://github.com/sam-goodwin/punchcard/blob/master/examples/lib/pet-store-apigw.ts) - implementation of the [Pet Store API Gateway canonical example](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-from-example.html).\n\n## License\n\nThis library is licensed under the Apache 2.0 License. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsam-goodwin%2Fpunchcard","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsam-goodwin%2Fpunchcard","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsam-goodwin%2Fpunchcard/lists"}