{"id":26658938,"url":"https://github.com/denisecase/beam-pagerank-go","last_synced_at":"2025-04-11T14:09:41.635Z","repository":{"id":97697081,"uuid":"472979286","full_name":"denisecase/beam-pagerank-go","owner":"denisecase","description":null,"archived":false,"fork":false,"pushed_at":"2022-03-25T00:33:46.000Z","size":34812,"stargazers_count":1,"open_issues_count":0,"forks_count":11,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T10:17:12.802Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/denisecase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-23T00:12:07.000Z","updated_at":"2022-03-25T16:42:49.000Z","dependencies_parsed_at":"2023-04-01T11:35:28.700Z","dependency_job_id":null,"html_url":"https://github.com/denisecase/beam-pagerank-go","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denisecase%2Fbeam-pagerank-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denisecase%2Fbeam-pagerank-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denisecase%2Fbeam-pagerank-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/denisecase%2Fbeam-pagerank-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/denisecase","download_url":"https://codeload.github.com/denisecase/beam-pagerank-go/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248413725,"owners_count":21099349,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-25T10:17:19.701Z","updated_at":"2025-04-11T14:09:41.615Z","avatar_url":"https://github.com/denisecase.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# beam-wordcount-go\nExample from https://github.com/apache/beam/blob/master/sdks/go/examples/wordcount/wordcount.go\n\n## Read More\n\n- [Apache Beam Go SDK Quickstart](https://beam.apache.org/get-started/quickstart-go/)\n\n## Windows Tools\n\n- PowerShell (cross-platform)\n- Git\n- Chocolatey (Windows Package Manager)\n\n## Install Go and Verify\n\n```PowerShell\ngo version\n```\n\nReview the installed files at C:\\Program Files\\Go.\n\n## Initialize Project Module\n\n```PowerShell\ngo mod init github.com/denisecase/beam-pagerank-go\n```\n\nReview go.mod.\n\n## Get Beam Go SDK\n\nWhen building, read error messages. \nIf a package is needed, a command to add will be provided. \nRun the necessary commands. \n\nThe command `go get` will download the package to the local module cache, \nand add it to the go.mod file. The `-u` flag indicates update. \n\n```PowerShell\ngo get -u github.com/apache/beam/sdks/v2/go/pkg/beam\ngo get -u github.com/apache/beam/sdks/v2/go/pkg/beam/transforms/create\ngo get -u github.com/apache/beam/sdks/v2/go/pkg/beam/sideInput\ngo get github.com/apache/beam/sdks/v2/go/pkg/beam/io/filesystem/gcs@v2.37.0\n```\n\nThis allows us to use beam packages. Review go.mod. \n\n## Build to Executable\n\n```PowerShell\ngo build pagerank.go\n```\n\nReview go.sum. \nRead any error messages and run recommended commands as needed.\nVerify the new .exe executable file is created.\n\n## Run Executable\n\n```PowerShell\n.\\pagerank.exe --input \u003cPATH_TO_INPUT_FILE\u003e --output out.csv\n```\n\nExamples: \n\n```PowerShell\n.\\pagerank.exe --output denise.csv\n.\\pagerank.exe \n```\n\nReview the local dependencies at C:\\Users\\\u003cusername\u003e\\AppData\\Local\\go-build.\n\n## About Go\n\n- go get - updates dependencies/versions listed in go.mod and updates local cache\n- go install - used to build and install the provided source file in $GOPATH$\n- go build - compiles and builds executable locally\n- go fmt - format go code\n- go mod tidy - keep things updated\n\n## About Beam Pipelines\n\nFrom https://beam.apache.org/documentation/programming-guide/\n\nA typical Beam driver program works as follows:\n\n1. Create a Pipeline object and set the pipeline execution options, including the Pipeline Runner.\n\n2. Create an initial PCollection for pipeline data, either using the IOs to read data from an external storage system, or using a Create transform to build a PCollection from in-memory data.\n\n3. Apply PTransforms to each PCollection. Transforms can change, filter, group, analyze, or otherwise process the elements in a PCollection. A transform creates a new output PCollection without modifying the input collection. A typical pipeline applies subsequent transforms to each new output PCollection in turn until processing is complete. However, note that a pipeline does not have to be a single straight line of transforms applied one after another: think of PCollections as variables and PTransforms as functions applied to these variables: the shape of the pipeline can be an arbitrarily complex processing graph.\n\n4. Use IOs to write the final, transformed PCollection(s) to an external source.\n\n5. Run the pipeline using the designated Pipeline Runner.\n\n## Mapping Done by ParDo\n\nA ParDo transform considers each element in the input PCollection, performs some processing function (your user code) on that element, and emits zero or more elements to an output PCollection.\n\nThe emit func(...) is useful when the number of output elements differ to the number of input elements. If its a 1:1 mapping a return makes the function easier to read. \n\nFrom \u003chttps://blog.gopheracademy.com/advent-2018/apache-beam/\u003e\n\n## Storing in GitHub\n\n1. Create an empty repo (same folder name) in GitHub\n2. In your local folder, `git init`\n3. `git remote add origin https://github.com/denisecase/beam-pagerank-go.git`\n4. git add / commit / push to repo","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenisecase%2Fbeam-pagerank-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdenisecase%2Fbeam-pagerank-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdenisecase%2Fbeam-pagerank-go/lists"}