{"id":21680335,"url":"https://github.com/johandry/finder2d","last_synced_at":"2026-04-16T11:32:33.483Z","repository":{"id":57601216,"uuid":"200013982","full_name":"johandry/finder2d","owner":"johandry","description":"Package, CLI and Microservice to find patterns or matrix in a frame or source matrix","archived":false,"fork":false,"pushed_at":"2020-03-15T05:48:34.000Z","size":112,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-25T11:08:26.234Z","etag":null,"topics":["docker","docker-compose","go","grpc","rest-api"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/johandry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-01T08:51:35.000Z","updated_at":"2019-08-06T04:03:50.000Z","dependencies_parsed_at":"2022-09-26T20:00:33.613Z","dependency_job_id":null,"html_url":"https://github.com/johandry/finder2d","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johandry%2Ffinder2d","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johandry%2Ffinder2d/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johandry%2Ffinder2d/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johandry%2Ffinder2d/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/johandry","download_url":"https://codeload.github.com/johandry/finder2d/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244594245,"owners_count":20478232,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","docker-compose","go","grpc","rest-api"],"created_at":"2024-11-25T15:15:55.515Z","updated_at":"2026-04-16T11:32:28.455Z","avatar_url":"https://github.com/johandry.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/johandry/finder2d.svg?branch=master)](https://travis-ci.org/johandry/finder2d) [![codecov](https://codecov.io/gh/johandry/finder2d/branch/master/graph/badge.svg)](https://codecov.io/gh/johandry/finder2d) [![GoDoc](https://godoc.org/github.com/johandry/finder2d?status.svg)](https://godoc.org/github.com/johandry/finder2d) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\n# Finder 2D\n\nFinder 2D is a package to recognize or find a target string or pattern in a source string. Both, the pattern and the source string, are bi-dimensional arrays.\n\nThis package is created to solve the following exercise:\n\n```text\nCouple of years back, Google create a brain simulator in their labs, that learned\nhow to recognize cats in YouTube videos (http://www.wired.co.uk/news/archive/2012-06/26/google-brain-recognises-cats)\n\nYour task is a stripped down version of the same idea.\nImagine you have a single video frame (image_with_cats.txt) with some cat images.\nIn addition, you have a perfect cat image (perfect_cat_image.txt). Your goal is\nto find the cats in the video frame. You can return the position of the cat in the\nimage matrix, and the percentage match. You can optionally provide a threshold\nmatch value (like 85% confidence this is the image you are looking for).\n\nYou should expose the functionality above as a JSON REST service\nthat takes the video frame on the input (as a text matrix), and the threshold\nmatch value, and returns the output again as a REST response. You don't have to\npass the image you are looking for (the cat image) in the REST interface - it remains constant.\n\nThe above should be implemented in Java or Go. Feel free to use frameworks or libraries as needed to accomplish the task.\n\nNote: The video frame could be \"noisy\" - you may not find the perfect cat image,\ntherefore, you have to deal with certain possibilities what you have found is the\ncorrect answer.\n```\n\n## Installing `finder2d`\n\nTo get the package or binary use `go get`:\n\n```bash\ngo get github.com/johandry/finder2d\n```\n\nOr build it from source after cloning this repository:\n\n```bash\ngit clone https://github.com/johandry/finder2d\ncd finder2d\nmake\n```\n\nYou can also get the Docker image:\n\n```bash\ndocker pull johandry/finder2d\ndocker run --rm johandry/finder2d\n```\n\nOr, build and run the Docker image from source:\n\n```bash\nmake docker-build\nmake docker-run\n```\n\n## Using the `github.com/johandry/finder2d` package\n\nFirst, get the package with `go get` and add it into the `go.mod` file (if using modules).\n\nImport the package and create the finder with the matrix values for the On and Off cells, the matching percentage and the blurry image delta. \n\n```go\nimport \"github.com/johandry/finder2d\"\n\nvar on, off byte\non = []byte(`+`)[0]\noff = []byte(` `)[0]\npercentage := 70.0\ndelta := 1\n\nfinder := finder2d.New(on, off, percentage, delta)\n```\n\nBefore search any pattern it's required to load the frame or source matrix from a reader using the `LoadSource()` function. The following example provide the frame from a file reader.\n\n```go\nsourceFileName := \"frame.txt\"\nvar sourceFile *os.File\nif sourceFile, err := os.Open(sourceFileName); err != nil {\n\treturn fmt.Errorf(\"fail to open the frame file %q. %s\", sourceFileName, err)\n}\n\nif err := finder.LoadSource(sourceFile); err != nil {\n\treturn fmt.Errorf(\"fail to load the source file %q. %s\", sourceFileName, err)\n}\n```\n\nThen load the image to search or target matrix using the `LoadTarget()` function. In this example, the image come from a string.\n\n```go\nimage := `++ +++   +++ ++\\n++           ++\\n++ .... + \\n     +++++\\n`\nimgReader := bytes.NewBufferString(image)\n\nif err := finder.LoadTarget(imgReader); err != nil {\n\treturn fmt.Errorf(\"fail to load the target matrix. %s\", err)\n}\n```\n\nHaving the source and target matrix loaded the finder is ready to do the search with `SearchSimple()`\n\n```go\nif err := finder.SearchSimple(); err != nil {\n\treturn fmt.Errorf(\"failed to search the target matrix. %s\", err)\n}\n```\n\nTo have the list of matches in JSON format use the function `String()`.\n\n```go\nfmt.Println(finder)\n```\n\nTo know more about the package read the [GoDoc](https://godoc.org/github.com/johandry/finder2d).\n\n## Running `finder2d` in CLI mode\n\nAfter download or build `finder2d` you can execute the binary or run the container. Examples:\n\n```bash\n./bin/finder2d \\\n  --source test_data/image_with_cats.txt \\\n  --target test_data/perfect_cat_image.txt \\\n  -p 80\n```\n\n```bash\ndocker run --rm \\\n    -v $(pwd)/test_data:/data \\\n    -e FINDER2D_SOURCE=\"/data/image_with_cats.txt\" \\\n    johandry/finder2d \\\n    --target /data/perfect_cat_image.txt \\\n    -p 80\n```\n\nUsing the docker container requires to mount a volume with the source and target matrix files, to be used with the parameters `--source` and `--target` or the environment variables `FINDER2D_SOURCE` and `FINDER2D_TARGET`.\n\nThe `finder2d` has the following parameters in flags or environment variables:\n\n- `--source` or `FINDER2D_SOURCE`: is the source matrix file. The given image or target matrix will be searched into the frame or source matrix. It's required in CLI mode but not in Service mode.\n- `--target` or `FINDER2D_TARGET`:  is the target matrix file. If set `finder2d` is executed in CLI mode. \n- `--on` or `FINDER2D_ON`: is the character in the given matrixes to identify a one or on bit of the image. The default value is `+`.\n- `--off` or `FINDER2D_OFF`: is the character in the given matrixes to identify a one or on bit of the image. The default value is an space character.\n- `-p` or `FINDER2D_PERCENTAGE`: is the matching percentage. The finder will find multiple matches, some of them are noise. The higher the percentage the more the image is equal to the found match. The default value is `50.0`. With the examples matrix the best results are with percentages **61%**\n- `-d` or `FINDER2D_DELTA`: is the matches blurry delta. Read below the Delta section. The default delta value is **1**\n\nFor more information use `--help`\n\n### Delta\n\nThe finder finds multiple matches for the same image/pattern found, all these matches are near by 1, 2, or more bits. Just like a blurry image, all the blurry images are one next to the other in multiple directions.\n\nThe delta value is used to reduce all these blurry images to one. The higher the delta, the less blurry images will found or, in the best case, all these blurry images will be reduced to one.\n\nHowever, in the source matrix may be multiple images/patters near that may be falsely identified as blurry images. If this is the case, you need to reduce the delta, trying to not identify them as blurry images and identify them as just images one near to the other.\n\nUsing the default matching percentage (50%) the best results are found with **delta = 5**. Read the following table with the minimum percentage for possible delta values:\n\n| Delta | Minimum Percentage |\n| ----- | ----------- |\n| 6     | 48%            |\n| 5     | 48%            |\n| 4     | 51%            |\n| 3     | 51%            |\n| 2     | 54%            |\n| 1     | 61%            |\n\nThe maximum percentage is **96%** for any delta.\n\n## Running `finder2d` in server mode\n\nExecute `finder2d` either as a binary or in a container without the flag `--target`. The frame or source matrix file with the flag `--source` is optional, if no source file is provided it has to load it before any other action.\n\n```bash\nFINDER2D_SOURCE=/test_data/image_with_cats.txt ./bin/finder2d\n```\n\n```bash\ndocker run --rm \\\n    -v $(pwd)/test_data:/data \\\n    -p 8080:8080 \\\n    johandry/finder2d \\\n    --source /data/image_with_cats.txt\n```\n\nIn a different terminal access the server either to the gRPC or REST/HTTP API using `grpcurl` or `curl` with `jq` for better JSON formatting:\n\n```bash\ncurl -s \"http://localhost:8080/api/v1/matrixes/0\" | jq\n```\n\n```bash\ngrpcurl -plaintext -d '{\"name\": 0}' localhost:8080 finder2d.v1.Finder2D.GetMatrix\n```\n\nTo install `grpcurl` or `jq` on MacOS execute `brew install grpcurl jq`.\n\nTo list all the available gRPC methods from the Finder2D service, use:\n\n```bash\ngrpcurl -plaintext localhost:8080 list finder2d.v1.Finder2D\n```\n\nTo access the Swagger API definition open [localhost:8080/api/v1/swagger/finder2d.json](http://localhost:8080/api/v1/swagger/finder2d.json) or use the Swagger Docker container to see it with Swagger UI:\n\n```bash\ndocker run --rm \\\n  -p 80:8080 \\\n  -e API_URL=http://localhost:8080/api/v1/swagger/finder2d.json \\\n  swaggerapi/swagger-ui\n```\n\nThen navigate to [localhost](http://localhost).\n\nYou can also import the Swagger into Postman for a better API interaction and testing.\n\n## Running `finder2d` with Docker Compose\n\nIn the project repository there is a `docker-compose.yml` file. This file can be used to start the Finder2D server and the Swagger UI.\n\nExecute the following commands:\n\n```bash\ndocker-compose up -d\ndocker-compose ps\n```\n\nNavigate to [localhost](http://localhost) to see the Swagger UI or [localhost:8080/api/v1/swagger/finder2d.json](http://localhost:8080/api/v1/swagger/finder2d.json) to read the Swagger JSON API definition.\n\nExecute the commands to access the API using `curl` or `grpcurl` (refer to the [API version 1](#API_version_1) section below). For example:\n\n```bash\n# Load the image\nimg=$(awk '{printf \"%s\\\\n\" , $0}' test_data/perfect_cat_image.txt)\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"name\": 1, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  localhost:8080 finder2d.v1.Finder2D.LoadMatrix\n\n# Search the image in the frame\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"percentage\": 70.5, \"delta\": 1}' \\\n  localhost:8080 finder2d.v1.Finder2D.Search\n\n# Get all the matches\ncurl -s \"http://localhost:8080/api/v1/matches\" | jq\n\n# Get match number 2 and the matrix found\ncurl -s \"http://localhost:8080/api/v1/matches/1\" | jq\n\n# Get the 2nd image found\ncurl -s \"http://localhost:8080/api/v1/matches/1\" | \\\n  jq '.matrix.content' | \\\n  awk '{gsub(/\\\\n/,\"\\n\")}1'\n```\n\nWhen you are done, execute the following commands to destroy the services.\n\n```bash\ndocker-compose stop\ndocker-compose down\n```\n\n## API version 1\n\n### GetMatrix\n\nThe gRPC method `GetMatrix` is to request the frame or source matrix and the image or target matrix. The frame is identified by a `0` and the image by a `1`. The received object has the matrix content and size.\n\nThe REST/HTTP route is `/api/v1/matrixes/{name}` with the HTTP method `GET`.\n\nUsing `curl` and `jq`:\n\n```bash\n# Requesting the frame\ncurl -s \"http://localhost:8080/api/v1/matrixes/0\" | jq\n# Requesting the image\ncurl -s \"http://localhost:8080/api/v1/matrixes/1\" | jq\n```\n\nUsing `grpcurl`:\n\n```bash\n# Requesting the frame\ngrpcurl -plaintext -d '{\"name\": 0}' localhost:8080 finder2d.v1.Finder2D.GetMatrix\n# Requesting the image\ngrpcurl -plaintext -d '{\"name\": 1}' localhost:8080 finder2d.v1.Finder2D.GetMatrix\n```\n\nSample Output:\n\n```json\n{\n  \"api\": \"v1\",\n  \"name\": \"SOURCE\",\n  \"matrix\": {\n    \"width\": 15,\n    \"height\": 15,\n    \"content\": \"+             +\\n+++         +++\\n ++++++    .......        \\n\"\n  }\n}\n```\n\n### LoadMatrix\n\nThe gRPC method `LoadMatrix` is to load into the Finder2D the frame or source matrix and the image or target matrix.\n\nThe request is a JSON object with the matrix type (`\"name\"`) and the matrix object only with the content (`\"matrix\": {\"content\": \"....\"}`). The frame is identified by a `0` and the image by a `1`.\n\nThe response only contain the API version number, if there was an error it will be in the response.\n\nThe REST/HTTP route is `/api/v1/matrixes/{name}` with the HTTP method `POST`.\n\nUsing `curl` and `jq`:\n\n```bash\n# Load the content into an environment variable, replacing '\\n' for '\\\\n'\nimg=$(awk '{printf \"%s\\\\n\" , $0}' test_data/perfect_cat_image.txt)\n\n# Loading the frame\ncurl -s \\\n  -d '{\"api\": \"v1\", \"name\": 0, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  -H \"Content-Type: application/json\" \\\n  -X POST  \"http://localhost:8080/api/v1/matrixes/0\" | jq\n\n# Loading the image\ncurl -s \\\n  -d '{\"api\": \"v1\", \"name\": 1, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  -H \"Content-Type: application/json\" \\\n  -X POST  \"http://localhost:8080/api/v1/matrixes/1\" | jq\n```\n\nUsing `grpcurl`:\n\n```bash\n# Load the content into an environment variable, replacing '\\n' for '\\\\n'\nimg=$(awk '{printf \"%s\\\\n\" , $0}' test_data/image_with_cats.txt)\n\n# Loading the frame\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"name\": 0, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  localhost:8080 finder2d.v1.Finder2D.LoadMatrix\n\n# Loading the image\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"name\": 1, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  localhost:8080 finder2d.v1.Finder2D.LoadMatrix\n```\n\nSample Output:\n\n```json\n{\n  \"api\": \"v1\"\n}\n```\n\nOr an error, in this example the `\\n` was not replaced by `\\\\n`:\n\n```bash\n{\n  \"error\": \"invalid character '\\\\n' in string literal\",\n  \"message\": \"invalid character '\\\\n' in string literal\",\n  \"code\": 3,\n  \"details\": []\n}\n```\n\n### Search\n\nThe gRPC method `Search` is used to search the image or target matrix in the frame or source matrix using the `SearchSimple()` method of the Finder2D.\n\nIt's important to remember to load the target matrix before execute a search, otherwise an error will be received.\n\nThe request is a JSON object with the percentage (`\"percentage\"`) and the delta (`\"delta\"`) values. The response has the total number of matches found (`\"total_matches\"`).\n\nThe REST/HTTP route is `/api/v1/search` with the HTTP method `POST`.\n\n Using `curl` and `jq`:\n\n```bash\n# Load the image\nimg=$(awk '{printf \"%s\\\\n\" , $0}' test_data/perfect_cat_image.txt)\ncurl -s \\\n  -d '{\"api\": \"v1\", \"name\": 1, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  -H \"Content-Type: application/json\" \\\n  -X POST  \"http://localhost:8080/api/v1/matrixes/1\" | jq\n\n# Search the image in the frame\ncurl -s \\\n  -d '{\"api\": \"v1\", \"percentage\": 70.5, \"delta\": 1}' \\\n  -H \"Content-Type: application/json\" \\\n  -X POST  \"http://localhost:8080/api/v1/search\" | jq\n```\n\nUsing `grpcurl`:\n\n```bash\n# Load the image like in previous example\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"name\": 1, \"matrix\": {\"content\": \"'$img'\"}}' \\\n  localhost:8080 finder2d.v1.Finder2D.LoadMatrix\n\n# Search the image in the frame\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"percentage\": 70.5, \"delta\": 1}'  \\\n  localhost:8080 finder2d.v1.Finder2D.Search\n```\n\nSample Output:\n\n```json\n{\n  \"api\": \"v1\",\n  \"total_matches\": 6\n}\n```\n\n### GetMatches\n\nThe gRPC method `GetMatches` is used retrieve all the matches found from a previous search. This method will return an empty list if the search is not done before.\n\nThe request is a JSON object only with the API version. The response is a JSON object with an array/list of matches (`\"matches\"`). Each Match is a JSON object with the coordinates (`\"x\"`, `\"y\"`) and the matching percentage (`\"percentage\"`).\n\nThe REST/HTTP route is `/api/v1/matches` with the HTTP method `GET`.\n\nUsing `curl` and `jq`:\n\n```bash\n# Get the list of found matches\ncurl -s \"http://localhost:8080/api/v1/matches\" | jq\n```\n\nUsing `grpcurl`:\n\n```bash\n# Get the list of found matches\ngrpcurl -plaintext localhost:8080 finder2d.v1.Finder2D.GetMatches\n```\n\nSample Output:\n\n```json\n{\n  \"api\": \"v1\",\n  \"matches\": [\n    {\n      \"x\": 80,\n      \"y\": 0,\n      \"percentage\": 99.111115\n    },\n....\n    {\n      \"x\": 84,\n      \"y\": 84,\n      \"percentage\": 98.666664\n    }\n  ]\n}\n```\n\n### GetMatch\n\nThe gRPC method `GetMatch` return the requested match identified by it's index in the list. This method will return an error if the index is out of range.\n\nThe request is a JSON object  with the index or ID (`\"id\"`) of the required match. The response is a JSON object with the match (`\"match\"`) and the Matrix (`\"matrix\"`). The Match is a JSON object with the coordinates (`\"x\"`, `\"y\"`) and the matching percentage (`\"percentage\"`). The Matrix is a JSON object with the width (`\"width\"`), height (`\"height\"`) and the content of the matrix (`\"content\"`) as a string.\n\nThe REST/HTTP route is `/api/v1/match/{id}` with the HTTP method `GET`.\n\nUsing `curl` and `jq`:\n\n```bash\n# Get the list of found matches\ncurl -s \"http://localhost:8080/api/v1/match/0\" | jq\n```\n\nUsing `grpcurl`:\n\n```bash\n# Get the list of found matches\ngrpcurl -plaintext \\\n  -d '{\"api\": \"v1\", \"id\": 1}' \\\n  localhost:8080 finder2d.v1.Finder2D.GetMatch\n```\n\nSample Output:\n\n```json\n{\n  \"api\": \"v1\",\n  \"match\": {\n    \"x\": 84,\n    \"y\": 84,\n    \"percentage\": 98.666664\n  },\n  \"matrix\": {\n    \"width\": 15,\n    \"height\": 15,\n    \"content\": \"+             +\\n+++         +++\\n +++++++ ......             \\n\"\n  }\n}\n```\n\n## TODO\n\n- [x] Implement the LoadMatrix gRPC method\n- [x] Define and implement the Search gRPC method\n- [x] Create the Docker Compose for the services\n- [x] Start server mode by default when the image file is not provided\n- [x] Accept parameter in environment variables as well as with arguments\n- [x] CI/CD with Travis and Docker Hub\n- [ ] Create the Kubernetes Manifest for the services\n- [ ] Security: Implement TLS\n- [ ] Try CircleCI\n- [ ] Allow to load multiple targets\n- [ ] Create a DB service to store the matrixes\n- [ ] Make a client `finder2dctl`\n- [ ] Make a client in other language (Python? Ruby?)\n- [ ] Build a UI\n- [ ] Go serverless \n- [ ] Security: Implement JWT\n\n## Other algorithms to improve the search\n\n**2D Convolution**:\n\n- [Convolution Theorem](https://en.wikipedia.org/wiki/Convolution_theorem)\n- [Convolution 2D Example](http://www.songho.ca/dsp/convolution/convolution2d_example.html)\n- [Two Dimensional Convolution in Image Processing](https://www.allaboutcircuits.com/technical-articles/two-dimensional-convolution-in-image-processing/)\n- [Understanding Convolutions for Deep Learning](https://towardsdatascience.com/intuitively-understanding-convolutions-for-deep-learning-1f6f42faee1)\n- [Efficient 2D Approximate Matching of Non-rectangular Figures](https://www.cs.rutgers.edu/~farach/pubs/HalfRec.pdf)\n\n**Rabin–Karp algorithm**:\n\n- [Rabin–Karp Algorithm](https://en.wikipedia.org/wiki/Rabin–Karp_algorithm)\n- [Rabin Fingerprint](https://en.wikipedia.org/wiki/Rabin_fingerprint)\n\n**Boyer–Moore algorithm**:\n\n- [Boyer–Moore String Search Algorithm](https://en.wikipedia.org/wiki/Boyer–Moore_string-search_algorithm)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohandry%2Ffinder2d","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohandry%2Ffinder2d","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohandry%2Ffinder2d/lists"}