{"id":23134800,"url":"https://github.com/kfahn22/ml-facemesh","last_synced_at":"2025-09-02T06:35:38.910Z","repository":{"id":265895650,"uuid":"886860781","full_name":"kfahn22/ml-facemesh","owner":"kfahn22","description":"A collection of ml5 faceMesh sketches","archived":false,"fork":false,"pushed_at":"2024-12-01T11:51:35.000Z","size":4374,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-02T06:35:38.091Z","etag":null,"topics":["facemesh","ml5","shaders"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kfahn22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-11-11T18:29:49.000Z","updated_at":"2024-12-01T11:51:39.000Z","dependencies_parsed_at":null,"dependency_job_id":"39c02768-69c7-45cb-8c3c-0f3ec27aa562","html_url":"https://github.com/kfahn22/ml-facemesh","commit_stats":null,"previous_names":["kfahn22/ml-facemesh"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kfahn22/ml-facemesh","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfahn22%2Fml-facemesh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfahn22%2Fml-facemesh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfahn22%2Fml-facemesh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfahn22%2Fml-facemesh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kfahn22","download_url":"https://codeload.github.com/kfahn22/ml-facemesh/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kfahn22%2Fml-facemesh/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273244304,"owners_count":25070958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-02T02:00:09.530Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["facemesh","ml5","shaders"],"created_at":"2024-12-17T12:13:15.811Z","updated_at":"2025-09-02T06:35:38.896Z","avatar_url":"https://github.com/kfahn22.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ml5 FaceMesh Model\n\nDaniel Shiffman recently released a [tutorial](https://thecodingtrain.com/tracks/ml5js-beginners-guide/ml5/facemesh) about the [ml5 FaceMesh](https://docs.ml5js.org/#/reference/facemesh) model. While it is possible to use the FaceMesh model in ways that are inappropriate or unethical, I have used it to create some **silly** visual effects.\n\n## Ghost in the Material World\n\nThe first effect adapts some clever shader code from [Barney Codes](https://www.youtube.com/watch?v=ZcRptHYY3zM) to create a shockwave effect centered around the bounding box for the face. You can check it out [here](https://editor.p5js.org/kfahn/full/bmSF2Kcuv)\n\nFirst, we need to calculate the (centerX, centerY) to pass to the frag file. I have to confess that this was the hardest part. I initally tried using the `width/2 + box.width/2, height/2 + box.height/2`, but the tracking didn't work. Adding the box.width/2 and box.height/2 to the box.xMax and box.yMax seems counter-intuitive (at least to me), but after some trial and error this is what seems to center the effect properly. My guess is that is necessary because the video is flipped, although I would not be surprised to learn that I have made some silly mistake.\n\n```JavaScript\nlet centerX = box.xMax + box.width / 2;\nlet centerY = box.yMax + box.height / 2;\n```\n\nI then adapted a function from Barney Codes to update the \"center\" sent to the frag file in draw.\n\n```JavaScript\nfunction setCenter(cX, cY) {\n  faceShader.setUniform(\"center\", [cX / width, cY / height]);\n}\n```\n\nAlthough this is certainly not necessary, I am using the [Craniod](https://mathworld.wolfram.com/Cranioid.html) function to create the \"ghost\" head.\n\n```GLSL\nvec2 Spherical(vec2 pos)\n{\n   float r = sqrt(pos.x*pos.x + pos.y*pos.y);\n   float theta = atan(pos.y, pos.x);\n   vec2 w = vec2(r, theta);\n   return w;\n}\nfloat CraniodSDF(vec2 pos, vec2 radius) {\n    vec2 q;\n    float d = Spherical(pos).x;\n    float theta = Spherical(pos).y;\n    float r = 0.65 * sin(theta) +\n        2.15 * sqrt(1.0 - 0.75 * pow(cos(theta), 2.0));\n\n    q.x = radius.x * r * cos(theta);\n    q.y = radius.y * r * sin(theta);\n\n   return d -= length(q);\n}\n```\n\nWe are then using the shockwave function from Barney Codes to create a \"ghostly\" shockwave.\n\n```GLSL\nfloat shockwave(vec2 dir, float t) {\n  float d = CraniodSDF(dir/aspect, vec2(t * maxRadius));\n  d *= 1. - 0.9 * smoothstep(0., 0.05, abs(d)); // Mask the ripple\n  d *= smoothstep(0., 0.05, t); // Smooth intro\n  d *= 1. - smoothstep(0.5, 1., t); // Smooth outro\n  return d;\n}\n```\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/faceMesh.jpg\" alt=\"Ghosts in the material world\" width=\"800px\"\u003e\u003c/p\u003e\n\n## Flower Face\n\nFor the flower face, I adapted the [FaceMesh texture sketch](https://editor.p5js.org/codingtrain/sketches/zUKp9n4MW) from Daniel Shiffman to render the the center of a chrysanthemum flower for the face mask. I added \"petals\" around the face by translating to the faceOval keypoints, and adding a rotation based on the location of the keypoint in the faceOval. You can find the p5 sketch [here](https://editor.p5js.org/kfahn/full/dO7nermIUd).\n\n```JavaScript\nfunction petal(x, y, r, a, b, angle) {\n  push();\n  translate(x, y);\n  rotate(radians(angle));\n  beginShape();\n  for (let theta = adj; theta \u003c PI; theta += 0.01) {\n    let x = r * a * sin(theta);\n    let y = r * b * sin(theta) * cos(theta);\n    vertex(x, y);\n  }\n  endShape(CLOSE);\n  pop();\n}\n```\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/flower.jpg\" alt=\"Flower Face\" width=\"800px\"\u003e\u003c/p\u003e\n\n## Pumpkin Face\n\nTo make the pumpkin face, I used the quadrilateral curve to render black triangles at the eyes and nose and used the [bicorn curve](https://mathcurve.com/courbes2d.gb/bicorne/bicorne.shtml) to add a mouth. \n\n```JavaScript\nfunction bicorn(r) {\n  let points = [];\n  for (let theta = 0; theta \u003c TWO_PI; theta += 0.05) {\n    let x = r * sin(theta);\n    let y = (r * pow(cos(theta), 2)) / (2.3 + cos(theta));\n    points.push(createVector(x, y));\n  }\n  return points;\n}\n```\n\nI used the FaceMesh mesh map as a guide for the placement.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/meshmap-pumpkin.jpg\" alt=\"FaceMesh mesh map\" width=\"800px\"\u003e\u003c/p\u003e\n\nIt was a little tricky getting the eyes, nose, and mouth positioned so that there were no distortions in the uvTexture. Once I had the eyes, nose, and mouth positioned properly, I rendered them on top of a pumpkin image.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"pumpkin-face/pumpkin_face.jpg\" alt=\"Pumpkin face\" width=\"800px\"\u003e\u003c/p\u003e\n\nI then adapted the [FaceMesh texture sketch](https://editor.p5js.org/codingtrain/sketches/zUKp9n4MW) from Daniel Shiffman to render the image as a mask. The final touch was to add the pumpkin stem (with the background removed) at the top. You can play with the sketch [here](https://editor.p5js.org/kfahn/full/GyvntE4fG).\n\n```JavaScript\nlet k = face.keypoints[10];\npush();\ntranslate(k.x - w / 2, k.y - w);\nimage(stem, 0, 0, w, w);\npop();\n```\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"assets/pumpkin.jpg\" alt=\"Pumpkin Face\" width=\"800px\"\u003e\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkfahn22%2Fml-facemesh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkfahn22%2Fml-facemesh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkfahn22%2Fml-facemesh/lists"}