{"id":38513090,"url":"https://github.com/marcus-k/semfeaturedetection","last_synced_at":"2026-01-17T06:24:18.880Z","repository":{"id":57818232,"uuid":"520666471","full_name":"marcus-k/SEMFeatureDetection","owner":"marcus-k","description":"Collection of work aiming to detect various features in SEM images of nanobeam photonic crystals.","archived":false,"fork":false,"pushed_at":"2022-10-03T19:22:03.000Z","size":31895,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-01-27T06:39:47.595Z","etag":null,"topics":["ellipse-detection","ellipses","image-analysis","opencv","physics","python","research-project"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/marcus-k.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-02T22:27:32.000Z","updated_at":"2023-05-05T14:45:24.000Z","dependencies_parsed_at":"2023-01-19T03:16:17.555Z","dependency_job_id":null,"html_url":"https://github.com/marcus-k/SEMFeatureDetection","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/marcus-k/SEMFeatureDetection","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcus-k%2FSEMFeatureDetection","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcus-k%2FSEMFeatureDetection/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcus-k%2FSEMFeatureDetection/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcus-k%2FSEMFeatureDetection/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/marcus-k","download_url":"https://codeload.github.com/marcus-k/SEMFeatureDetection/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/marcus-k%2FSEMFeatureDetection/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28502214,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T04:31:57.058Z","status":"ssl_error","status_checked_at":"2026-01-17T04:31:45.816Z","response_time":85,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ellipse-detection","ellipses","image-analysis","opencv","physics","python","research-project"],"created_at":"2026-01-17T06:24:18.801Z","updated_at":"2026-01-17T06:24:18.866Z","avatar_url":"https://github.com/marcus-k.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SEM Feature Detection\nCollection of work aiming to detect various features in SEM images of nanobeam photonic \ncrystals, notable the hole sizes and beam width. This work was done in a summer research \nproject from May 2022 to August 2022 under the supervision of Dr. Paul Barclay.\n\nThis will mainly be an archival tool of the progress I make on this. Commits will mainly\nbe various updates as I understand and discover different things. \n\n## Table of Contents\n* [Goals](#goals)\n* [Installation](#installation)\n  * [Installing Python Packages](#installing-python-packages)\n  * [Installing AAMED](#installing-aamed)\n* [Usage](#usage)\n  * [Ellipse Finder](#ellipse-finder)\n  * [Width Finder](#width-finder)\n* [Creating Ground Truth Images](#creating-ground-truth-images)\n  * [ImageJ GT Steps](./ImageJ%20GT%20Steps.md)\n* [Development](#future-development)\n* [Wrap-Up](#wrap-up)\n\n## Goals\n\nThe idea behind this project was to try and create a general purpose application\nwhich can detect and extract various features of SEM images of nanobeams. In particular,\nthe sizes of the holes and width of the beam was of interest. To my knowledge, there\nwas not standardized open source methods to achieve this, so I wanted to make easily \nexpandable and implementable code for others to use. \n\n## Installation\n\nInstallation is broken up into two section: the Python packages and the AAMED algorithm.\nIf the AAMED algorithm is not needed, one can skip installing it.\n\n### Installing Python Packages\n\nThe Anaconda version of Python was used and tested in this work. Instruction for such \nwill be given. If needed, first create a new environment with the desired Python \nversion. Versions 3.8-3.10 have been tested.\n```\nconda create -n ellipse python=3.8\n``` \nNext, install the required packages\n```\nconda install -n ellipse opencv=4.5.5 numpy pandas matplotlib tifffile ipykernel ipympl pysimplegui tabulate \n```\nThis assumes Jupyter Notebooks/Lab is already installed (which it is in the default\nAnaconda setup).\n\nOpenCV version 4.5.5 is specified since the provided AAMED binaries were compiled with\nthis version. Other mismatched versions may work, but to be sure, it is best to compile\nthe AAMED algorithm for that version if the functionality is desired. See the next \nsection for details.\n\n### Installing AAMED\n\n[AAMED](https://github.com/Li-Zhaoxi/AAMED) (Arc Adjacency Matrix Based Fast Ellipse \nDetection) was a key algorithm used when trying to find more sophisticated ellipse \nfinding algorithm. It gave a good point of comparison and for some images worked \nsignificantly better than brute force attempts.\n\nIn order to use the AAMED algorithm, the AAMED binaries should be downloaded and placed \ninto the [`./ellipsefinder/methods`](./ellipsefinder/methods/) folder. The binaries are \nprovided for AAMED in the releases section on the right side. Provided are versions for \nthe Anaconda distribution of Python 3.8-3.10 for Windows and Linux compiled for OpenCV \n4.5.5. The system distribution in Linux should also work. For other distributions, one \nwill need to build the AAMED binary themselves. \n\nCython and a C++ compiler are needed for building. See the \n[AAMED GitHub page](https://github.com/Li-Zhaoxi/AAMED) for building details.\n\nIt is not necessary to install the AAMED binaries. If they are not found, an error will\nbe shown, but the code can be used normally without access to the AAMED algorithm.\n\n## Usage\n\n### Ellipse Finder\n\nThe main code is in [`./ellipsefinder`](./ellipsefinder/) and it consists of the main \n[`find_ellipses.py`](./ellipsefinder/find_ellipses.py) file and the \n[`methods`](./ellipsefinder/methods/) folder. In the \n[`find_ellipses.py`](./ellipsefinder/find_ellipses.py) file is all the functions \nwrapping the functionality of the various ellipse finding algorithms in the \n[`methods`](./ellipsefinder/methods/) folder.\n\nMost of the testing of the algorithms I did with the Jupyter notebooks in \n[`./notebooks`](./notebooks/). It is a bit unorganized, but examples of using \n[`find_ellipses.py`](./ellipsefinder/find_ellipses.py) are best shown in\n[`ellipses.ipynb`](./notebooks/ellipses.ipynb).\n\nIn general, a couple different sections should be written. First, for the images that\nwe have used, we open then and remove any banners that are present. Second, we set\nup any filtering that we want done on the output results. Next, run our selected\ndetection algorithm in the standard way seen in the examples. Finally, we can calculate\nsome things with results and save all the output image.\n\nBelow is an example of the AAMED algorithm used on one of the nanobeams.\n\n![](screenshots/example1.png)\n\n### Width Finder\n\nFor finding the widths, there are two related files. \n[`find_width.py`](./widthfinder/find_width.py) has the functions which are required for \nthe algorithm. The main function goes through the required steps in order. For ease of \nuse, [`find_width_sg.py`](./widthfinder/find_width_sg.py) attempts to wrap the \nfunctionality of in a [PySimpleGUI](https://github.com/PySimpleGUI/PySimpleGUI) GUI \ninterface. Using this may be preferable, but it is still very much in an alpha state.\nA screenshot of the interface is below.\n\n![](screenshots/example2.png)\n\nThe main procedure at the moment is as follows:\n- Remove the banner from the image\n- Preprocess and use OpenCV's HoughLinesP to obtain lines segments of the beam edges\n- Categorize the lines by their slope and intercept\n- Remove any obvious outliers via the interquartile range in the slopes\n- Group and find the four main edges the define the beam via k-means where k=4\n- Manually adjust the intercepts of the lines to better fit the beam\n\nThere are still many improvements to be made, in particular to the outlier removal,\nmanual adjustment of the lines, and grouping via k-means, but this algorithm tended \nto work well for \"nice\" images.\n\nThe main unsolved issue at the moment with the width finding algorithm is that it sorts \nlines by their slopes and intercepts. While this is usually fine, images whose nanobeam \nis vertically aligned cannot be categorized since these properties are undefined. For a \nsimple fix, one can simply manually crop out the banner and rotate the image so the \nalgorithm can proceed. Alternatively, in the GUI version, there is a transpose button \nthat will transpose the image in the backend for use the algorithm. The resultant lines \nare then transposed back so they can be drawn correctly on the image. Since the only \nimportant point is the distance between the lines, transposes should not affect this in \nany way.\n\n## Creating Ground Truth Images\n\nTo compare the accuracy of the ellipse finding methods, and for possible use in future \nmachine learning methods, ground truth images/masks were created using \n[ImageJ](https://imagej.net/software/fiji/downloads). For the details in creating the\nselection regions of interest (ROIs) and masks, see the \n[ImageJ GT Steps.md](./ImageJ%20GT%20Steps.md) file which goes through the steps.\n\nThree files are created from this method: the ground truth mask, the ROI zip file, and \nthe CSV file of the selection regions. I tested out both extraction the ellipses from \nthe mask and creating the ellipse from the CSV selection region data. Both are okay, \nusing [`./ellipsefinder/format_roi_csv.py`](./ellipsefinder/format_roi_csv.py) with the \nCSV data is probably better.\n\nThese images were mainly used as points of reference when testing the binarization \nmethods for each algorithm, but they have great potential to be used as training data \nfor ML models down the line.\n\n## Future Development\n\n### More Algorithms\n\nI tried to make adding new future methods relatively easy. All the current ellipse\nfinder algorithms implement the abstract base class (ABC) \n[`finder.py`](./ellipsefinder/methods/finder.py) which has the various functions needed \nto get going. Bare minimum, the two abstract methods `preprocess()` and `extract()` \nshould be implemented. `preprocess()`, if needed, should return a preprocessed image \nready for the extraction process to start. `extract()` should actually implement the \nextraction process and return the DataFrame with the found ellipses' details. Is this \nthe best way to organize this process? I don't know, but I thought it worked well for \nme.\n\nThe ABC also has a few convenience functions that can be utilized when creating a new\nextraction method, as well as a default function to plot the found ellipses onto the\noriginal image.\n\nA standard that I have used when storing the data in the DataFrame is storing the angle\nof the ellipse with the commonplace counterclockwise positive direction starting from\nthe x-axis. In the OpenCV methods I have used, it adapts a clockwise positive direction\nstarting from the x-axis as its standard. This is something to keep in mind when\ncreating the `extract()` function and storing results.\n\n### Removing Banners\n\nRemoving the banners was an unexpected important step for the algorithms to succeed\nmore. Seen in [`rmbanner.py`](./ellipsefinder/preprocess/rmbanner.py), the current method\nbuilds on [this one](https://github.com/lwang94/sem_size_analysis/blob/803251cdcab3d8304a365df9ac5879fcd9346270/experiments/3_Label_Data.ipynb)\nadding a connected components analysis. At the moment, it is assumed that the banner\nis at the bottom of the image and I simply crop the height of the largest connected\ncomponent off the bottom. For the data I had, this was sufficient but this may need\nto change for banners in different locations.\n\n### Adding Ellipse Finder to GUI\n\nIt was only near the end of the project before I got a chance to add a graphical \ninterface to complement the code. The current width finder GUI is still quite buggy and \nthere are likely many edge cases that have yet to be discovered. As well, the ellipse \nfinder does not have a GUI yet and it would be good to incorporate this together with \nthe width finder GUI. This would also allow the many options of each algorithm to be \neasily changed and dynamically updated. Similar to the structure of the ellipse finder, \nit may be good to have an abstract GUI to inherit from such that each algorithm can \nhave its own set of options to adjust. This however, may also require programming in \nthe edge cases for each state the GUI may be in which is often time-consuming as well.\n\n### Width Finder Limitations\n\nAs mentioned in the [Width Finder](#width-finder) section, the main limitation for this\nalgorithm is the reliance on calculating the slope and y-intercept to remove outliers\nand find the widths. Certainly other methods exist to solve this problem but I have not\nhad the time to explore all the different avenues here. Switching to line segments may\nwork better for distances, but then outlier detection becomes harder to manage.\nExploring different possibilities would be very interesting.\n\n## Wrap-Up\n\nThis work so far is a great exploratory effort in creating a general purpose tool for\ndetecting and extracting features in SEM nanobeam images. An expandable methods for \nmeasuring hole diameters as well as a graphical interface for finding the width was\ncreated here. Going forwards, adding more algorithms, refining the work, and unifying\nboth sections together in a collective GUI would allow for much better efficiency\nand productivity.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcus-k%2Fsemfeaturedetection","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarcus-k%2Fsemfeaturedetection","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarcus-k%2Fsemfeaturedetection/lists"}