{"id":22452498,"url":"https://github.com/thomasafink/optimal_knn_distance_for_data_science","last_synced_at":"2026-07-12T17:31:17.483Z","repository":{"id":173108992,"uuid":"538227334","full_name":"ThomasAFink/optimal_knn_distance_for_data_science","owner":"ThomasAFink","description":"Plotting the Optimal Distance for Data Scientists in Python using the K-Nearest Neighbour K-NN Algorithm","archived":false,"fork":false,"pushed_at":"2023-02-06T15:12:39.000Z","size":1370,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-24T09:30:33.443Z","etag":null,"topics":["data-science","data-structures","data-visualization","k-nearest-neighbor","knn","knn-classification","map","mapping","open-street-map","osm","osmnx","python"],"latest_commit_sha":null,"homepage":"https://thomasafink.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ThomasAFink.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-18T20:05:22.000Z","updated_at":"2023-02-06T16:17:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"f6780ad5-f5e7-4987-b66d-3df4bfaae9fa","html_url":"https://github.com/ThomasAFink/optimal_knn_distance_for_data_science","commit_stats":null,"previous_names":["thomasafink/optimal_knn_distance_for_data_science"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ThomasAFink/optimal_knn_distance_for_data_science","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasAFink%2Foptimal_knn_distance_for_data_science","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasAFink%2Foptimal_knn_distance_for_data_science/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasAFink%2Foptimal_knn_distance_for_data_science/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasAFink%2Foptimal_knn_distance_for_data_science/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ThomasAFink","download_url":"https://codeload.github.com/ThomasAFink/optimal_knn_distance_for_data_science/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ThomasAFink%2Foptimal_knn_distance_for_data_science/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35398566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-12T02:00:06.386Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","data-structures","data-visualization","k-nearest-neighbor","knn","knn-classification","map","mapping","open-street-map","osm","osmnx","python"],"created_at":"2024-12-06T06:11:30.125Z","updated_at":"2026-07-12T17:31:17.466Z","avatar_url":"https://github.com/ThomasAFink.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Plotting the Optimal Distance for Data Scientists in Python using the K-Nearest Neighbour K-NN Algorithm\n\n\u003cimg src=\"https://github.com/ThomasAFink/optimal_knn_distance_for_data_science/blob/main/output/knn_map.jpg?raw=true\" width=\"450\" align=\"right\"\u003e\n\nThis example uses the K-Nearest Neighbour K-NN to connect several points and plotting those as a graph on a Plotly map. This is a quick and simple example that data scientists can use to illustrate a graph for their professional or academic papers.\n\nA brief file structure overview of the repository is provided. The knn_map.py is in the root directory. The data folder houses a list of target geocoordinates in a csv file. The out map is generated in the output folder.\n\n    /\n    knn_map.py\n\n    - / data /\n    geocoordinates.csv\n\n    - / output /\n    knn_map.jpg\n  \nThe geocoordinates.csv includes the following target geocoordinates.\n\n    LATITUDE\tLONGITUDE\n    48.13485\t11.5173913\n    48.1348182\t11.577103\n    48.1492002\t11.5592469\n    48.11005\t11.59344\n    48.158903\t11.5856\n  \nBefore jumping into the code the following requirements and packages are needed to run the code:\n\n    Python 3.10.6\n    pip3 install numpy\n    pip3 install scipy\n    pip3 install pandas\n    pip3 install plotly\n    pip3 install kaleido\n\nFirst the packages that were just installed are imported into our file knn_map.py\n\n    import numpy as np\n    from numpy import sqrt \n    from scipy.spatial.distance import pdist, squareform\n    import pandas as pd\n    import plotly.express as px\n    import plotly.graph_objects as go\n\nNext a function generate_knn_grid is created which returns the K-NN grid. This function takes the dataset of geocoordinates in the data folder as a parameter.\n\n    def generate_knn_grid(df): \n\nInside the function the parameters for the KNN algorithm are defined. N is the number of nearest neighbours or nearest nodes we want to connect. As a standard practice K is set to the square root of the number of nearest neighbours. X stacks the geocoordinate data point into a 2D Array.\n\n            # Parameters for the k-nearest neighbor algorithm\n            N = len(list(df.LATITUDE))\n            k = round(sqrt(N))\n            X = np.column_stack((lat, lon))\n\nNext the Euclidean distances between the geocoordinates or nodes can be generated. pdist returns a matrix of distance between each pairs of nodes.\n\n            # matrix of pairwise Euclidean distances will determine which dector node \n            is connected to another dector node\n            distmat = squareform(pdist(X, 'euclidean'))\n\nFor each node the nearest neighbours are determined using the Euclidean distance matrix.\n\n            # select the kNN for each datapoint\n            neighbors = np.sort(np.argsort(distmat, axis=1)[:, 0:k])\n\nUsing the this information the geocoordinates belong to connectors or edges between the nodes can be calculated.\n\n            # get edge coordinates for the lines connecting the nodes\n            coordinates = np.zeros((N, k, 2, 2))\n\n            for i in np.arange(len(list(df.LATITUDE))):\n                for j in np.arange(k):\n                    coordinates[i, j, :, 0] = np.array([X[i,:][1], X[neighbors[i, j], :][1]])\n                    coordinates[i, j, :, 1] = np.array([X[i,:][0], X[neighbors[i, j], :][0]])\n\nThe calculated information is then returned along with N and k.\n\n            return N, k, coordinates\n\nNext the function for plotting the map is created. It takes the returned information from the pervious function as parameters.\n\n      def plot_map(N, k, coordinates):\n\nThe map figure is created and styled using Plotly. An API access token from Mapbox can be acquired for free after signing up. Mapbox has many beautiful maps from a variety of contributors. These look nice in academic papers. Remember to attribute the map creator. Times New Roman was used and the shape was set to a square. The nodes are added from the dataset.\n\n            fig = px.scatter_mapbox(\n                df, \n                lat=\"LATITUDE\", \n                lon=\"LONGITUDE\", \n                zoom=12.2,\n                color_discrete_sequence=[\"rgb(255, 203, 3)\"],\n                title=\"\u003cspan style='font-size: 32px;'\u003e\u003cb\u003eK-Nearest Neighbor KNN\n                Map\u003c/b\u003e\u003c/span\u003e\",\n                opacity=.8,\n                width=1000,\n                height=1000,\n                center=go.layout.mapbox.Center(\n                        lat=48.14,\n                        lon=11.57,\n                    ),\n                size_max=15\n                )\n\n            # Now using Mapbox\n            fig.update_layout(mapbox_style=\"light\", \n                            mapbox_accesstoken=\"\",\n                            legend=dict(yanchor=\"top\", y=1, xanchor=\"left\",x=0.9),\n                            title=dict(yanchor=\"top\", y=.85, xanchor=\"left\",x=0.085),\n                            font_family=\"Times New Roman\",\n                            font_color=\"#333333\",\n                            title_font_size = 32,\n                            font_size = 18)\n\nThe node diameters are set to 15.\n\n            # diameter of the plots \n            fig.update_traces(marker={'size': 15})\n\nUsing the information from the parameters the connectors are generated and added to the map.\n\n            # add line connectors\n            lines = coordinates.reshape((N*k, 2, 2))\n            \n            i = 0\n            for row in lines:\n                fig.add_trace(go.Scattermapbox(lon=[lines[i][0].[0],\n                    lines[i][1][0]], \n                    lat=[lines[i][0][1],\n                    lines[i][1][1]], \n                    mode='lines', \n                    showlegend = False, \n                    line=dict(color='#ffcb03')))\n                i += 1\n\nThe line connectors are z-indexed below the nodes.\n\n            # line connectors layered below plots    \n            fig.data = fig.data[::-1]  \n\nFinally the map is saved in the output folder and rendered in the browser.\n\n            # Save map in output folder\n            print(\"Saving image to output folder...\");\n            fig.write_image('output/knn_map.jpg', scale=5)\n            \n            # Show map in web browser\n            print(\"Generating map in browser...\");\n            fig.show()\n\nAfter defining the functions we import our data from the data directory and format it.\n\n    # Data Import Path\n    SENSORS_CSV   = 'data/geocoordinates.csv'\n    \n    # Data Import Path\n    df = pd.read_csv(SENSORS_CSV)\n    \n    # Keep only relevant columns\n    df = df.loc[:, (\"LATITUDE\", \"LONGITUDE\")]\n    \n    # Remove missing geocoordinates\n    df = df[(df[\"LATITUDE\"] != \"NEIN\") \u0026 (df[\"LONGITUDE\"] != \"NEIN\")]\n    \n    # Format the latitudes and longitudes\n    lon = []; lat = [];\n    \n    for row in df[\"LATITUDE\"]:\n        lat.append(float(row))\n        \n    for row in df[\"LONGITUDE\"]:\n        lon.append(float(row))\n\nFinally we call our two functions to compute and create our KNN map.\n\n    N, k, coordinates = generate_knn_grid(df)\n\n    plot_map(N, k, coordinates)\n\nKNN could be used, for instance, to predict how viruses and other pathogens will move across a given region or spatial area by creating a graph from the individual cases. However for traffic-related forecasting, it is preferable to construct the graph directly from the road network using the Dijkstra algorithm.\n\nSources for KNN: https://stackoverflow.com/questions/50040310/efficient-way-to-connect-the-k-nearest-neighbors-in-a-scatterplot-using-matplotl\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasafink%2Foptimal_knn_distance_for_data_science","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomasafink%2Foptimal_knn_distance_for_data_science","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomasafink%2Foptimal_knn_distance_for_data_science/lists"}