{"id":19507052,"url":"https://github.com/jaketarnow/intrusiondetector","last_synced_at":"2026-06-11T13:31:13.877Z","repository":{"id":85809516,"uuid":"71810433","full_name":"jaketarnow/IntrusionDetector","owner":"jaketarnow","description":"Intrusion Detection Learning adapted from http://kdd.ics.uci.edu/databases/kddcup99/task.html","archived":false,"fork":false,"pushed_at":"2016-12-07T21:48:02.000Z","size":1424,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-03-27T23:43:27.535Z","etag":null,"topics":["algorithms","data-science","machine-learning","models","r","statistics"],"latest_commit_sha":null,"homepage":"","language":"R","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaketarnow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-10-24T16:46:52.000Z","updated_at":"2017-11-22T18:10:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"1c938360-3905-4e74-a66e-00b874ee31ca","html_url":"https://github.com/jaketarnow/IntrusionDetector","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jaketarnow/IntrusionDetector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaketarnow%2FIntrusionDetector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaketarnow%2FIntrusionDetector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaketarnow%2FIntrusionDetector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaketarnow%2FIntrusionDetector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaketarnow","download_url":"https://codeload.github.com/jaketarnow/IntrusionDetector/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaketarnow%2FIntrusionDetector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34201840,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","data-science","machine-learning","models","r","statistics"],"created_at":"2024-11-10T22:39:19.509Z","updated_at":"2026-06-11T13:31:13.856Z","avatar_url":"https://github.com/jaketarnow.png","language":"R","funding_links":[],"categories":[],"sub_categories":[],"readme":"# IntrusionDetector\nIntrusion Detection Learning adapted from [KDD](http://kdd.ics.uci.edu/databases/kddcup99/task.html)\n\n## Data\nThe dataset for this project has been supplied via KDD Cup 1999 Data Information and\nComputer Science, University of California, Irvine. Last modified on October 28, 1999. The raw\ntraining data is about 4GB of compressed binary. The TCP dump data contains seven weeks of\nnetwork traffic. The data was processed into roughly five million connection records.\nSource: http://kdd.ics.uci.edu/databases/kddcup99/kddcup99.html\n\n### Features\nGathered from the KDD Cup Data Set, the features outlined by\nStolfo, defined features that help in distinguishing normal connections from bad connection, i.e.\nattacks. They categorized the features into the following: same host, same service, time-based\ntraffic, host-based traffic, and content features. Same host features examine only connections in\nthe past two seconds. These features have the same destination host as the current\nconnections. Same service features examine connections that have the same service as the\ncurrent connection in the past two seconds. Both of these together, same host and same\nservice, are defined as time-based traffic. Host-based traffic on the other hand, involves sorting\nconnection records by destination host. Thus, focusing on the same host instead of a specific\ntime window. Finally, content features are added features that help in determining other\npredictors that may add to certain behaviors in the data.\n\n### Statistically Significant Features \n24 of the 41 predictors are featured here that are significant for the models below\n+flag\n+src_bytes\n+logged_in\n+num_root\n+num_file_creations\n+count\n+srv_count\n+serror_rate\n+srv_serror_rate\n+rerror_rate\n+srv_rerror_rate\n+same_srv_rate\n+diff_srv_rate\n+srv_diff_host_rate\n+dst_host_count\n+dst_host_srv_count\n+dst_host_same_srv_rate\n+dst_host_diff_srv_rate\n+dst_host_same_src_port_rate\n+dst_host_srv_diff_host_rate\n+dst_host_serror_rate\n+dst_host_srv_serror_rate\n+dst_host_rerror_rate\n+dst_host_srv_rerror_rate\n\n## Models\nDue to the fact that our target variable is a class with k \u003e 2 possible values based on the\nnetwork connections and types of attacks, we are considering the following models: Logistic\nRegression, LDA, (linear discriminant analysis) and QDA (quadratic discriminant analysis). If the\ndata was less skewed and did not take hours to run, we would have implemented: SVM,\nBagging, and Random Forests. We believe that those models would be good options for this\ndata, yet testing led to memory errors and system crashes. An alternative for future work is to\nover-sample all of the bootstrapping, or to use no random sampling, with the method call of\nstratified. The use of bagging and random forest would benefit us as we have a large amount of\ndata and a large overhead of computation. As mentioned in previous sections, we need to\ncreate a specific sample size for our model testing, thus stratifying our sampling brings many\nbenefits. The benefits of stratifying the sampling is that it allows us to determine the best\nstratification for our data, yet still ensuring the minimum sample necessary to satisfy our\nconstraints (network connection of good/bad).\n\n## Challenges\nThe main challenge is the amount of time spent for computation. Our current dataset is a\nreduced number of rows, roughly 10% from the original dataset of 400K records. If we work with\na smaller dataset we could possibly get relatively more accurate predictions, yet we still need to\nmeasure skewness. When we find skewness we can then try to balance our data more\nappropriately. As mentioned above, we determined the predictors that caused a significant level\nof skewness and removed them from the dataset after creating our new sample. Through the\nuse of doParallel and iterating through the dataset we create a sample of every possible\ncombination of non-numeric features. This proved to aid in our predictions and gain some solid\ninsight into the data.\n\n## Results\nFrom the above prediction accuracy results, we are able to distinguish that LDA is the best\noption out of the three for predicting the access type of a TCP connection. In our first iteration\nwe found that logistic regression would be the best option. Yet, after balancing the data with our\nnew dataset and sample, we found that LDA best predicts the access type. While gaining a\ngood prediction, users can determine whether an incoming connection is good or bad in\nadvance. This would protect people from various DOS(denial of service), R2L(unauthorized\naccess from remote), U2R(unauthorized access to root), and probing attacks.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaketarnow%2Fintrusiondetector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaketarnow%2Fintrusiondetector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaketarnow%2Fintrusiondetector/lists"}