{"id":20620537,"url":"https://github.com/blacklight/snort_aipreproc","last_synced_at":"2025-04-15T12:13:44.186Z","repository":{"id":1013725,"uuid":"837575","full_name":"blacklight/Snort_AIPreproc","owner":"blacklight","description":"A preprocessor module for Snort that uses ML algorithms for pruning, clustering and finding correlation between alerts","archived":false,"fork":false,"pushed_at":"2019-06-27T10:10:23.000Z","size":3081,"stargazers_count":31,"open_issues_count":0,"forks_count":10,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-15T12:13:21.780Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://www.fabiomanganiello.com","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blacklight.png","metadata":{"files":{"readme":"README","changelog":"ChangeLog","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2010-08-14T12:28:31.000Z","updated_at":"2025-04-13T13:56:40.000Z","dependencies_parsed_at":"2022-08-16T11:50:13.066Z","dependency_job_id":null,"html_url":"https://github.com/blacklight/Snort_AIPreproc","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blacklight%2FSnort_AIPreproc","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blacklight%2FSnort_AIPreproc/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blacklight%2FSnort_AIPreproc/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blacklight%2FSnort_AIPreproc/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blacklight","download_url":"https://codeload.github.com/blacklight/Snort_AIPreproc/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249067778,"owners_count":21207396,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T12:14:49.515Z","updated_at":"2025-04-15T12:13:44.147Z","avatar_url":"https://github.com/blacklight.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"============================================================================\n   ,,_        ____                   _        _    ___ \n  o\"  )~     / ___| _ __   ___  _ __| |_     / \\  |_ _|\n   ''''      \\___ \\| '_ \\ / _ \\| '__| __|   / _ \\  | | \n              ___) | | | | (_) | |  | |_   / ___ \\ | | \n             |____/|_| |_|\\___/|_|   \\__| /_/   \\_\\___|\n\n\n              _ __  _ __ ___ _ __  _ __ ___   ___ ___  ___ ___  ___  _ __ \n             | '_ \\| '__/ _ \\ '_ \\| '__/ _ \\ / __/ _ \\/ __/ __|/ _ \\| '__|\n             | |_) | | |  __/ |_) | | | (_) | (_|  __/\\__ \\__ \\ (_) | |   \n             | .__/|_|  \\___| .__/|_|  \\___/ \\___\\___||___/___/\\___/|_|   \n             |_|            |_|                                           \n\n ~ A REALLY smart preprocessor module for Snort ~\n by BlackLight \u003cblacklight@autistici.org\u003e, http://0x00.ath.cx\n============================================================================\n\n\nThis document describes the AI preprocessor module for Snort.\nIt also describes how to get it, install it, configure it and use it correctly.\n\nTable of contents:\n\t1. What's Snort AI preprocessor\n\t2. How to get Snort AI preprocessor\n\t3. Installation\n\t\t3.1 Dependancies\n\t\t3.2 Configure options\n\t4. Basic configuration\n\t5. Correlation rules\n\t6. Output database\n\t7. Web interface\n\t8. Additional correlation modules\n\t9. Additional documentation\n\n\n===============================\n1. What's Snort AI preprocessor\n===============================\n\nSnort AI preprocessor is a preprocessor module for Snort whose purpose is making\nthe reading of Snort's alerts more comfortable, clustering false positive alarms\nemphasizing  their  root  cause  in  order  to  reduce log pollution, clustering\nsimilar  alerts  in  function  of the type and hierarchies over IP addresses and\nports  that  can  be  decided  by the user, depending on the kind of traffic and\ntopology  of  the  network, and constructing the flows of a multi-step attack in\nfunction  of  correlation  rules  between  hyperalerts provided by the developer\nitself,  by third parts or created by the user itself, again, in function of the\nscenario  of  the  network.  It will furthermore possible, in a close future, to\ncorrelate  the  hyperalerts  automatically,  by self-learning on the base of the\nacquired alerts.\n\n\n===================================\n2. How to get Snort AI preprocessor\n===================================\n\nIt  it strongly suggested to get the latest and always-fresh release of Snort AI\npreprocessor   from   GitHub   -\u003e   http://github.com/BlackLight/Snort_AIPreproc\n\ngit clone git://github.com/BlackLight/Snort_AIPreproc.git\n\nIf git is not available on the machine or cannot be used, from the same page you\ncan also choose \"download source\" and download the source code in tar.gz format.\n\n\n===============\n3. Installation\n===============\n\nThe installation procedure is the usual one:\n\n$ ./configure\n$ make\n$ make install\n\nIf  you did not install Snort in /usr directory you may need to use the --prefix\noption with configure for selecting the directory where you installed Snort (for\n example    ./configure   --prefix=$HOME/local/snort).   If   the   prefix   was\nspecified  correctly,  and  it  actually  points to the location where Snort was\ninstalled,      the      module     binaries     should     be     placed     in\n$SNORT_DIR/lib/snort_dynamicpreprocessor    after    the    installation,    and\nautomatically  loaded  by  Snort  at  the  next start. Moreover, a new directory\nnamed  corr_rules  will  be  created, in /etc/snort if the prefix was /usr or in\n$SNORT_DIR/etc  otherwise,  containing  XML files describing default correlation\nrules provided by the developer. This set can be enriched in any moment with new\nXML  files,  provided  by  third parts or created by the user itself, describing\nmore hyperalerts.\n\n\n================\n3.1 Dependancies\n================\n\nDependancies required for a correct compilation and configuration:\n\n- pthread (REQUIRED), used for running multiple threads inside of the module. On\na Debian-based  system,  install  libpthread-dev  if  you don't already have it.\n\n- libxml2 (REQUIRED), used for parsing XML files from corr_rules directory. On a\nDebian-based   system,  install  libxml2-dev  if  you  don't  already  have  it.\n\n- libgraphviz  (RECOMMENDED),  used  for  generating  PNG (and in future PS too)\n files    representing   hyperalert   correlation   graphs   from   .dot   files\n generated   from  the  software.  You  can  remove  this  dependancy  from  the\n compilation  process  by  specifying  --without-graphviz to ./configure, but in\n this  case  you  will  have  .dot  files, not easily understandable by a human,\n for  representing  correlation  graphs,  and  you  may  need  an external graph\n rendering  software  for  converting  them in a more easily readable format. On\n a Debian  system,  install  libgraphviz-dev  if  you  don't  already  have  it.\n\n- libmysqlclient  (OPTIONAL),  used if you want to read alerts information saved\non MySQL DBMS, or enable MySQL support in the module. This option is disabled by\ndefault  (if not specified otherwise, the module will read the alerts from Snort\n plain   log   files),   and   can   be   enabled   by   specifying  the  option\n--with-mysql  to  ./configure.  On a Debian-based system you may need to install\nlibmysqlclient-dev.\n\n- libpq  (OPTIONAL),  used  if  you  want  to  read  alerts information saved on\nPostgreSQL  DBMS,  or  enable  PostgreSQL  support in the module. This option is\ndisabled   by  the  default,  and  can  be  enabled  by  specifying  the  option\n--with-postgresql  to  ./configure.  On  a  Debian-based  system you may need to\ninstall libpq-dev.\n\n- A  DBMS (RECOMMENDED), MySQL and PostgreSQL are supported for now, for writing\nclusters,  correlations  and  packet  streams  information on a DBMS, making the\nanalysis                                                                 easier.\n\n- Perl  (RECOMMENDED),  used  for  the  CGI  script  in  the  web interface that\nsaves  a  packet  stream  associated to an alert in .pcap format, to be analyzed\nby            tools           like             tcpdump         and     Wireshark.\n\n- XML::Simple  Perl module (RECOMMENDED), used by 'correlate.cgi' CGI script for\nreading  and  writing  manual  (un)correlations  XML  files.  A  quick  way  for\ninstalling     it     on     a     Unix     system    is    by    using    CPAN.\n\n- Python  2.6  (OPTIONAL), used for interfacing SnortAI module to Python scripts\nthrough   snortai  module  (see  README  file  in  pymodule/)  and  writing  new\ncorrelation       modules      (see      example.py      in      corr_modules/).\nCompile the module passing --with-python option to the ./configure script if you\nwant  this  feature.  You  need Python interpreter and libpython2.6 installed on\nyour system.\n\n# cpan XML::Simple\n\n=====================\n3.2 Configure options\n=====================\n\nYou  can  pass  the  following  options  to ./configure script before compiling:\n\n--with-mysql  -  Enables  MySQL  DBMS  support  into  the  module  (it  requires\n libmysqlclient)\n\n--with-pq  - Enables PostgreSQL DBMS support into the module (it requires libpq)\n\n--without-graphviz  -  Disables  Graphviz  support from the module, avoiding the\ngeneration  of  PNG  or  PS  files  representing hyperalerts correlation as well\n\n\n======================\n4. Basic configuration\n======================\n\nAfter  installing the module in Snort installation directory a configuration for\nthis  is  required  in  snort.conf.  A  sample configuration may appear like the\nfollowing:\n\n\npreprocessor ai: \\\n\talertfile \"/your/snort/dir/log/alert\" \\\n\talert_bufsize 30 \\\n\talert_clustering_interval 300 \\\n\talert_correlation_weight 5000 \\\n\talert_history_file \"/your/snort/dir/log/alert_history\" \\\n\talert_serialization_interval 3600 \\\n\tbayesian_correlation_interval 1200 \\\n\tbayesian_correlation_cache_validity 600 \\\n\tcluster ( class=\"dst_port\", name=\"privileged_ports\", range=\"1-1023\" ) \\\n\tcluster ( class=\"dst_port\", name=\"unprivileged_ports\", range=\"1024-65535\" ) \\\n\tcluster ( class=\"src_addr\", name=\"local_net\", range=\"192.168.1.0/24\" ) \\\n\tcluster ( class=\"src_addr\", name=\"dmz_net\", range=\"155.185.0.0/16\" ) \\\n\tcluster ( class=\"src_addr\", name=\"vpn_net\", range=\"10.8.0.0/24\" ) \\\n\tcluster ( class=\"dst_addr\", name=\"local_net\", range=\"192.168.1.0/24\" ) \\\n\tcluster ( class=\"dst_addr\", name=\"dmz_net\", range=\"155.185.0.0/16\" ) \\\n\tcluster ( class=\"dst_addr\", name=\"vpn_net\", range=\"10.8.0.0/24\" ) \\\n\tcluster_max_alert_interval 14400 \\\n\tclusterfile \"/your/snort/dir/log/clustered_alerts\" \\\n\tcorr_modules_dir \"/your/snort/dir/share/snort_ai_preproc/corr_modules\" \\\n\tcorrelation_graph_interval 300 \\\n\tcorrelation_rules_dir \"/your/snort/dir/etc/corr_rules\" \\\n\tcorrelated_alerts_dir \"/your/snort/dir/log/correlated_alerts\" \\\n\tcorrelation_threshold_coefficient 0.5 \\\n\tdatabase ( type=\"dbtype\", name=\"snort\", user=\"snortusr\", password=\"snortpass\", host=\"dbhost\" ) \\\n\tdatabase_parsing_interval 30 \\\n\thashtable_cleanup_interval 300 \\\n\tmanual_correlations_parsing_interval 120 \\\n\tmax_hash_pkt_number 1000 \\\n\tneural_clustering_interval 1200 \\\n\tneural_network_training_interval 43200 \\\n\tneural_train_steps 10 \\\n\toutput_database ( type=\"dbtype\", name=\"snort\", user=\"snortusr\", password=\"snortpass\", host=\"dbhost\" ) \\\n\toutput_neurons_per_side 20 \\\n\ttcp_stream_expire_interval 300 \\\n\tuse_knowledge_base_correlation_index 1 \\\n\tuse_stream_hash_table 1 \\\n\twebserv_banner \"Snort AIPreprocessor module\" \\\n\twebserv_dir \"/prefix/share/htdocs\" \\\n\twebserv_port 7654\n\n\nThe options are the following:\n\n- alertfile:  The file where Snort saves its alerts, if they are saved to a file\nand   not  to  a  database  (default  if  not  specified:  /var/log/snort/alert)\n\n\n- alert_correlation_weight:  When this number of alert is stored in the \"memory\"\nof  the software (i.e. in the alert history file or in the output database), the\nweight  for  the  heuristical  correlation  indexes (bayesian network and neural\n network)  will  be  more  or  less  equal  to  0.95,  on  a  scale from 0 to 1.\nThis parameter expresses how much the heuristical indexes should be weighted and\nit  can  be  considered like a kind of \"learning rate\" for the alert correlation\nalgorithm       (default       value      if      not      specified:      5000)\n\n\n- alert_history_file:  The   file keeping track of the history, in binary format,\nof   all  the  alerts  received  by  the  IDS, so that the module can build some\nstatistical        correlation        inferences       over       the       past\n\n\n- alert_serialization_interval:   The   interval   that   should  occur  from  a\nserialization  of  a  buffer  of  alerts  on  the  history file and the next one\n(default if not specified: 1 hour, as it is a quite expensive operation in terms\nof       resources      if      the     system     received     many     alerts)\n\n\n- alert_bufsize:  Size of the buffer containing the alerts to be sent, in group,\nto   the  serializer  thread.  The  buffer is sent when full and made empty even\nwhen   the  alert_serialization_interval  parameter  is  not  expired  yet,  for\navoiding   overflows,  other  memory  problems  or  deadlocks  (default value if\nnot                                 specified:                               30)\n\n\n- alert_clustering_interval:  The interval that should occur from the clustering\nof  the  alerts  in the log according to the provided clustering hierarchies and\nthe     next     one     (default     if    not    specified:    300    seconds)\n\n\n- bayesian_correlation_interval: Interval, in seconds, that should occur between\ntwo  alerts  in  the  history  for  considering  them as, more or less strongly,\ncorrelated (default: 1200 seconds). NOTE: A value of 0 will disable the bayesian\ncorrelation.  This  setting  is  strongly suggested when your alert log is still\n\"learning\",  i.e.  when you don't have enough alerts yet. After this period, you\ncan      set      the      correlation      interval      to      any     value.\n\n\n- bayesian_correlation_cache_validity:  interval, in seconds, for which an entry\nin  the  bayesian  correlation  hash  table  (i.e.  a  pair  of  alerts with the\n associated   historical   bayesian   correlation)   is   considered   as  valid\nbefore         being         updated        (default:        600        seconds)\n\n\n- corr_modules_dir:  This  software supports a kind of plugins, or \"modules over\nthe  module\",  that  allow  the user to specify some extra correlation rules and\nindexes.  These  modules  are .so files placed in this directory (default if not\n specified:  PREFIX/share/snort_ai_preproc/corr_modules),  dynamically loaded by\nthe  module.  For  more  information  on  how  to write your own module, see the\ndedicated              section             in             this             file.\n\n\n- correlation_graph_interval:  The  interval that should occur from the building\nof  the correlation graph between the clustered alerts and the next one (default\n if              not             specified:             300             seconds)\n\n\n- correlation_rules_dir: Directory where the correlation rules are saved, as XML\nfiles      (default      if      not      specified:      /etc/snort/corr_rules)\n\n\n- correlated_alerts_dir:  Directory  where  the  information  between correlated\nalerts  will  be  saved,  as  .dot  files ready to be rendered as graphs and, if\nlibgraphviz  support  is  enabled, as .png and .ps files as well (default if not\n specified: /var/log/snort/clustered_alerts)\n\n\n- correlation_threshold_coefficient: The threshold the software uses for stating\ntwo   alerts   are   correlated   is   avg(correlation   coefficient)   +   k  *\nstd_deviation(correlation_coefficient). The value of k is specified through this\noption,  whose  value  is 0.5 by default, and is dependant on how \"sensible\" you\nwant  the  correlation algorithm. A value of k=0 means \"consider all the couples\nof  alerts  whose  correlation  coefficient  is  greater than the average one as\ncorrelated\"  (negative values of k are allowed as well, but unless you know what\nyou're  doing  they're unrecommended, as some correlation constraints may appear\nwhere  no correlation exists). When the value of k raises also the threshold for\ntwo alerts for being considered as correlated raises. A high value of k may just\nlead           to          an          empty          correlation          graph\n\n\n- clusterfile:  File  where  the  clustered  alerts  will be saved by the module\n(default       if      not      specified:      /var/log/snort/clustered_alerts)\n\n\n- cluster_max_alert_interval:   Maximum  time  interval,  in  seconds,  occurred\nbetween  two  alerts  for considering them as part of the same cluster (default:\n14400   seconds,  i.e.  4  hours).  Specify  0  for  this  option if you want to\ncluster   alerts   regardlessly   of   how   much  time  occurred  between  them\n\n\n- cluster:  Clustering  hierarchy  or  list  of  hierarchies  to  be applied for\ngrouping     similar     alerts.     This     option     needs    to    specify:\n\t-- class: Class of the cluster node. It may be src_addr, dst_addr, src_port\n\t\t\tor dst_port\n\t-- name: Name for the clustering node\n\t-- range: Range of the clustering node. It can include a single port or IP\n\t\t\taddress, an IP range (specified as subnet x.x.x.x/x), or a port\n\t\t\trange (specified as xxx-xxx)\n\n\n- database:  If Snort saves its alerts to a database and the module was compiled\n with   database   support   (e.g.   --with-mysql)  this  option  specifies  the\n information   for   accessing   that   database.   The   fields   in  side  are\n\t-- type: DBMS to be used (so far MySQL and PostgreSQL are supported)\n\t-- name: Database name\n\t-- user: Username for accessing the database\n\t-- password: Password for accessing the database\n\t-- host: Host holding the database\n\n\n- database_parsing_interval:  The  interval  that should occur between a read of\nthe alerts from database and the next one (default if not specified: 30 seconds)\n\n\n- hashtable_cleanup_interval: The interval that should occur from the cleanup of\nthe  hashtable  of  TCP  streams and the next one (default if not specified: 300\nseconds).  Set  this  option  to  0 for performing no cleanup on the stream hash\ntable\n\n\n- max_hash_pkt_number: Maximum number of packets that each element of the stream\nhash  table  should  hold,  set  it  to  0  for  no  limit (default value if not\nspecified: 1000)\n\n\n- manual_correlations_parsing_interval: Interval in seconds between an execution\nof  the  thread for parsing the alert correlations manually set and the next one\n(default       value       if       not       specified:       120      seconds)\n\n\n- neural_clustering_interval:  Interval  in  seconds between an execution of the\nthread  for  clustering  (using  k-means)  the alerts on the output layer of the\nneural  network in order to recognize likely attack scenarios, and the next one.\nSet  this  to  0  if  you want no clusterization (default if not specified: 1200\nseconds)\n\n\n- neural_network_training_interval:  Interval in seconds between an execution of\nthe  thread  for  training the neural network using the set of recent alerts and\nthe     next     one    (default    if    not    specified:    43200    seconds)\n\n\n- neural_train_steps:  Number  of  steps  to take in each training cycle for the\nneural                   network                  (default:                  10)\n\n\n- output_database:  Specify this option if you want to save the outputs from the\nmodule  (correlated  alerts,  clustered  alerts,  alerts  information  and their\nassociated    packets   streams,   and  so  on)  to  a  relational  database  as\nwell  (by  default  the module only saves the alerts on static plain files). The\noptions    here   are   the   same   specified   for   the   'database'   option.\nThe  structure  of this database can be seen in the files schemas/*.sql (replace\nto  * the name of your DBMS). If you want to initialize the tables needed by the\nmodule,   just   give   the   right  file  to  your  database,  e.g.  for  MySQL\n$ mysql -uusername -ppassword dbname \u003c schemas/mysql.sql\n\n\n- output_neurons_per_side: Number of output neurons per side on the output layer\nof  the  neural network (that is a rectangular matrix). A higher number allows a\nhigher  granularity  over  similar  alerts, but a linear increment of this value\nproduces  a  squared  increment of the computational complexity for the training\nand    evaluation    algorithms   (default   value   if   not   specified:   20)\n\n\n- tcp_stream_expire_interval:  The  interval that should occur for marking a TCP\nstream as \"expired\", if no more packets are received inside of that and it's not\n\"marked\"    as    suspicious   (default   if   not   specified:   300   seconds)\n\n\n- use_knowledge_base_correlation_index: Set this option to 0 if you do not want\nto use the knowledge base alert correlation index (default value if not\nspecified: 1)\n\n\n- use_stream_hash_table: Set this option to 0 if you do not want to use the\nhash table for storing the streams of packets associated to alerts, this is a\ngood choice on a system where many alerts are triggered (default value if not\nspecified: 1)\n\n\n- webserv_banner:  Banner of the web server, to be placed on the error pages and\nin           the           \"Server\"          HTTP          reply          header\n\n\n- webserver_dir:  Directory  containing  the  contents of the web server running\nover      the      module      (default      if      none      is     specified:\n$PREFIX/share/snort_ai_preprocessor/htdocs)\n\n\n- webserver_port:  Port  where  the  web  server will listen (default if none is\nspecified:  7654).  Set  this value to 0 if you don't want to run the web server\nover  the  module for having the web interface (in this case, if you want to see\nthe  web  graphical  visualization  of  the alerts, you should manually copy the\nfiles     contained     in     htdocs/    in    a    web    server    directory)\n\n====================\n5. Correlation rules\n====================\n\nThe  hyperalert  correlation  rules  are  specified in $SNORT_DIR/etc/corr_rules\ndirectory  through a very simple XML syntax, and any user can add some new ones.\nThe files in there must be named like the Snort alert ID they want to model. For\nexample,  if  we  want  to  model  a TCP portscan alert (Snort ID: 122.1.0) as a\nhyperalert,  i.e.  as  an  alert  with  pre-conditions and post-conditions to be\ncorrelated  to  other  alerts,  then  we need to create a file named 122-1-0.xml\ninside   $SNORT_DIR/etc/corr_rules   with   a   content   like   the  following:\n\n\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003c!DOCTYPE hyperalert PUBLIC \"-//blacklighth//DTD HYPERALERT SNORT MODEL//EN\" \"http://0x00.ath.cx/hyperalert.dtd\"\u003e\n\n\u003chyperalert\u003e\n\t\u003csnort-id\u003e122.1.0\u003c/snort-id\u003e\n\t\u003cdesc\u003e(portscan) TCP Portscan\u003c/desc\u003e\n\n\t\u003cpre\u003eHostExists(+DST_ADDR+)\u003c/pre\u003e\n\t\u003cpost\u003eHasService(+DST_ADDR+, +ANY_PORT+)\u003c/post\u003e\n\u003c/hyperalert\u003e\n\n\nThe  \u003cdesc\u003e  tag  is  optional,  same  for  \u003cpre\u003e  and \u003cpost\u003e if an alert has no\npre-conditions and/or post-conditions, while the \u003csnort-id\u003e tag is mandatory for\nidentifying the hyperalert. In this case we say that the pre-condition for a TCP\nportscan  for  being  successful  is  that the host +DST_ADDR+ exists (the macro\n +DST_ADDR+   will   automatically   be  expanded  at  runtime  and  substituted\n with   the   target   address   of  the  portscan).  The  post-condition  of  a\nportscan  consists  in  the  attacker  knowing that +DST_ADDR+ runs a service on\n+ANY_PORT+  (+ANY_PORT+  is another macro that will be expanded on runtime). The\nhyperalerts  model  in  corr_rules  are  the knowledge base used for correlating\nalerts triggered by Snort, the more information it has inside, the more accurate\nand  complete  the  correlation will be. The macros recognized and automatically\nexpanded           from           these          XML          files          are\n\n\n- +SRC_ADDR+: The IP address triggering the alert\n- +DST_ADDR+: The target IP address in the alert\n- +SRC_PORT+: The port from which the alert was triggered\n- +DST_PORT+: The port on which the alert was triggered\n- +ANY_ADDR+: Identifies any IP address\n- +ANY_PORT+: Identifies any port\n\n\nThe correlation between two alerts A and B is made comparing the post-conditions\nof  A  with  the  pre-conditions of B. If the correlaton coefficient computed in\nthis  way  is  greater  than  a  certain  threshold (see \"Basic configuration -\u003e\n correlation_threshold_coefficient\")    then    the   alerts   are   marked   as\ncorrelated, i.e. the alert A determines the alert B. This supports an elementary\nreasoning  algorithm  for  doing inferences on the conditions. For example, if A\nhas  the  post-condition  \"HasService(+DST_ADDR+,  +ANY_PORT+)\"  and  B  has the\npre-condition \"HasService(+DST_ADDR, 22)\", a match between A and B is triggered.\nSame   if   A   has   \"HostExists(10.8.0.0/24)\"  as  post-condition  and  B  has\n\"HostExists(10.8.0.1)\" as pre-condition.\n\nThere  is  no  fixed  semantics  for  the  the  predicates in pre-conditions and\npost-conditions,  any  predicates  can  be used. The only constraint is that the\nsame  predicates have the same semantic and prototype in all of the hyperalerts.\nFor  example,  if  HasService  has a prototype like \"HasService(ip_addr, port)\",\nthen   all    the  hyperalerts  should  follow  this  prototype,  otherwise  the\nmatching    would   fail.   Any  new  predicates  can  be  defined  as  well  in\nhyperalerts,      provided     that     it     respects     this     constraint.\n\n\n==================\n6. Output database\n==================\n\nIf the output_database option is specified in the documentation, the alerts, and\nthe  relative  clusters,  correlations  and  packet streams information, will be\nsaved  on  a  database as well. This is strongly suggested, first for making the\nmanagement  of  the module's information easier (a SELECT query needs to be done\n for   doing   also   complex   searches   instead   of   grep-ing  or  manually\n searching  inside  of  a  text  file),  second  because  the  web  interface of\nthe  module  can  work  ONLY if the output_database option is specified (the web\n interface  strongly  depends  on  the  unique  IDs  assigned  to  the alerts by\n the   database   interface).  Note  that  for  using  this  option  you  should\nexplicitly  tell  to  the  ./configure script which DBMS you're going to use, so\nthat  it  knows  which  APIs  to use for interfacing with the database, e.g. via\n--with-mysql                        or                        --with-postgresql.\n\nAfter  you  compile  the  module,  you  should  pick up the right .sql file from\nschemas/   directory   (for   example  mysql.sql  or  postgresql.sql),  or  from\n$PREFIX/share/snort_ai_preprocessor/schemas   after   the  installation  of  the\nmodule,        and        import        it        in        your       database,\n\n$ mysql -uusername -ppassword dbname \u003c schemas/mysql.sql (for MySQL)\n$ psql -U username -W dbname \u003c schemas/postgresql.sql (for PostgreSQL)\n\nYou  can check the structure of the database from the SQL file for your DBMS, or\nfrom      the      E/R      schema     saved     in     schemas/database_ER.png.\n\n\n================\n7. Web interface\n================\n\nThe  module  provides  an  optional (but strongly recommended) web interface for\nbrowsing   the   triggered   (and  already  clustered)  security  alerts,  their\ncorrelations  and  their  packet  streams  information  from  your browser. This\nfeature  can  be switched off by setting the configuration option \"webserv_port\"\nof  the module to 0. Otherwise, if none between webserv_dir and webserv_port are\nspecified,  the  web server thread starts with the module picking by default the\ndirectory  $PREFIX/share/snort_ai_preproc/htdocs  as document root and listening\nfor        incoming        connections       on       the       port       7654.\n\nYou  should  use  a  browser supporting JavaScript, AJAX and SVG technologies in\norder  to  view  correctly the alert web interface on your browser (successfully\n tested  with  Firefox  3.5,  Chrome  and  Opera  10),  for  example, connecting\nto  the  address  http://localhost:7654.  You can drag and drop the nodes in the\ngraph,  modifying  the  layout  of  the  graph  on the fly or using the \"redraw\"\nfunction.  Each  node  represents a clustered alert. For viewing the information\nover  that  cluster and the alerts group inside, just click on the node. You can\noptionally  save  the  stream  of packets associated to a certain alert in .pcap\nformat  (analyzable  by  tools  like  tcpdump  and  Wireshark)  from  this  same\ninterface.  This  feature, anyway, is based on the CGI script pcap.cgi inside of\nthe  document  root, and it requires the Perl interpreter to be installed on the\nmachine.\n\nThe  web  server  running  over  the  module  is  a true web server with its own\ndocument  path,  so  you  can use it as stand-alone web server as well and place\nyour  documents  and  files  inside.  You can moreover place some CGI scripts or\napplications  made  in  the  language  you  prefer,  as  long  as they are files\nexecutable    by    any    users   and   they   have   the   extension   \".cgi\".\n\nA powerful featured offered by the web interface is the one that allows the user\nto manually \"mark\" two alerts as correlated, if the system didn't do that, or as\nnot  correlated,  if  the  system  made  a  mistake correlating two uncorrelated\nalerts.  These decisions are made simply by clicking the right button on the web\npage  and  clicking  the two alerts to mark as correlated or uncorrelated. After\nthat,  all  the  alerts  of  those  types  will  be  marted  as  correlated,  or\nuncorrelated.\n\n\n=================================\n8. Additional correlation modules\n=================================\n\nIt  is  possible  to  add  extra  parameters  and  indexes  for  evaluating  the\ncorrelation  between  two  alerts  in  an  extremely  simple  way. The directory\nspecified  in  the  configuration  option  \"corr_modules_dir\" contains the extra\nmodules  (as  binary  shared  libraries  -\u003e  .so).  Each of these modules should\ncontain          a          function          whose         prototype         is\n\n\ndouble  AI_corr_index ( AI_snort_alert*, AI_snort_alert* )\n\n\ntaking  two alerts as parameters and returning a correlation value between them,\nand               one              whose              prototype              is\n\n\ndouble  AI_corr_index_weight ()\n\n\nreturning  a  coefficient  in  [0,1]  expressing  the  weight  of that index. An\nexample  module  is  contained  in  the  corr_modules  directory  in  the source\ndirectory,  or in PREFIX/share/snort_ai_preproc/corr_modules after installation.\n\nWhen  you  write  your  own module, just add in the Makefile in the corr_modules\ndirectory  a  line  like  the one already present there for compiling, then type\n`make'.   You   may   need   to   link   your   module  source  file(s)  against\nlibsf_ai_preproc.la  if  you  want to use some of the functions from the module,\n for  example,  for  reading  the  alerts  stored  in  the  history file, in the\n database,       the       current       correlations,      and      so      on.\n\nIt  is  also possible to write your own modules in Python language. See the file\n'example_module.py'  in  corr_modules/  for  a  quick  overview. All you need to\ndo  is  to  declare  in  your  module  the  functions  AI_corr_index (taking two\narguments,      two     alert     descriptions)     and     AI_corr_index_weight\n(taking     no    argument),    both    returning   a   real   value   descibing,\nrespectively,    the   correlation   value   between  the  two  alerts  and  the\nweight  of  that   index,  both  between  0  and  1.  You  can  also  access the\nalert   information    and  all  the  alerts  acquired  so  far  by  the  module\nby   importing   in    your   Python   code   the   'snortai'  module.  You  can\ncompile     it     and      install     it     by    moving    to    'pymodule/'\ndirectory and running\n\n$ python setup.py build\n$ [sudo] python setup.py install\n\nYou  can  acquire  the  current  alerts  by  writing  a code like the following:\n\nimport snortai\n\nalerts = snortai.alerts()\n\nfor alert in alerts:\n\t# Access the alerts information\n\nThe     fields     in     the     alert     class     can     be    viewed    in\npymodule/test.py   and   corr_modules/example_module.py   examples.  Take  these\nfiles  as  guides  for  interfacing  your  Python  scripts  with  SnortAI module\nor       writing       new       correlation       modules       in      Python.\n\n\n===========================\n9. Additional documentation\n===========================\n\nThe additional documentation over the code, functions and data structures can\nbe automatically generated by Doxygen by typing `make doc', and installed  in\n$PREFIX/share/snort_ai_preproc/doc then after `make install'.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblacklight%2Fsnort_aipreproc","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblacklight%2Fsnort_aipreproc","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblacklight%2Fsnort_aipreproc/lists"}