{"id":21456165,"url":"https://github.com/cworld1/gaussian-nb","last_synced_at":"2026-01-07T13:03:01.947Z","repository":{"id":113839428,"uuid":"586344599","full_name":"cworld1/gaussian-nb","owner":"cworld1","description":"Simulating the Parsimonious Bayesian Algorithm with Python and Finding the Probability Prediction of Breast Cancer","archived":false,"fork":false,"pushed_at":"2023-01-08T09:32:19.000Z","size":63,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-27T16:22:13.212Z","etag":null,"topics":["data-science","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cworld1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-07T19:55:17.000Z","updated_at":"2023-04-11T12:35:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"288e7335-47d9-4c16-a3e0-53f11eb74443","html_url":"https://github.com/cworld1/gaussian-nb","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/cworld1/gaussian-nb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fgaussian-nb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fgaussian-nb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fgaussian-nb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fgaussian-nb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cworld1","download_url":"https://codeload.github.com/cworld1/gaussian-nb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cworld1%2Fgaussian-nb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28235502,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2026-01-07T02:00:05.975Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","python"],"created_at":"2024-11-23T05:14:38.050Z","updated_at":"2026-01-07T13:03:01.923Z","avatar_url":"https://github.com/cworld1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Simulating the Parsimonious Bayesian Algorithm with Python and Finding the Probability Prediction of Breast Cancer\n\nThe full probability formula and Bayes' formula are important formulas in probability theory, mainly used to calculate the probability of more complex events, and they are essentially a combination of the addition formula and the multiplication formula. Probability theory and mathematical statistics is a mathematical discipline that studies the statistical regularity of random phenomena and originated in the 17th century. It has developed into many fields of science and society. The two formulas have been studied in many countries from the 17th century to the present in many ways.\n\nOne of the important topics of probability theory is the desire to derive the probability of unknown complex events from the known probability of simple events. To achieve this, a complex event is often divided into a number of mutually incompatible events, and then the final result is obtained by calculating the probabilities of these simple events separately and finally using the additivity of probabilities. This is the basic idea of the full probability formula. Putting the above in order is the full probability formula. The full probability formula is a very important basic formula in probability theory. Through the study of probability theory courses, it is found that there are multiple tolerances that can be further deepened and mined to obtain a broader, more concise, and more practical conclusion to enrich and improve the theoretical system of probability theory. It provides an effective way to calculate the probability of complex events and simplifies the problem of calculating the probability of a complex event. It plays a very important role in probability theory, and the flexible use of the full probability formula will bring us great convenience in solving problems. The full probability formula contains the mathematical ideas of turning whole into zero and complexity into simplicity; the essence of the full probability formula: $P(B)$ in the full probability formula is an average probability, which is the weighted average of the conditional probabilities $P(B|A_i)$, where the weight added to each conditional probability is the probability of the occurrence of the event $A_i$ as a condition.\n\nThe Bayesian formula first appeared in a work by the English scholar T. Bayes (1702-1761) in 1763, after his death. In terms of formal derivation, this formula is bland, it is just a simple derivation of the definition of conditional probability and the full probability formula. What makes it famous is its realistic and even philosophical interpretation: a situation that was thought to be not very likely can become very likely by the occurrence of a certain event; or, on the contrary, Bayes' formula quantitatively The Bayesian formula portrays this change quantitatively.\n\n## Principle chapter\n\n### Conditional probability\n\nConditional probability is the probability that something will happen under certain conditions. For example, the probability of a man confessing his love to a girl is 20%, which is written as $P(A)$ = 20%, where A stands for \"the man confesses his love to the girl\". If the man drives a Mercedes-Benz, the probability of a successful confession to a girl is 50%, then $P(A|B)$ = 50%, where B stands for \"the man drives a Mercedes-Benz\". After all, girls do prefer the grand scene. This $P(A|B)$ is the conditional probability.\n\n### Joint probability\n\nWhat is the joint probability that the man drives a Mercedes and has a 1% chance of confessing his love to the girl, then write $P(AB)$ = 1%. You may wonder, isn't the probability of a successful confession 20%? Isn't the joint probability as high as 50%? Why is the joint probability so low? This may be because the man is particularly poor, the probability of driving a Mercedes is really low so dragged down the joint probability.\n\n### Difference and connection between conditional probability and joint probability\n\nTo summarize, conditional probability is the probability of event A occurring under the condition of B. And the joint probability is the probability that the event A and B occur simultaneously. The two are related and can be expressed by the following formula:\n\n$$\nP(AB) = P(A|B)P(B) = P(B|A)P(A)\n$$\n\nThat is, the joint probability is equal to the conditional probability multiplied by the probability of the condition.\nWhen $A$ and $B$ are independent events, for example, Michelle bought a tape one day in 2001 and at the same time China's Olympic bid was successful, these two events can be considered as independent events.\n\n$$\nP(AB) = P(A|B)P(B) = P(A)P(B)\n$$\n\n### Full probability formula\n\nThe problem is more complicated if the event $B$ is not a condition, but a set of conditions that are mutually exclusive and exhaustive of all possibilities. For example, a boy confesses his love to a girl during the day and courts her at night. The confession is recorded as event $A$, the daytime as event $B_1$, and the nighttime as event $B_2$. It is not difficult to obtain:\n\n$$\nP(A) = P(AB1) + P(AB2) = P(B1|A)P(A) + P(B2|A)P(A) = P(A|B1)P(B1) + P(A|B2)P(B2)\n$$\n\nIts general form is:\n\n$$\nP(A) = \\normalsize\\sum_{1}^{m}P(A|B_{i})P(B_{i})\n$$\n\n### Bayesian formula\n\nThe name of the algorithm, \"plain Bayesian\", comes from the Bayesian formula. To make it easier to understand, instead of using the event $A$ and the event $B$, we use the common $x, y$ for machine learning.\n\n$$\nP(y_{i}|x) = P(x|y_{i})P(y_{i})/P(x)\n$$\n\nExpanding the denominator $P(A)$ according to the full probability formula, we can obtain:\n\n$$\nP(y_{i}|x) = P(x|y_{i})P(y_{i})/\\sum_{1}^{m}P(x|y_{j})P(y_{j})\n$$\n\nThat is to say, if the feature $x$ is known and you want to solve for $y_i$, you only need to know the prior probability $P(y_i)$, and the likelihood $P(x|y_i)$ to solve for the posterior probability $P(y_i|x)$. And for the same sample $x$, $P(x)$ is a constant and can be left out of the calculation.\n\nIn the full probability formula and Bayesian formula, $y_1, y_2, y_3, ... , y_n$ are the various causes that accompany the occurrence of the outcome $x$, and $P(y)$ is the probability of the occurrence of the various causes, which is generally given empirically and is called the prior probability. $P(y_i|x)$ reflects the new result of the probability of occurrence of various conditions after the experiment and can be used to correct $P(y_i)$. The full probability formula is used for \"from cause to effect\" and the Bayesian formula is used for \"from effect to cause\".\n\n### Gaussian distribution\n\nIf $x$ is a continuous variable, how to estimate the likelihood $P(x|y_i)$? We can assume that $x$ obeys the Gaussian distribution (normal distribution) conditional on $y_i$. The $P(x|y_i)$ can be calculated from the probability density function of the normal distribution with the following equation.\n\n$$\nP(x) = \\large\\frac{1}{\\sigma\\sqrt{2\\pi}}\\normalsize e^{-\\frac{(x-\\mu)^{2}}{2\\sigma^2{}}}\n$$\n\n### Gaussian Parsimonious Bayes\n\nIf $x$ is multidimensional data, then we can assume that $P(x_1|y_i),P(x_2|y_i)\\ ... \\ P(x_n|y_i)$ correspond to events that are independent of each other, and these values are multiplied together to obtain $P(x|y_i)$, \"independent of each other\" is the simplicity of plain Bayes.\n\n## Realization chapter\n\n### Create GaussianNB class\n\nInitialize and store the prior probabilities, the mean and variance of the training set and the number of categories of the label.\n\n```Python\nclass GaussianNB:\n    def __init__(self):\n        self.prior = None\n        self.avgs = None\n        self.vars = None\n        self.n_class = None\n```\n\n### Calculate prior probabilities\n\nCalculate the percentage of each category with Python's own Counter, and store the results in a numpy array.\n\n```Python\ndef _get_prior(label: ndarray)-\u003endarray:\n    cnt = Counter(label)\n    prior = np.array([cnt[i] / len(label) for i in range(len(cnt))])\n    return prior\n```\n\n### Calculate the training set mean\n\nCalculate the mean value for each label category separately.\n\n```Python\ndef _get_avgs(self, data: ndarray, label: ndarray)-\u003endarray:\n    return np.array([data[label == i].mean(axis=0) for i in range(self.n_class)])\n```\n\n### Calculate training set variance\n\nThe variance is calculated separately for each label category.\n\n```Python\ndef _get_vars(self, data: ndarray, label: ndarray)-\u003endarray:\n    return np.array([data[label == i].var(axis=0) for i in range(self.n_class)])\n```\n\n### Calculating the likelihood\n\nThe likelihood is calculated from the probability density function of the Gaussian distribution and then multiplied together to obtain the likelihood.\n\n```Python\ndef _get_likelihood(self, row: ndarray)-\u003endarray:\n    return (1 / sqrt(2 * pi * self.vars) * exp(\n        -(row - self.avgs)**2 / (2 * self.vars))).prod(axis=1)\n```\n\n### Training Model\n\n```Python\ndef fit(self, data: ndarray, label: ndarray):\n    self.prior = self._get_prior(label)\n    self.n_class = len(self.prior)\n    self.avgs = self._get_avgs(data, label)\n    self.vars = self._get_vars(data, label)\n```\n\n### Predicting prob\n\nThe prob of each label is obtained by multiplying the prior probability by the likelihood and normalizing it.\n\n```Python\ndef predict_prob(self, data: ndarray)-\u003endarray:\n    likelihood = np.apply_along_axis(self._get_likelihood, axis=1, arr=data)\n    probs = self.prior * likelihood\n    probs_sum = probs.sum(axis=1)\n    return probs / probs_sum[:, None]\n```\n\n### Predict label\n\nFor a single sample, the category corresponding to the maximum value of prob is taken, which is the predicted value of label.\n\n```Python\ndef predict(self, data: ndarray)-\u003endarray:\n    return self.predict_prob(data).argmax(axis=1)\n```\n\n## Effect evaluation\n\n### main function\n\nUse a well-known breast cancer dataset, split into training and test sets in the ratio of 7:3, train the model, and count the accuracy.\n\n```Python\ndef main():\n    print(\"Tesing the performance of Gaussian NaiveBayes...\")\n    data, label = load_breast_cancer()\n    data_train, data_test, label_train, label_test = train_test_split(data, label, random_state=100)\n    clf = GaussianNB()\n    clf.fit(data_train, label_train)\n    y_hat = clf.predict(data_test)\n    acc = _get_acc(label_test, y_hat)\n    print(\"Accuracy is %.3f\" % acc)\n```\n\n### Effect show\n\nACC 0.942, running time 11 milliseconds.\n\n```Bash\npython -u \"d:\\Project\\Python-Project\\Gaussian-NB\\main.py\"\n#\u003e Tesing the performance of Gaussian NaiveBayes...\n#\u003e Accuracy is 0.942\n#\u003e Total run time is 11.2 ms\n```\n\n## Summary\n\nThis paper describes in detail the full probability formula, the applications of the full probability formula, the generalization of the full probability formula and its applications, the Bayesian formula and its connection to the full probability formula, and the generalization theorem of the Bayesian formula and its applications. Through these detailed accounts, it can be seen that the applications of the two probability formulas are multifaceted. The extension of the two probability formulas will further expand the use of the two probability formulas and become an effective tool for us to solve more complex problems. However, due to the short research period, there are still many shortcomings in this study, and this paper only gives a few examples to illustrate their applications, and in fact their applications go far beyond that, and can be used to solve a series of uncertain problems such as investment, insurance, and engineering. What other kinds of problems should be solved with full probability formulas? What kind of problems should be solved with Bayesian formulas? What kind of problems should be solved by combining two formulas? In what kind of problems should the full probability formula or Bayesian formula be applied in several specific steps to solve? None of the paper has come up with a clear method and classification, and these are the issues to be further studied in depth in the future. With the rapid development of society and the increasingly fierce competition in the market, decision makers have to make comprehensive judgments by examining past information and the current situation, and the discipline of probabilistic analysis for decision making has become increasingly important. The use of mathematical methods to quantitatively correlate medical problems gives credibility to their conclusions and facilitates the treatment of patients. In conclusion the proper application of these two probability formulas and their extended forms helps to further study the probability of the target event and its conditions for each induced event in the experiment of multiple stochastic processes, helps to grasp the interactions between random events, provides more valuable decision information for production practice, and then becomes an effective tool for our problem solving.\n\n## License\n\nThe MIT License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcworld1%2Fgaussian-nb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcworld1%2Fgaussian-nb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcworld1%2Fgaussian-nb/lists"}