{"id":13689380,"url":"https://github.com/graykode/distribution-is-all-you-need","last_synced_at":"2025-05-16T06:07:36.523Z","repository":{"id":39873684,"uuid":"206713596","full_name":"graykode/distribution-is-all-you-need","owner":"graykode","description":"The basic distribution probability Tutorial for Deep Learning Researchers","archived":false,"fork":false,"pushed_at":"2020-10-01T19:14:25.000Z","size":1288,"stargazers_count":1629,"open_issues_count":3,"forks_count":385,"subscribers_count":53,"default_branch":"master","last_synced_at":"2025-04-08T16:06:07.853Z","etag":null,"topics":["deep-learning","distribution","gaussian","mathmatics","probability"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/graykode.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-06T04:35:53.000Z","updated_at":"2025-04-08T07:53:01.000Z","dependencies_parsed_at":"2022-07-06T00:02:07.856Z","dependency_job_id":null,"html_url":"https://github.com/graykode/distribution-is-all-you-need","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2Fdistribution-is-all-you-need","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2Fdistribution-is-all-you-need/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2Fdistribution-is-all-you-need/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/graykode%2Fdistribution-is-all-you-need/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/graykode","download_url":"https://codeload.github.com/graykode/distribution-is-all-you-need/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254478193,"owners_count":22077676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","distribution","gaussian","mathmatics","probability"],"created_at":"2024-08-02T15:01:45.332Z","updated_at":"2025-05-16T06:07:31.494Z","avatar_url":"https://github.com/graykode.png","language":"Python","funding_links":[],"categories":["Tutorials","Python"],"sub_categories":["Machine Learning"],"readme":"## **distribution-is-all-you-need**\n\n**distribution-is-all-you-need** is the basic distribution probability tutorial for **most common distribution focused on Deep learning** using python library.\n\n#### Overview of distribution probability\n\n![](overview.png)\n\n- `conjugate` means it has relationship of **conjugate distributions**.\n\n  \u003e In [Bayesian probability](https://en.wikipedia.org/wiki/Bayesian_probability) theory, if the [posterior distributions](https://en.wikipedia.org/wiki/Posterior_probability) *p*(*θ* | *x*) are in the same [probability distribution family](https://en.wikipedia.org/wiki/List_of_probability_distributions) as the [prior probability distribution](https://en.wikipedia.org/wiki/Prior_probability_distribution) *p*(θ), the prior and posterior are then called **conjugate distributions,** and the prior is called a **conjugate prior** for the [likelihood function](https://en.wikipedia.org/wiki/Likelihood_function).\n  \u003e [Conjugate prior, wikipedia](https://en.wikipedia.org/wiki/Conjugate_prior)\n  \n- `Multi-Class` means that Random Varivance are more than 2.\n\n- `N Times` means that we also consider prior probability P(X). \n\n- To learn more about probability, I recommend reading [pattern recognition and machine learning, Bishop 2006].\n\n\n\n\n## distribution probabilities and features\n\n1. **Uniform distribution(continuous)**, [code](uniform.py)\n   - Uniform distribution has same probaility value on [a, b], easy probability.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/uniform.png\" /\u003e\u003c/p\u003e\n\n2. **Bernoulli distribution(discrete)**, [code](bernoulli.py)\n   - Bernoulli distribution is not considered about prior probability P(X). Therefore, if we optimize to the maximum likelihood, we will be vulnerable to overfitting.\n   - We use **binary cross entropy** to classify binary classification. It has same form like taking a negative log of the bernoulli distribution.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/bernoulli.png\" /\u003e\u003c/p\u003e\n\n3. **Binomial distribution(discrete)**, [code](binomial.py)\n   - Binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments.\n   - Binomial distribution is distribution considered prior probaility by specifying the number to be picked in advance.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/binomial.png\" /\u003e\u003c/p\u003e\n\n4. **Multi-Bernoulli distribution, Categorical distribution(discrete)**, [code](categorical.py)\n   - Multi-bernoulli called categorical distribution, is a probability expanded more than 2.\n   - **cross entopy** has same form like taking a negative log of the Multi-Bernoulli distribution. \n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/categorical.png\" /\u003e\u003c/p\u003e\n\n5. **Multinomial distribution(discrete)**, [code](multinomial.py)\n   - The multinomial distribution has the same relationship with the categorical distribution as the relationship between Bernoull and Binomial.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/multinomial.png\" /\u003e\u003c/p\u003e\n\n6. **Beta distribution(continuous)**, [code](beta.py)\n   - Beta distribution is conjugate to the binomial and Bernoulli distributions.\n   - Using conjucation, we can get the posterior distribution more easily using the prior distribution we know.\n   - Uniform distiribution is same when beta distribution met special case(alpha=1, beta=1).\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/beta.png\" /\u003e\u003c/p\u003e\n\n7. **Dirichlet distribution(continuous)**, [code](dirichlet.py)\n   - Dirichlet distribution is conjugate to the MultiNomial distributions.\n   - If k=2, it will be Beta distribution.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/dirichlet.png\" /\u003e\u003c/p\u003e\n\n8. **Gamma distribution(continuous)**, [code](gamma.py)\n   - Gamma distribution will be beta distribution, if `Gamma(a,1) / Gamma(a,1) + Gamma(b,1)` is same with `Beta(a,b)`.\n   - The exponential distribution and chi-squared distribution are special cases of the gamma distribution.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/gamma.png\" /\u003e\u003c/p\u003e\n\n9. **Exponential distribution(continuous)**, [code](exponential.py)\n   - Exponential distribution is special cases of the gamma distribution when alpha is 1.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/exponential.png\" /\u003e\u003c/p\u003e\n\n10. **Gaussian distribution(continuous)**, [code](gaussian.py)\n    - Gaussian distribution is a very common continuous probability distribution\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/gaussian.png\" /\u003e\u003c/p\u003e\n\n11. **Normal distribution(continuous)**, [code](normal.py)\n    - Normal distribution is standarzed Gaussian distribution, it has 0 mean and 1 std.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/normal.png\" /\u003e\u003c/p\u003e\n\n12. **Chi-squared distribution(continuous)**, [code](chi-squared.py)\n    - Chi-square distribution with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.\n    - Chi-square distribution is special case of Beta distribution\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/chi-squared.png\" /\u003e\u003c/p\u003e\n\n13. **Student-t distribution(continuous)**, [code](student-t.py)\n    - The t-distribution is symmetric and bell-shaped, like the normal distribution, but has heavier tails, meaning that it is more prone to producing values that fall far from its mean.\n\u003cp align=\"center\"\u003e\u003cimg width=\"400\" src=\"graph/student_t.png\" /\u003e\u003c/p\u003e\n\n\n## Author\n\nIf you would like to see the details about relationship of distribution probability, please refer to [this](https://en.wikipedia.org/wiki/Relationships_among_probability_distributions).\n\n![](https://upload.wikimedia.org/wikipedia/commons/thumb/6/69/Relationships_among_some_of_univariate_probability_distributions.jpg/2880px-Relationships_among_some_of_univariate_probability_distributions.jpg)\n\n- Tae Hwan Jung [@graykode](https://github.com/graykode), Kyung Hee Univ CE(Undergraduate).\n- Author Email : [nlkey2022@gmail.com](mailto:nlkey2022@gmail.com)\n- **If you leave the source, you can use it freely.**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraykode%2Fdistribution-is-all-you-need","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgraykode%2Fdistribution-is-all-you-need","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgraykode%2Fdistribution-is-all-you-need/lists"}