Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/graykode/distribution-is-all-you-need
The basic distribution probability Tutorial for Deep Learning Researchers
https://github.com/graykode/distribution-is-all-you-need
deep-learning distribution gaussian mathmatics probability
Last synced: 13 days ago
JSON representation
The basic distribution probability Tutorial for Deep Learning Researchers
- Host: GitHub
- URL: https://github.com/graykode/distribution-is-all-you-need
- Owner: graykode
- License: mit
- Created: 2019-09-06T04:35:53.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2020-10-01T19:14:25.000Z (about 4 years ago)
- Last Synced: 2024-10-15T19:03:23.818Z (27 days ago)
- Topics: deep-learning, distribution, gaussian, mathmatics, probability
- Language: Python
- Homepage:
- Size: 1.23 MB
- Stars: 1,621
- Watchers: 53
- Forks: 385
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## **distribution-is-all-you-need**
**distribution-is-all-you-need** is the basic distribution probability tutorial for **most common distribution focused on Deep learning** using python library.
#### Overview of distribution probability
- `conjugate` means it has relationship of **conjugate distributions**.
> In [Bayesian probability](https://en.wikipedia.org/wiki/Bayesian_probability) theory, if the [posterior distributions](https://en.wikipedia.org/wiki/Posterior_probability) *p*(*θ* | *x*) are in the same [probability distribution family](https://en.wikipedia.org/wiki/List_of_probability_distributions) as the [prior probability distribution](https://en.wikipedia.org/wiki/Prior_probability_distribution) *p*(θ), the prior and posterior are then called **conjugate distributions,** and the prior is called a **conjugate prior** for the [likelihood function](https://en.wikipedia.org/wiki/Likelihood_function).
> [Conjugate prior, wikipedia](https://en.wikipedia.org/wiki/Conjugate_prior)
- `Multi-Class` means that Random Varivance are more than 2.
- `N Times` means that we also consider prior probability P(X).
- To learn more about probability, I recommend reading [pattern recognition and machine learning, Bishop 2006].
## distribution probabilities and features
1. **Uniform distribution(continuous)**, [code](uniform.py)
- Uniform distribution has same probaility value on [a, b], easy probability.
2. **Bernoulli distribution(discrete)**, [code](bernoulli.py)
- Bernoulli distribution is not considered about prior probability P(X). Therefore, if we optimize to the maximum likelihood, we will be vulnerable to overfitting.
- We use **binary cross entropy** to classify binary classification. It has same form like taking a negative log of the bernoulli distribution.
3. **Binomial distribution(discrete)**, [code](binomial.py)
- Binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent experiments.
- Binomial distribution is distribution considered prior probaility by specifying the number to be picked in advance.
4. **Multi-Bernoulli distribution, Categorical distribution(discrete)**, [code](categorical.py)
- Multi-bernoulli called categorical distribution, is a probability expanded more than 2.
- **cross entopy** has same form like taking a negative log of the Multi-Bernoulli distribution.
5. **Multinomial distribution(discrete)**, [code](multinomial.py)
- The multinomial distribution has the same relationship with the categorical distribution as the relationship between Bernoull and Binomial.
6. **Beta distribution(continuous)**, [code](beta.py)
- Beta distribution is conjugate to the binomial and Bernoulli distributions.
- Using conjucation, we can get the posterior distribution more easily using the prior distribution we know.
- Uniform distiribution is same when beta distribution met special case(alpha=1, beta=1).
7. **Dirichlet distribution(continuous)**, [code](dirichlet.py)
- Dirichlet distribution is conjugate to the MultiNomial distributions.
- If k=2, it will be Beta distribution.
8. **Gamma distribution(continuous)**, [code](gamma.py)
- Gamma distribution will be beta distribution, if `Gamma(a,1) / Gamma(a,1) + Gamma(b,1)` is same with `Beta(a,b)`.
- The exponential distribution and chi-squared distribution are special cases of the gamma distribution.
9. **Exponential distribution(continuous)**, [code](exponential.py)
- Exponential distribution is special cases of the gamma distribution when alpha is 1.
10. **Gaussian distribution(continuous)**, [code](gaussian.py)
- Gaussian distribution is a very common continuous probability distribution
11. **Normal distribution(continuous)**, [code](normal.py)
- Normal distribution is standarzed Gaussian distribution, it has 0 mean and 1 std.
12. **Chi-squared distribution(continuous)**, [code](chi-squared.py)
- Chi-square distribution with k degrees of freedom is the distribution of a sum of the squares of k independent standard normal random variables.
- Chi-square distribution is special case of Beta distribution
13. **Student-t distribution(continuous)**, [code](student-t.py)
- The t-distribution is symmetric and bell-shaped, like the normal distribution, but has heavier tails, meaning that it is more prone to producing values that fall far from its mean.
## Author
If you would like to see the details about relationship of distribution probability, please refer to [this](https://en.wikipedia.org/wiki/Relationships_among_probability_distributions).
- Tae Hwan Jung [@graykode](https://github.com/graykode), Kyung Hee Univ CE(Undergraduate).
- Author Email : [[email protected]](mailto:[email protected])
- **If you leave the source, you can use it freely.**