This is an implementation of the Google BERT model [paper] in Pytorch. To better understand the BERT model details, I decided to write my own codes [github], and I was strongly inspired by HuggingF...

Known variance \( \sigma^2 \) and unknown mean \( \mu \)1 Consider a set of \( N \) observations \( \mathbf{X} = {x_1, \cdots, x_n} \) from a Gaussian, where the variance \(\sigma^2 \) is known, an...

In the multivariate Gaussian distribution, the term in the exponent has the form \begin{align} (\mathbf{x} - \mathbf{\mu})^{T}\Sigma(\mathbf{x} - \mathbf{\mu}) \end{align} Note that the precision m...