statduck

Calculation $$ \delta_k(x) =-\dfrac{1}{2}log|\hat{\Sigma}_k|-\dfrac{1}{2}(x-\mu_k)^T\hat{\Sigma}_k^{-1}(x-\mu_k)+log\pi_k $$ $\hat{\Sigma}_k=U_kD_kU_k^T$can make this calculation more faster. $$ ✔️ (x-\hat{\mu}k)^T\hat{\Sigma}^{-1}_k(x-\hat{\mu}_k)=[U_k^T(x-\hat{\mu}_k)]^TD_k^{-1}[U_k^T(x-\hat{\mu}_k)] \ ✔️ log|\hat{\Sigma}_k|=\Sigma_llogd{kl}$$ In normal dsitrubiton, the quadratic form means ma..

Linear method: Classification Classification problem. Classification means target variable has a categorical value. Let's assume Y has only zero or one, and g is the value. Connect Y and g. $$ Y_{ij}=I(G(x)=g_j) $$ Linear Model If $\delta_k(x)$is linear, then $D$ is also linear. $E(Y_k|X=x)=P(G|X)$ When we estimate this with $X^T\beta$, the range can be out of range 0 to 1. $\min_{\mathbf{B}} \S..

Orthogonalization is important. 1) Vector to span{vector} Let's consider two vectors $v,w$ in n dimension. $$ Project \; v \; onto \; span\{w\}=Proj_wv=\dfrac{v \cdot w}{||w||^2}w=\dfrac{v\cdot w}{||w||}\dfrac{w}{||w||} $$ $\dfrac{v \cdot w}{||w||}$$: length $$\dfrac{w}{||w||}$: direction Gram-schmidt process Every non-zero subspace of $\mathbb{R}^n$ has an orthonormal basis. $v_1=w_1$ $v_2=w_2-..

Problem Definition Cow Milk(Y) Age(X1) Weight(X2) #1 10 1 2 #2 11 3 3 #3 12 4 1 We want to find the model which well explains our target variable($y$) with $x$ variables. The model looks like this $$ Y_i =\beta_1X_{1i}+\beta_2X_{2i}+\epsilon_i $$ We can evaluate how precise our model it is with a fluctuation of our error. When we assume that our expected error is zero, the fluctuation represents..

Statistical Decision Theory Statistical Decision Theory When it comes to decide which model we use, the important thing to consider is to minimize error. Let's generalize it. $$ X \in \mathbb{R}^p, Y\in \mathbb{R}, f:\mathbb{R}^p ; to ; \mathbb{R} $$ The goal is to find the $f(X)$ that predict $Y$ well. Loss function is needed to find this $f(X)$, and this function gives penalizing errors in pre..

To estimate unknown parameters like mean and variance, we sample a data set from a target population. After we sample a data set, we should check whether this set well represents our target population. If it doesn't well represent our target, we need to adjust our estimator. Let's see how to adjust our estimator. Before we adjust it, we first need to construct estimating equation which of soluti..

Curse of dimensionality There are three problems in high dimension First, More length is needed to capture same rate of data. Second, All sample points are close to an edge of the sample. Third, We need much more sample in high dimension to capture same percentage of data in low dimension. ✏️ First Problem Let's think about a unit hypercube in p dimension, $\{x|; x_i

Two Simple Approaches to Prediction: Least Squares and Nearest Neighbors. When input vectors are given, how do we predict the output vector? There are many modeling methods available and the choice of methods depends on the nature of the problem and data Linear Models and Least Squares Linear regression has been a commonly used method to solve problems where we need to predict a continuous outpu..