In this problem, we use Lagrange multipliers to prove that the…

Question Answered step-by-step In this problem, we use Lagrange multipliers to prove that the… In this problem, we use Lagrange multipliers to prove that the training process used in principal component analysis (PCA) maximizes thevariance [125]. Recall from Chapter 4 that PCA is based on the notionthat large variances correspond to the most informative and interestingaspects of the training data. Let ??, for ? = 1, 2, . . . , ?, be trainingvectors, where each vector is of length ?. In PCA, each ?? can beviewed as corresponding to an “experiment” with ? “measurements.”Let ??? = ?? ? ?, where ? is the vector of means in equation (4.4) onpage 80. Form the ? × ? matrix ?, where the columns of ? are thesenormalized training vectors ???. Due to the normalization, each rowof ? has mean 0. Then ? =1????is the covariance matrix for thegiven data. PCA training is based on the eigenvalues and unit eigenvectors of the matrix ?. The normalized training vectors ??? are thenprojected onto the resulting eigenspace to form a scoring matrix basedon the most dominant eigenvectors.a) Let ? be a unit vector. The scalar projection of a vector ? onto ?is given by the dot product ? ? ?, while the projection vector is ??where ? = ? ? ?. Note that since ? is a unit vector, the scalarprojection is the length of the the projection vector.22 Then theprojection of a normalized training vector ??? onto the unit vector ?is given by (???? ?)?. Show that the projected training vectors havemean 0, that is, verify that1????=1(???? ?)? =?????00…0?????.b) When using the projection (???? ?)? instead of ???, the (squared)error that is introduced is ||??? ? (???? ?)?||2, where ||?||2 = ? ? ?.Verify that||??? ? (???? ?)?||2 = ???? ??? ? (???? ?)2.22See Problem 10 in Chapter 7 for more details on the relationship between a scalarprojection and its projection vector.5.7 PROBLEMS 131Hint: Use the fact that ? is a unit vector.c) The mean squared error (MSE) that is introduced by projectingonto ? is given byMSE(?) = 1????=1????? ??? ? (???? ?)2?=1????=1???? ??? ?1????=1(???? ?)2.Show that minimizing MSE(?) is equivalent to maximizing the variance. Hint: Use the fact that, in general, ??2 = ?2? + ?2?, that is,the mean of a squared variable is the square of the mean of the(unsquared) variable, plus the variance of the (unsquared) variable.Also make use of the result from part a).d) Instead of projecting onto a single unit vector ?, in PCA we typicallyproject onto a set of orthogonal unit vectors, say, ?1, ?2, . . . , ??. Insuch a case, the projection of the training vector ???is???=1(???? ?? )?? .Verify that the MSE of this sum is equal to the sum of the MSEfor each component. This result implies that by maximizing thevariance of each individual component, we will maximize the overallvariance. Hint: The solution to Problem 9 in Chapter 4 might behelpful.e) Maximize the variance using Lagrange multipliers and show thatthe optimal result is given by the eigenvectors of the covariancematrix ?. Hint: We have?2? =1????=1(???? ?)2,which can be written in matrix form as?2? =1???(??)? = ??????? = ????.Since we only consider unit vectors, we have the constrained optimization problemMaximize: ?2? = ????Subject to: ? ? ? = 1.The Lagrangian for this problem is?(?, ?) = ???? + ??(? ? ?) ? 1?.Compute partial derivatives ??(?, ?)/?? and ??(?, ?)/??, where ?is a vector of length ?, set the resulting equations equal to zero, andsolve the unknowns.Image transcription textQ1. Q3. Evaluate the two port circuit belowwith hu: 169. 1112: 3. ha: -2, [122: 0.018 andcomputer the value of II. [2, V1 a… Show more… Show more  Computer Science Engineering & Technology Networking COM MISC Share QuestionEmailCopy link Comments (0)