PCA,LCA,T-SNE(Sophia Sarah 2023UCA1863).pptx

principal component analysis

1. Principal Component Analysis (PCA) By Sophia Sarah 2023UCA1863

2. Introduction to PCA Principal Component Analysis (PCA) is a statistical technique used for dimensionality reduction. It transforms correlated variables into a set of uncorrelated variables called principal components. PCA helps in data visualization and reducing computational complexity. Developed by Karl Pearson in 1901, PCA is widely used in data science and machine learning.

3. Why PCA? Reduces the number of dimensions while preserving important information. Helps in removing noise and redundancy. Useful for feature extraction in machine learning and data analysis. Commonly used in image processing, genomics, finance, and text analysis. Reduces overfitting in machine learning models.

4. Mathematical Foundation PCA is based on linear algebra concepts like eigenvectors and eigenvalues. Steps involve computing the covariance matrix and performing eigenvalue decomposition. Principal components are ordered based on variance explained. The first principal component captures the highest variance, followed by the second, and so on. Formula: X' = X * W , where W is the eigenvector matrix.

5. Step-by-Step PCA Process Standardize the dataset (mean = 0, variance = 1) to ensure equal importance of features. Compute the covariance matrix to measure feature relationships. Compute eigenvalues and eigenvectors to determine principal components. Select top k principal components based on variance explained. Transform original data into a new subspace using principal components. Reconstruct the data if necessary by reversing the PCA transformation.

6. Example Dataset & Implementation

7. PCA vs Other Dimensionality Reduction Techniques Method Linear/Non-linear Use Case Pros Cons PCA Linear Data compression Fast, simple Loss of interpretability LDA Linear Classification Maximizes class separability Requires labeled data ICA Non-Linear Signal separation Finds independent signals Computationally expensive t-SNE Non-Linear Data visualization Preserves local structure Hard to interpret quantitatively

8. Choosing the Number of Components Explained variance ratio helps determine the optimal number of components. Scree plot visualizes cumulative variance to select components. Rule of thumb: Select components that explain at least 95% variance. Use pca.explained_variance_ratio _ in Python to determine variance contribution.

9. Advantages & Disadvantages of PCA Advantages Reduces dimensionality and computation cost. Enhances visualization of high-dimensional data. Removes multicollinearity, improving model performance. Works well with continuous numerical data. Disadvantages Loss of interpretability, making it hard to understand transformed features. Sensitive to scaling of data—requires standardization. Assumes linear relationships, which may not hold in real-world cases. Not suitable for categorical data.

10. Real Life applications Image Processing: Face recognition, object detection, handwriting analysis. Finance: Stock market prediction, risk management. Genomics: Gene expression analysis, bioinformatics. Medical Diagnosis: Disease classification, MRI scan analysis. Speech Recognition: Audio signal processing.

11. What is LDA( Linear Discriminant Analysis ) ? . A supervised learning technique for dimensionality reduction. Finds a linear combination of features that separates different classes. Used for classification (e.g., face recognition, speech recognition). How LDA Works Compute the Mean Vectors for each class. Compute Scatter Matrices Within-class scatter matrix SWS_WSW​ Between-class scatter matrix SBS_BSB​ Compute the Eigenvalues & Eigenvectors of SW−1SBS_W^{-1} S_BSW−1​SB​. Select Top K Eigenvectors to project data.

12. What is ICA (Independent Component Analysis)? How ICA Works A statistical technique to separate a mixed signal into independent sources. Unlike PCA, it assumes non-Gaussian, statistically independent components . Commonly used in blind source separation (e.g., separating mixed audio signals) Maximizes statistical independence of components. Uses techniques like FastICA or Infomax Algorithm . Example: Separating voice signals from multiple speakers (cocktail party problem) .

13. What is t-SNE (t-Distributed Stochastic Neighbor Embedding)? How t-SNE Works A nonlinear dimensionality reduction technique. Preserves local structures in high-dimensional data. Mainly used for visualization of complex datasets (e.g., clustering in ML). . Converts high-dimensional Euclidean distances into probability distributions . Minimizes Kullback-Leibler divergence between high and low-dimensional distributions. Works well for image, text, and genomic data visualization

14. Comparison of LCA, ICA, and t-SNE Technique Type Strengths Common Applications LCA Linear Sparse representation Image processing, deep learning ICA Nonlinear Independent components Signal processing, EEG analysis t-SNE Nonlinear Data visualization Clustering, deep learning

15. References I. T. Jolliffe, "Principal Component Analysis: A Review and Recent Developments." Python documentation for sklearn PCA . Laurens van der Maaten & Geoffrey Hinton, "Visualizing Data using t-SNE" (for comparison purposes). Pearson (1901) for PCA, Fisher (1936) for LDA, Comon (1994) for ICA, and van der Maaten & Hinton (2008) for t-SNE. McLachlan (2004) for LDA vs. PCA and Hyvärinen & Oja (2000) for ICA vs. PCA. Shlens (2014) for PCA, van der Maaten (2014) for t-SNE, and Hyvärinen (1999) for ICA.