Denoising in ML Pipelines
Positioned securely between data collection and feature extraction, denoising prevents errors from compounding in later stages. It ensures models learn genuine patterns instead of sensor errors or environmental interference.
The Core Equation
- Y: The observed noisy signal.
- X: The desired clean signal.
- N: Unwanted noise.
Mitigating Overfitting
Deep learning models can memorize random noise. Denoising forces focus on underlying signals.
Enhancing PCA
Smoother data allows Principal Component Analysis to identify variance directions more accurately.
Interactive Signal Denoising
Mean Removal & Detrending
Feature extraction aims to summarize distinctive patterns. If raw data isn't cleaned, models focus on meaningless shifts. This process occurs strictly before extraction begins.
Method 1: Mean Removal (Zero-Centering)
Shifts data so its average becomes zero, eliminating the "DC offset" or constant bias. Mandatory for algorithms like PCA.
Method 2: Detrending
Removes long-term, slow-moving changes (linear or non-linear slopes) that are not part of the signal's core info.
Signal Centering Simulation
Mastering Data Imputation
Replaces missing entries with substituted values. Removing rows (Listwise Deletion) leads to significant data loss and systematic bias if missingness isn't random.
MCAR (Missing Completely at Random)
Probability of missingness is unrelated to any data points.
MAR (Missing at Random)
Missingness can be explained by other observed variables.
MNAR (Missing Not at Random)
Missingness depends on the unobserved value itself (e.g. high income).
Prevent Data Leakage: Never impute the target variable (label), as this artificially inflates model performance.
Univariate Imputation Strategies
| ID | Age (Years) | Salary ($k) | Time-Series Sensor |
|---|
Data Normalization
Equalizes feature influence. Prevents features with larger ranges (e.g., 0-1,000,000) from dominating distance metrics in models like k-NN or SVM.
Accelerating Gradient Descent
Scaled data creates spherical cost function contours, allowing optimization algorithms to converge significantly faster.
Min-Max Scaling
Scales linearly into a bounded interval [0, 1]. Preserves distribution shape but sensitive to outliers.
Standardization (Z-Score)
Transforms data to Mean = 0, StdDev = 1. Ideal for Gaussian distributions and PCA.