| Machine Learning Types and Project Lifecycle | - Introduction to Machine Learning: Definition, goals, and core concepts of ML.
- Types of Learning: Supervised, unsupervised, semi-supervised, and reinforcement learning.
- Machine Learning Project Lifecycle: From business problem framing to deployment, monitoring, and maintenance.
- Data Engineering Pipeline: Collection, transformation, ingestion, and storage.
- Machine Learning Pipeline: Data preprocessing, feature engineering, model selection and training, model evaluation, model deployment, and model monitoring and maintenance.
- Data File Formats: JSON, XML, CSV, Avro, Pickle, Parquet, Numpy, etc.
- Databases: Relational and NoSQL databases,
- ETL vs. ELT: Extract-Transform-Load versus Extract-Load-Transform.
- Data-related Roles: Overview of roles (Data Analyst, Data Engineer, Data Scientist, and ML Engineer).
In-class Labs:- Lab 1: Data integration and analysis
- Lab 2: Extract, Transform, and Load (ETL)
|
| Crash Course in Programming for Machine Learning (Python) | - Getting Started with Python: Setting up Python, running scripts, and basic syntax.
- Supported Programming Paradigms: Procedural, object-oriented, and functional programming.
- Built-in Types: Integer (
int), Boolean (bool), and Floating point (float). - Looping and Branching:
for loops, while loops, if/elif/else statements. - Built-in Collections:
list, tuple, set, dict, bytes, and bytearray. - Functions: Defining, calling, arguments, return values, and scope.
- Concrete Classes: Class definitions, instantiation, attributes, and methods.
- File I/O:Reading from and writing to files.
- Necessary Packages: Overview of NumPy, Pandas, SciPy, and Matplotlib.
- Other Selected Topics: Additional Python features and concepts for advanced use.
In-class Labs:- Lab 3: Python for data scientists
|
| Review of Probability, Calculus, and Linear Algebra | - Randomness in Machine Learning: Role of randomness in algorithms, initialization, and sampling.
- Probability and Random Variables: Fundamentals of probability theory and types of random variables.
- Entropy: Measure of uncertainty and its role in information theory and machine learning.
- Basic Operations on Matrices and Vectors: Addition, multiplication, transposition, and inversion.
- Capital Sigma (Σ) and Pi (Π) Notations: Summation and product notation for mathematical expressions.
- Local and Global Maxima and Minima: Concepts in optimization and their importance in ML algorithms.
- Min/Max and Arg Min/Max: Finding extreme values and their corresponding indices or arguments.
- Similarity Measures: Techniques such as cosine similarity, Euclidean distance, and correlation.
- Derivative, Gradient, and the Chain Rule: Core calculus concepts for optimization in ML models.
In-class Labs:- Lab 4: Introduction to statistics
- Lab 5: Web scraping using Pandas
- Lab 6: Web scraping using Beautiful Soup
|
| Supervised Learning (Regression) | - Regression Techniques: Linear and polynomial regression for predictive modeling.
- Overfitting and Underfitting: Concepts, detection, and mitigation strategies.
- Regularization Methods: Ridge, Lasso, and Elastic Net.
- Gradient Descent Variants: Batch, mini-batch, and stochastic approaches for optimization.
In-class Labs:- Lab 7: Detailed implementation of a single-feature linear regression
- Lab 8: Polynomial regression
- Lab 9: Introduction to Python classes
|
| Supervised Learning (Classification) | - Classification Models: Perceptron, Adaline, Logistic Regression, Naïve Bayes.
- Support Vector Machines (SVM): Hard-margin, soft-margin, and kernel-based approaches.
- Tree-based Methods: Decision Trees, Random Forests.
- Instance-based Learning: K-nearest Neighbors (KNN).
- Ensemble Learning: Bagging, Boosting, AdaBoost, hard vs. soft majority voting.
- Multiclass Classification: Strategies for extending binary classifiers.
- Evaluation Metrics: Confusion matrix, precision, recall, F1-score, cost-sensitive metrics, ROC AUC.
In-class Labs:- Lab 10: Perceptron, Adaboost, Logistic Regression
- Lab 11: Boosting algorithms (XGBoost, LightGBM, and CatBoost)
|
| Feature Engineering and Performance Assessment | - Anatomy of a Learning Algorithm – Key components and workflow.
- Data Splits – Training, validation, and test sets.
- Model Performance Concepts – Approximation, generalization, and the bias-variance tradeoff.
- Data Preprocessing – Handling missing data, scaling, discretization, and encoding categorical features.
- Data Leakage – Understanding and preventing information leakage between datasets.
- Feature Importance & Selection – Techniques including:
- Dropping columns with many missing values
- Dropping features with low variance
- Backward and forward feature selection
- Recursive feature elimination
- Filter-based methods (Pearson’s r, Chi-squared, Spearman’s Rho, mutual information, ANOVA F-score)
- Embedded methods (LASSO, positive LASSO regression, XGBoost)
In-class Labs:- Lab 12: Hyperparameter tuning
- Lab 13: Time series prediction (stock market forecasting)
- Lab 14: Feature selection
|
| Unsupervised Learning | - Unsupervised Learning Use Cases: Applications in pattern discovery, anomaly detection, and data exploration.
- Clustering Methods –
- Prototype-based: K-means
- Hierarchical: Agglomerative and divisive clustering
- Density-based: DBSCAN, HDBSCAN
- Dimensionality Reduction:
- Principal Component Analysis (PCA)
- Kernel PCA (KPCA)
- Overview of the Linear Discriminant Analysis (LDA)
- Market Basket Analysis & Association Rules: Discovering relationships between items in transactional datasets.
In-class Labs:- Lab 15: Clustering and principal component analysis
|
| Introduction to Shallow and Deep Neural Networks | - Shallow Neural Networks: Basic architecture with input, hidden, and output layers.
- Deep Neural Networks (DNNs): Multi-layer architectures.
- Key Concepts: Neurons, weights, biases, activation functions.
- Forward Propagation: Flow of data through the network for predictions.
- Backpropagation: Gradient-based optimization for training neural networks.
- Common Activation Functions: Sigmoid, tanh, ReLU, and softmax.
- Use Cases: Regression, image recognition, natural language processing, with focus on classification.
In-class Labs:- Lab 16: Tensors using PyTorch
- Lab 17: Shallow neural networks
- Lab 18: Feedforward neural networks
|
| Introduction to Convolutional Neural Networks | - Overview of CNNs: Processing grid-like data, including images.
- Convolutional Layers: Filters/kernels feature extraction.
- Pooling Layers: Max pooling and average pooling downsampling.
- Fully Connected Layers: Combining extracted features for classification or regression tasks.
- Activation Functions
- Use cases: Image recognition, object detection, medical imaging, and malware analysis.
In-class Labs:- Lab 19: Basic convolutional neural network
|
| Introduction to Transfer Learning | - Overview of Transfer Learning: Leveraging pre-trained models to solve related tasks.
- Benefits of Transfer Learning: Reduced training time, improved performance with limited data, and lower computational cost.
- Common Approaches:
- Feature extraction using pre-trained models
- Fine-tuning model's partial subset of the weights for the target task
- Popular Pre-trained Models: Examples include VGG, ResNet, EfficientNet, BERT, and GPT.
- Applications: Computer vision, natural language processing, and medical diagnostics.
In-class Labs:- Lab 20: Pretrained models and transfer learning
|
