A collection of various gradient descent algorithms implemented in Python from scratch

[ICML 2021] The official PyTorch Implementations of Positive-Negative Momentum Optimizers.

NAG-GS: Nesterov Accelerated Gradients with Gauss-Siedel splitting

Intelligent Detection for RIS-Assisted MIMO Systems: A First-and-Second Momentum Approach

Overshoot: Taking advantage of future gradients in momentum-based stochastic optimization

In this project it is used a Machine Learning model based on a method called Extreme Learning, with the employment of L2-regularization. In particular, a comparison was carried out between: (A1) which is a variant of incremental extreme learning machine that is QRIELM and (A2) which is a standard momentum descent approach, applied to the ELM.

Python code for Gradient Descent, Momentum, and Adam optimization methods. Train neural networks efficiently.

Using Matrix Factorization/Probabilistic Matrix Factorization to solve Recommendation。矩阵分解进行推荐系统算法。

Simple Document Classification using Multi Class Logistic Regression & SVM Soft Margin from scratch

This is an implementation of different optimization algorithms such as: - Gradient Descent (stochastic - mini-batch - batch) - Momentum - NAG - Adagrad - RMS-prop - BFGS - Adam Also, most of them are implemented in vectorized form for multi-variate problems

Lightweight neural network library written in ANSI-C supporting prediction and backpropagation for Convolutional- and Fully Connected neural networks

Historical NumPy neural-network learning project (2020-2021).

Numerical Optimization for Machine Learning & Data Science

EE456 2022 mini project implementation of two-moons problem using multi-layer-perceptron with back-propagation with analyzing performance of initializing methods and momentum rule

Machine Learning, Deep Learning Implementations

A fully vectorized Deep Neural Network (DNN) implementation built from scratch using only NumPy - no deep learning frameworks involved. Covers forward/backward propagation, activation functions, modular architecture, and training with different optimizers - a hands-on deep dive into the fundamentals of deep learning.

This repository contains a python implementation of Feed Forward Neural Network with Backpropagation, along with the example scripts for training the network to classify images from mnist and fashion_mnist datasets from keras.

ML class projects @ ECE/TTU - Same content - Converted from MATLAB to Python

This repository provides implementations of numerical optimization algorithms for machine learning and deep learning. It includes clear explanations, mathematical formulas, Python code, and visualizations to help understand the behavior of each optimizer.