Week 5: Introduction to Deep Learning in Python

An introduction to deep learning in Python by Dr. Yi-Xin Liu at Fudan University (lyx@fudan.edu.cn).

This is a part of the course: Road to Scientific Research: Powerful Computer Applications (XDSY118019.01).

Lecture date: 2025.10.16

Machine Learning

• Traditional machine learning: decision trees, random forests, support vector machines, gradient boosting machines, etc.
• Deep learning: Deep learning: neural networks with multiple layers (e.g., CNN, RNN, Transformer, etc.)

Introduction to Deep Learning

• Deep Learning In 5 Minutes | What Is Deep Learning? | Deep Learning Explained Simply | Simplilearn (length: 5:51)

Fundamentals of Deep Learning

• 3Blue1Brown - But what is a neural network? | Chapter 1, Deep learning (length: 19:13)

Fundamentals of Deep Learning

• 3Blue1Brown - Gradient descent, how neural networks learn | Chapter 2, Deep learning (length: 21:00)

Fundamentals of Deep Learning

• 3Blue1Brown - What is backpropagation really doing? | Chapter 3, Deep learning (length: 13:54)

Fundamentals of Deep Learning

• 3Blue1Brown - Backpropagation calculus | Chapter 4, Deep learning (length: 10:17)

Deep Learning Experiments

Do the following experiments in A neural network playgound (http://playground.tensorflow.org/):

• Choose Gaussian data, use linear activation function to train.
• Choose Circle data, use linear activation function to train. Does it succeed? How to obtain a successful training?
• (Optional) Choose Spiral data, find a successful classification neural network model.

Neural Network Architectures

• MLP or ANN (Multi-Layer Perceptron)
• CNN (Convolutional Neural Network)
• RNN (Recurrent Neural Network)
• Transformer

Deep Learning with Pytorch

Why PyTorch?

PyTorch is currently the most popular deep learning framework not only in Python but all programming languages.

Other options

• TensorFlow
• JAX
• Keras
• Lux.jl (Julia)

Watch the video for the current state of machine learning frameworks here.

Introduction to PyTorch

• Introduction to PyTorch (length: 23:22)

Additional Resources for PyTorch

• PyTorch Beginner Series
• Youtube: Learn PyTorch for deep learning in a day. Literally.
• Online Course: Learn PyTorch for Deep Learning: Zero to Mastery

A Simple Walkthrough

Go to 03_python_deep_learning.ipynb.

More Resources for Machine Learning

Why Biological Neurons Are Deep Neural Networks (Youtube Video)

General Machine Learning

• scikit-learn: Machine learning in Python
• Boosting: XGBoost, LightGBM, CatBoost
• Kaggle: Largest AI & ML community, competitions, datasets, notebooks, etc.
• Hugging Face: AI community for ML models and datasets

LLM

• Transformer
• Ollama: open-source machine learning models
• Popular models: GPT, Gemini, Claude, Deepseek, Qwen