Outline of deep learning

Precursors

• Cybernetics
• Perceptron
• Connectionism
• Neocognitron
• Backpropagation

Milestones

• LeNet
• Long short-term memory
• Deep belief network
• AlexNet
• Sequence to sequence learning
• Generative adversarial network
• Residual neural network
• Transformer
• BERT
• Generative pre-trained transformer
• Diffusion model

Related histories

• History of artificial intelligence
• History of machine learning
• Timeline of machine learning

Feedforward and convolutional architectures

• Feedforward neural network
• Multilayer perceptron
• Convolutional neural network
• Radial basis function network
• Residual neural network
• U-Net

Recurrent and sequence architectures

• Recurrent neural network
• Long short-term memory
• Gated recurrent unit
• Sequence to sequence learning
• Recursive neural network

Representation-learning architectures

• Autoencoder
• Denoising autoencoder
• Sparse autoencoder
• Variational autoencoder
• Restricted Boltzmann machine
• Deep belief network

Attention and transformer architectures

• Attention (machine learning)
• Transformer
• BERT
• Generative pre-trained transformer
• Vision transformer

Generative and probabilistic architectures

• Autoregressive model
• Diffusion model
• Energy-based model
• Generative adversarial network
• Mixture of experts

Graph and memory architectures

• Graph neural network
• Graph convolutional network
• Siamese network
• Neural Turing machine
• Memory network^[4]
• Echo state network
• Capsule neural network

Computer vision

• Computer vision
• Facial recognition system
• Image classification
• Image segmentation
• Medical imaging
• Object detection
• Optical character recognition

Natural language processing

• Automatic summarization
• Chatbot
• Information retrieval
• Large language model
• Natural language processing
• Neural machine translation
• Question answering
• Sentiment analysis

Speech and audio

• Automatic speech recognition
• Music information retrieval
• Speaker recognition
• Speech synthesis

Science and medicine

• Bioinformatics
• Computational biology
• Drug discovery
• Medical diagnosis
• Protein structure prediction

Robotics and control

• Autonomous car
• Computer game bot
• Control theory
• Robotics

Recommendation, search, and forecasting

• Anomaly detection
• Forecasting
• Fraud detection
• Recommender system
• Search engine

Generative artificial intelligence

• Deepfake
• Generative artificial intelligence
• Large language model
• Speech synthesis
• Text-to-image model

Computer graphics and video games

• Deep Learning Anti-Aliasing (DLAA)
• Deep Learning Super Sampling (DLSS)

Supporting software platforms

• CUDA^[13]
• Metal^[14][15]
• ROCm

Open-source frameworks and libraries

Neural network software

• EDLUT
• Emergent
• Encog
• JOONE
• Neuroph
• NeuroSolutions
• OpenNN
• Peltarion Synapse
• SNNS

Platforms, tools, and deployment

• Amazon SageMaker
• Google Colab
• Hugging Face
• Kaggle
• Kubeflow
• MLflow^[16]
• ONNX
• OpenVINO
• TensorFlow Hub

Representation and metric learning

• Contrastive learning
• Embedding
• Feature learning
• Manifold learning
• Metric learning

Generative modeling

• Autoregressive model
• Diffusion model
• Generative adversarial network
• Generative model
• Variational inference

Efficient and scalable deep learning

• Knowledge distillation
• Low-rank approximation
• Mixture of experts
• Quantization
• Sparsity

Reliability, safety, and interpretability

• Adversarial machine learning
• AI alignment
• Algorithmic bias
• Catastrophic forgetting
• Differential privacy
• Explainable artificial intelligence
• Federated learning
• Hallucination (artificial intelligence)

Research laboratories and institutions

• Allen Institute for AI
• Alberta Machine Intelligence Institute
• European Laboratory for Learning and Intelligent Systems
• Google DeepMind
• Meta AI
• Mila
• Microsoft Research
• Vector Institute

Companies

• Anthropic
• Cerebras
• Cohere
• DeepSeek
• Mistral AI
• OpenAI
• Stability AI
• xAI

Books

• Deep Learning^[17] – Ian Goodfellow and Yoshua Bengio
• Neural Networks and Deep Learning^[18] – Michael Nielsen
• Perceptrons – Marvin Minsky and Seymour Papert

Journals

• IEEE Transactions on Neural Networks and Learning Systems
• Neural Networks
• Neural Computation