Explaining Artificial Intelligence (AI) in 10 minutes

Brief History of Artificial Intelligence (AI) Development

• Two main approaches in AI Development
  1. Based on statistics and optimization (SO)
  2. Based on neural networks (NN)

• The Early Beginnings (1940s - 1950s)
  SO:
  Pioneers like Ronald Fisher developed statistical methods for data analysis,
  which could be used in machine learning.
  Pioneers like George Dantzig (1947) developed Linear programming and
  the simplex algorithm that laid the groundwork for solving optimization problems.
  NN:
  1943: Warren McCulloch and Walter Pitts proposed a model of aritificial neurons,
  laying the groundwork for neural networks.
  1950: Turing Test (by Alan Turing): a way to measure machine intelligence.
  
  
• 1956: The term Artificial Intelligence was coined at the Dartmouth Conference.

• 1950-1970: Golden Years
  SO:
  Bayesian methods: The use of probability theory for inference and decision-making.
  Hidden Markov Models (HMMs): Used for speech recognition and time-series analysis.
  NN:
  Frank Rosenblatt developed Perceptron, a simple neural network capable of learning.
  The perceptron is a supervised learning algorithm.
  It learns from labeled data to make predictions and adjusts its parameters to minimize errors.
  While it is simple and limited in its capabilities, it laid the foundation for more advanced
  supervised learning techniques in AI and machine learning.

• The AI Winter (1970-1980)
  In 1969, two MIT professors, Minsky and Papert published a book entitled Perceptrons:

  

  It demonstrated the limitations of neural network and it was impractical to solve problems.

  Around this time, statistical methods and optimization continued to advance:

  Expectation-Maximization (EM) algorithm

  
  Support Vector Machines (SVMs)

  
  
  
• The Revival (1980s - 1990s)
  Statistical methods became dominate
  Neural Network research was a niche field.
  backpropagation popularized (David E. Rumelhart, Geoffrey E. Hinton, and
  Ronald J. Williams in 1986)

  
  
  
• The Modern Era (2000s - present)
  In the early 2000s, large tech companies like Microsoft spent a huge amount of money on developing AI,
  solely concentrating on utilizing statistical and optimization methods to advance.
  The rise of big data and exponential increase in computational power brought together
  neural networks, statistics, and optimization:

  Deep Learning: Neural networks were revived using optimization techniques like
  stochastic gradient descent (SGD) and backpropagation.

  

  Probabilistic Programming: Frameworks like Pyro and Stan made it easier to build and
  infer probabilistic models.
  In late 2010s, the integration of optimization and statistics with neural network led to breakthroughs.
  

Machine Learning (ML) and Deep Learning (DL)

• Machine learning (ML) is a branch of artificial intelligence (AI) that enables
  computer systems to learn and improve from data without being explicitly programmed.

• Deep learning (DL) is a specialized subfield of ML that uses artificial neural networks with many layers (hence 'deep').

Paradgims of Machine Learning

Transformer