AI history

Symbolic vs Connectionist AI

AI has two main approaches:

symbolic AI
- tries to model intelligence by using symbols, rules, and logical statements or associations
- assumes that intelligence can be broken down into explicit rules and symbols
- problems are solved by manipulating these symbols
- follows a top-down approach:
  - starts with high level tasks
  - breaks them down into smaller and smaller components
connectionist AI
- tries to model intelligence by building networks of simpler components
- inspired by how the human brain works
- uses artificial neural networks trained on data
- follows a bottom-up approach:
  - starts with smaller pieces
  - combines them to produce complex behavior

Historical context:

connectionism suffered for a long time because of speed, algorithm, and data issues
with the advent of deep learning, connectionism has become the dominant approach

the dream of intelligent machines dates back to antiquity (e.g., Talos)
philosophers and early scientists began to consider that human thought might arise from the physical brain rather than the spiritual soul

Artificial intelligence emerged as a legitimate research field due to two key developments:

the shift from fantastical stories to a serious investigation of whether mathematics can capture thought and reasoning
the realization that digital computers can execute any process expressible as an algorithm (Alan Turing)

Early ideas (1943):

Warren McCulloch and Walter Pitts introduced a mathematical abstraction of a neuron
they showed that networks of such neurons could perform logical operations, linking math, computing, and neurobiology

1956 - Dartmouth Summer Research Project on Artificial Intelligence workshop
- regarded as the birthplace of AI
- fewer than 50 participants
1957 - Frank Rosenblatt created the perceptron
- simplified mathematical model of a neuron
1958 - Mark I Perceptron
1963 - Leonard Uhr and Charles Vossler
- described a program similar to the convolutional neural networks that appeared over 30 years later
1967 - Thomas Cover and Peter Hart created the nearest neighbors model
- the first "classical" machine learning model
1969 - Marvin Minsky and Seymour Papert published "Perceptrons"
- proved perceptrons couldn't solve non-linear problems (like XOR)
1973 - James Lighthill published "Artificial Intelligence: A General Survey"
- known as "the Lighthill report"
late 1970s - First AI Winter
- "Perceptrons" and "the Lighthill report" led to a decline in interest and funding called the AI winter
1979 - Kunihiko Fukushima introduced the Neocognitron
- first convolutional neural network

early 1980s - rise of expert systems
- AI went commercial with the advent of computers designed to run Lisp
- expert systems are software designed to capture knowledge of an expert in a narrow domain ("business rule management systems")
1982 - John Hopfield
- introduced Hopfield networks, demonstrating the utility of connectionist models
1986 - Rumelhart, Hinton, and Williams introduced backpropagation
- this overcame earlier limitations and sparked the rise of deep learning
- this paper influenced neural network researchers the most

1995 - support vector machines (SVMs) by Corinna Cortes and Vladimir Vapnik
- use clever mathematics to simplify difficult classification problems
- became a cornerstone of classical machine learning
1997 - IBM's Deep Blue defeats Garry Kasparov
1998 - advent of convolutional neural networks
- Yann LeCun, Léon Bottou, Yoshua Bengio and Patrick Haffner published "Gradient-Based Learning Applied to Document Recognition"
- escaped public notice but was a watershed moment for AI and the world

2001 - Leo Breiman introduced random forests
some of the most powerful AIs in the 2010s involved autoencoders as a component of larger systems
- autoencoder
- denoising autoencoder
- stacked architectures

2012 - AlexNet wins the ImageNet challenge
- a convolutional neural network architecture (AlexNet) won the ImageNet challenge
- the new model began to accomplish tasks like:
  - reimagining images in the style of another image or painting
  - generating a text description of the contents of an image
2013 - DeepMind
- Google's DeepMind group introduced a deep reinforcement learning-based system that could successfully learn to play Atari 2600 video games as well or better than human experts
2014 - Generative Adversarial Networks (GANs)
- called by Yann LeCun the most significant breakthrough in neural networks in 20 to 30 years
- lets models "create" output that's related to but different from the data on which they were trained
- major step toward generative AI (ChatGPT, Stable Diffusion, etc.)
2016 - Google's AlphaGo defeats Lee Sedol (4 to 1)
2017 - AlphaGo Zero
- trained from scratch by playing against itself
- beat AlphaGo scoring perfectly (100 wins, no loss)
2017 - Asilomar Conference on Beneficial AI
- raised awareness of the potential hazards associated with AI
2019 - DeepMind's AlphaStar system
- outperformed 99.8 percent of human players in StarCraft II
2022 - KataGo was defeated
- KataGo was defeated by a system trained not to win but to reveal the brittleness inherent in modern AI systems
- the moves used by the system were outside the range encountered by KataGo when it was trained
- models are good at interpolating but bad at extrapolating

explosion of new models
rapid expansion of large-scale models capable of generating text, images, audio, and video
systems such as ChatGPT, DALL·E, Stable Diffusion, and Midjourney bring AI into mainstream use

Published on 29/03/2026