Welcome to the captivating world of neural networks and deep learning! In this chapter, we'll embark on a journey to understand how these remarkable techniques are transforming the field of artificial intelligence. Prepare to unravel the mysteries behind machines that can learn and solve complex problems, much like the incredible human brain!

1. Introduction to Neural Networks

1.1 What are Neural Networks?

Imagine your brain as a vast, interconnected network of tiny cells called neurons. These neurons work together harmoniously to process information, make decisions, and control your every action. In the realm of AI, we have artificial neural networks that strive to emulate the functioning of the human brain.

An artificial neural network is essentially a computer program designed to recognize patterns and learn from examples, similar to how you learn from your own experiences. It comprises layers of interconnected nodes, analogous to the neurons in your brain. Each node has an associated weight that determines its influence on the final output.

Let's take a look at a simple example in Python to create a basic neural network layer using the powerful Keras library:

from keras.layers import Dense

# Create a dense layer with 10 nodes and an input shape of 5
layer = Dense(10, input_shape=(5,))

In this code snippet, we create a dense layer (fully connected layer) with 10 nodes, and it expects an input shape of 5. Dense layers are the most commonly used type of layers in neural networks.

1.2 The Building Blocks: Neurons and Layers

Just as your brain consists of billions of interconnected neurons, artificial neural networks are composed of nodes organized into layers. Each layer plays a specific role in processing the input data and passing it to the next layer. The most common types of layers are:

• Input Layer: This layer receives the initial input data and passes it to the subsequent layers for processing.
• Hidden Layers: These are the intermediate layers between the input and output layers. They perform complex computations and feature extraction on the input data.
• Output Layer: The final layer produces the desired output based on the processed data from the previous layers.

A fully-connected, also known as “Dense” in code, also known as Multi-Layer Perceptron is show below

A fully-connected Neural Network illustration. Credit: **Designing Your Neural Networks** In the image above, each node (neuron), is weighting the inputs and then summing them adding a bias term. Then the resultant scalar (a number) is undergoing a nonlinear transformation through an activation function. The below illustration is how a neuron of 3 inputs is weighting the inputs and summing them together with a bias term (constant, number, scalar) and the result is fed into an activation function $f(\cdot)$,

An illustration of a Neuron/Node in a fully connected network.

An illustration of a Neuron/Node in a fully connected network.

Here's an example of a simple neural network architecture (fully-connected network) with one hidden layer:

from keras.models import Sequential
from keras.layers import Dense

# Create a sequential model
model = Sequential()

# Add an input layer and a hidden layer with 10 nodes
model.add(Dense(10, input_shape=(5,)))

# Add an output layer with 1 node
model.add(Dense(1))

# Generate and print a summary of the model
model.summary()

Model: "sequential_1"
┌─────────────────────────────────┬────────────────────────┬───────────────┐
│ Layer (type)                    │ Output Shape           │       Param # │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_3 (Dense)                 │ (None, 10)             │            60 │
├─────────────────────────────────┼────────────────────────┼───────────────┤
│ dense_4 (Dense)                 │ (None, 1)              │            11 │
└─────────────────────────────────┴────────────────────────┴───────────────┘
 Total params: 71 (284.00 B)
 Trainable params: 71 (284.00 B)
 Non-trainable params: 0 (0.00 B)