Editing Neural Networks (section)

== <span style="color: #FFFFFF;">Understanding</span> ==
A neural network learns by adjusting its weights to minimize a loss function. The learning process has two phases repeated iteratively:

'''Forward Pass''': Input data flows through each layer. Each neuron computes a weighted sum of its inputs, adds a bias, and applies an activation function. This produces progressively more abstract representations until the final output layer produces a prediction.

'''Backward Pass (Backpropagation)''': The error between the prediction and the true label is computed using the loss function. Using the chain rule of calculus, the gradient of this error with respect to every weight in the network is computed, flowing backward from output to input. Weights are then nudged in the direction that reduces the error.

Think of it like tuning a radio dial — you make small adjustments (gradient steps) based on how much static (loss) you hear, until the signal (prediction) becomes clear.

The power of neural networks comes from their ability to learn hierarchical representations. Early layers detect simple features (edges in an image, individual characters in text), while deeper layers combine these into increasingly complex abstractions (shapes, words, concepts). This emergent feature learning is why neural networks outperform hand-engineered feature extraction on most complex tasks.

The choice of '''activation function''' is critical. Without non-linear activations, stacking layers would be mathematically equivalent to a single linear transformation, giving no benefit. ReLU (Rectified Linear Unit) — which simply outputs max(0, x) — has become the default because it avoids the vanishing gradient problem that plagued earlier sigmoid and tanh activations.
</div>

<div style="background-color: #8B0000; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">