The original motive behind the creation of neural networks was to mimic the human brain. An artificial neural network is a collection of connected nodes, with each connection transmitting a signal to another node.
Typically, nodes are aggregated into layers, with different layers performing different transformations. Signals travel from the input layer to the output layer (maybe passing through other layers in between), while possibly traversing the layers multiple times. The connections between nodes (edges) have weight, that adjusts as the learning proceeds.
A neuron can be thought of as a singular algorithm. It takes an input layer, performs machine learning algorithms on it (activation $a$), then outputs a probability. Nodes in the middle layers and output layer are activations. A layer can have one or many nodes.
In actuality, it is hard to manually direct the relationships between nodes in each layer, so a neural network starts with edges for every possible connection. What makes the model so powerful is it automatically determines the optimal relationships between nodes. Layers build upon the relationships found in the previous layer, which allows them to take simple data (i.e. pixels in a photo) and detect entire real world objects (i.e. faces). The activations at each layer are higher level features.
a neural network for facial recognition - whitepaper