That "Least Validation Error" point is where it is decided when to end the training.
As you mentioned, the training ends at the point where the Validation error is lowest. This graph represents that moment.
If we do not stop training at this point, the model will begin to memorize and the Validation error will increase, even though the Training error continues to decrease.