How is the quality of a model best assessed?

Assessing the quality of a model is most effectively done by examining its performance on unseen datasets. This is because the primary goal of any machine learning model is to generalize well to new, previously unencountered data, rather than merely fitting well to the training data.

When a model performs well on unseen datasets, it demonstrates that it has learned the underlying patterns in the training data without overfitting to that specific dataset. Overfitting occurs when a model memorizes the training data, including its noise and outliers, which can lead to poor performance on new data. Therefore, using a separate validation or test set of unseen data provides a more accurate assessment of how the model is likely to perform in real-world applications.

In contrast, evaluating a model’s performance based solely on the training dataset or its accuracy on that dataset does not provide a complete picture of its effectiveness. The complexity of the model architecture can play a role in its capacity to learn from data, but it does not directly measure the model's prediction accuracy or generalization ability. Hence, the most reliable indicator of a model's quality is how well it can handle data it has not seen during training.