What is a weighted sum of the feature crossed values referred to?

A weighted sum of the feature crossed values is referred to as an embedding. In machine learning, particularly in deep learning, embeddings are a way to represent categorical variables in a continuous vector space. This representation allows models to learn relationships between categories and their features more effectively, capturing complex interactions through dense representations.

The process of embedding involves assigning a vector to represent each category, which is often initialized randomly and then learned during training. When features are crossed, they create interactions between different categories, and the weighted sum of these crossed features can be effectively captured using embeddings. This method reduces dimensionality and helps the model learn more nuanced patterns.

In contrast, the output layer refers to the final layer in a neural network that produces the predictions. An activation function is a mathematical operation applied to a neuron’s output, determining whether it should be activated based on its input. Normalization is a preprocessing technique used to scale input data so that it fits within a specific range, often improving model performance and convergence but does not pertain to the concept of representing crossed feature values through a weighted sum.