In transformer models, value vectors are one of the three types of vectors used in the [[self-attention]] mechanism. The other two types of vectors involved in self-attention are [[query]] vectors and [[key]] vectors. Value vectors in transformer models are computed through a linear transformation of the [[input embedding|input embeddings]]. These input embeddings can be the original word embeddings or embeddings that have undergone certain modifications, such as the addition of [[positional encoding|positional encodings]]. The resulting embedding vectors for each word are passed through a linear transformation. This linear transformation is often implemented using a learned weight matrix specific to value vectors. The result of the linear transformation is the value vector for each word. Value vectors capture the semantic meaning or content of each word in the context of the input sequence. [[key]] < [[Hands-on LLMs]]/[[2 LLMs and Transformers]] > [[attention score]]