Keys, queries, and values are all vectors from the LLMs. RoPE [66] will involve the rotation on the query and essential representations at an angle proportional to their absolute positions of the tokens from the input sequence. In textual unimodal LLMs, textual content is the exceptional