Aug 11, 2021
Yep, all those terms just mean the same thing. In regular reading comprehension terms I think the correct term is "context".
I think the author meant "Hello", 'Hey', 'Hi" instead of duplicating "hey".
From my understand, these dense vectors are the same thing as contextual word embeddings we obtain from the self-attention portion of a transformer model (ie. BERT).