Word Vector Arithmetic in deep learning concept

Word Vector Arithmetic various mathematical examples in deep learning concept

1. Module 1 Introducing Deep Learning Word Vectors Arithmetic

2. Word Vector Arithmetic Remarkably, because particular movements across vector space turn out to be an efficient way for relevant word information to be stored in the vector space, these movements come to represent relative particular meanings between words. If we calculate the direction and distance between the coordinates of the words Paris and France and then trace this direction and distance from London, we should find ourselves in the neighborhood of the coordinate representing the word England.

3. Word Vector Arithmetic As a second example, we can calculate the direction and distance between the coordinates for man and woman. This movement through vector space represents gender and is symbolized by the green arrows in Refer cube square Figure. If we trace the green direction and distance from any given male-specific term (e.g., king, uncle), we should find our way to a coordinate near the term’s female-specific counterpart (queen, aunt).

4. Word Vector Arithmetic

5. Word Vector Arithmetic A by-product of being able to trace vectors of meaning (e.g., gender, capital-country relationship) from one word in vector space to another is that we can perform word-vector arithmetic. Direction = vector difference (where to move) Ex: d = v(Paris) - v(France) Distance = length of that vector (how far to move) Ex: ||d|| = sqrt(sum of squared components)

6. Word Vector Arithmetic - Example If embeddings encode the “capital-of” relation uniformly, then the same vector d should relate London and England: v(London) - v(England) ≈ d Rearranged to solve for England: v( England ) ≈ v(London) - d = v(London) - (v(Paris) - v(France)) = v(London) - v(Paris) + v(France)

7. Word Vector Arithmetic - Example Given vectors: v(France) = [2, 1] v(Paris) = [5, 5] v(London) = [6, 6] (we’ll show v(England) ends up at [3,2])

8. Word Vector Arithmetic - Example Compute the direction vector d d = v(Paris) - v(France) = [5 - 2, 5 - 1] = [3, 4] (we subtracted component-wise: 5-2 = 3, 5-1 = 4)

9. Word Vector Arithmetic - Example 2. Compute the distance (length) of d ||d|| = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5 So the distance is 5. The unit direction (normalized vector) is

10. Word Vector Arithmetic - Example 2. Compute the distance (length) of d ||d|| = sqrt(3^2 + 4^2) = sqrt(9 + 16) = sqrt(25) = 5 So the distance is 5. The unit direction (normalized vector) is u = d / ||d|| = [3/5, 4/5] = [0.6, 0.8]

11. Word Vector Arithmetic - Example 3. Trace that direction/distance from London to get England Using the algebra above: v(England) ≈ v(London) - d = [6 - 3, 6 - 4] = [3, 2] (6-3 = 3, 6-4 = 2) So the traced point is [3, 2]. If our toy v(England) equals [3,2], we land exactly on England.

12. Word Vector Arithmetic - Example 4. Verify by computing distances to candidates Distance from the traced point v' = [3,2] to v(England) = [3,2]: distance = sqrt((3-3)^2 + (2-2)^2) = sqrt(0 + 0) = 0

13. Word Vector Arithmetic - Example To another word, e.g. v(Germany) = [0, -1], distance: (3 - 0) = 3 (2 - (-1)) = 2 - (-1) = 3 squares: 3^2 = 9, 3^2 = 9 sum = 9 + 9 = 18 sqrt (18) ≈ 4.242640687...

14. THANK YOU