Natural Language Processing (NLP) has transformed the way machines interact with human language. From search engines and chatbots to recommendation systems and virtual assistants, NLP powers many of the intelligent applications we use every day.<\/p>\n\n\n\n
However, one fundamental challenge exists: computers do not understand words the way humans do.<\/p>\n\n\n\n
When humans read words such as king<\/em>, queen<\/em>, doctor<\/em>, or hospital<\/em>, we automatically understand their meanings and relationships. Computers, on the other hand, see text as a sequence of characters without any inherent meaning.<\/p>\n\n\n\n To bridge this gap, researchers developed techniques that convert words into numerical representations while preserving their semantic meaning. These representations are known as word embeddings<\/strong>.<\/p>\n\n\n\n In this article, we will explore Vector Spaces, Word2Vec, and GloVe\u2014three foundational concepts that helped machines move beyond simple text processing toward understanding language more intelligently.<\/p>\n\n\n\n Computers are designed to process numbers, not words.<\/p>\n\n\n\n Consider the following sentence:<\/p>\n\n\n\n A human immediately understands its meaning, but a computer sees only a collection of characters:<\/p>\n\n\n\n The machine does not understand:<\/p>\n\n\n\n Traditional machine learning algorithms require numerical input.<\/p>\n\n\n\n This creates an important question:<\/p>\n\n\n\n How can we convert words into numbers while preserving their meaning?<\/p>\n<\/blockquote>\n\n\n\n Early NLP techniques attempted to solve this problem using methods such as:<\/p>\n\n\n\n Example vocabulary:<\/p>\n\n\n\n Representations:<\/p>\n\n\n\n While simple, this approach has major limitations:<\/p>\n\n\n\n For example:<\/p>\n\n\n\n To a computer, “cat” and “dog” appear completely unrelated, even though both are animals.<\/p>\n\n\n\n Word embeddings are dense numerical vectors that capture semantic meaning.<\/p>\n\n\n\n Instead of representing words as isolated entities, embeddings place similar words close together in a mathematical space.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n Notice how related words receive similar vector representations.<\/p>\n\n\n\n This allows machines to learn relationships between words rather than simply memorizing them.<\/p>\n\n\n\n A vector space is a mathematical environment where words are represented as points.<\/p>\n\n\n\n Instead of viewing words as text, we represent them as vectors.<\/p>\n\n\n\n Imagine a simple 2-dimensional space:<\/p>\n\n\n\n In reality, modern embeddings use hundreds of dimensions.<\/p>\n\n\n\n Common dimensions include:<\/p>\n\n\n\n Each dimension captures different characteristics of language.<\/p>\n\n\n\n The most important idea is:<\/p>\n\n\n\n Words with similar meanings appear closer together in vector space.<\/p>\n<\/blockquote>\n\n\n\n For example:<\/p>\n\n\n\n The relationships between these words can be learned mathematically.<\/p>\n\n\n\n Word2Vec is one of the most influential word embedding techniques developed by Google in 2013.<\/p>\n\n\n\n Instead of manually defining relationships, Word2Vec learns word representations automatically by analyzing large amounts of text.<\/p>\n\n\n\n The core idea is simple:<\/p>\n\n\n\n Words appearing in similar contexts tend to have similar meanings.<\/p>\n<\/blockquote>\n\n\n\n For example:<\/p>\n\n\n\n Because “cat” and “dog” appear in similar contexts, Word2Vec learns that they are semantically related. sentences = [ model = Word2Vec( print(model.wv[“machine”]) CBOW predicts a target word using surrounding words.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n Context words:<\/p>\n\n\n\n Target word:<\/p>\n\n\n\n Skip-Gram works opposite to CBOW.<\/p>\n\n\n\n Instead of predicting the center word, it predicts surrounding words.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n Target Word:<\/p>\n\n\n\n Predicted Context:<\/p>\n\n\n\n Example:<\/p>\n\n\n\n This famous example demonstrates how Word2Vec learns relationships mathematically.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n Both meanings receive the same vector.<\/p>\n\n\n\n These limitations motivated the development of improved embedding techniques.<\/p>\n\n\n\n GloVe (Global Vectors for Word Representation) is a word embedding model developed by Stanford University.<\/p>\n\n\n\n Unlike Word2Vec, which primarily learns from local context, GloVe combines both:<\/p>\n\n\n\n This helps generate richer word representations.<\/p>\n\n\n\n GloVe analyzes how frequently words appear together across an entire corpus.<\/p>\n\n\n\n Example:<\/p>\n\n\n\n By examining these relationships globally, GloVe learns meaningful vector representations.<\/p>\n\n\n\n Instead of focusing only on neighboring words, it studies broader language patterns. glove = api.load(“glove-wiki-gigaword-100”)<\/p>\n\n\n\n print(glove[“king”][:10])<\/p>\n\n\n\n <\/p>\n\n\n\n [ The development of Word2Vec and GloVe marked a significant milestone in Natural Language Processing. By transforming words into meaningful vector representations, these techniques enabled machines to move beyond simple keyword matching and begin understanding relationships between words.<\/p>\n\n\n\n Vector spaces provide a powerful framework where semantic meaning can be represented mathematically, allowing algorithms to identify similarities, analogies, and contextual relationships. Although newer transformer-based models have emerged, Word2Vec and GloVe remain fundamental concepts for understanding how modern NLP evolved.<\/p>\n\n\n\n As we continue our NLP journey, the next step is learning how these embeddings are compared and utilized in real-world systems. This leads naturally to concepts such as Cosine Similarity, Semantic Similarity, and Vector Space Semantics, which form the foundation of modern semantic search and retrieval systems.<\/p>\n","protected":false},"excerpt":{"rendered":" Natural Language Processing (NLP) has transformed the way machines interact with human language. From search engines and chatbots to recommendation systems and virtual assistants, NLP powers many of the intelligent applications we use every day. However, one fundamental challenge exists: computers do not understand words the way humans do. When humans read words such as […]<\/p>\n","protected":false},"author":72,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3515","post","type-post","status-publish","format-standard","hentry","category-support"],"_links":{"self":[{"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/posts\/3515","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/users\/72"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/comments?post=3515"}],"version-history":[{"count":1,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/posts\/3515\/revisions"}],"predecessor-version":[{"id":3517,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/posts\/3515\/revisions\/3517"}],"wp:attachment":[{"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/media?parent=3515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/categories?post=3515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mhtechin.com\/support\/wp-json\/wp\/v2\/tags?post=3515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n\n\n\nWhy Computers Struggle with Text<\/h4>\n\n\n\n
I love machine learning.\n<\/code><\/pre>\n\n\n\n['I', 'love', 'machine', 'learning']\n<\/code><\/pre>\n\n\n\n\n
\n
\n\n\n\nFrom Words to Numbers: The Need for Word Embeddings<\/h4>\n\n\n\n
One-Hot Encoding<\/h5>\n\n\n\n
[\"cat\", \"dog\", \"car\"]\n<\/code><\/pre>\n\n\n\ncat = [1, 0, 0]\ndog = [0, 1, 0]\ncar = [0, 0, 1]\n<\/code><\/pre>\n\n\n\n\n
cat = [1,0,0]\ndog = [0,1,0]\n<\/code><\/pre>\n\n\n\n
\n\n\n\nWhat Are Word Embeddings?<\/h4>\n\n\n\n
King \u2192 [0.72, 0.14, 0.89]\nQueen \u2192 [0.70, 0.16, 0.87]\nMan \u2192 [0.60, 0.22, 0.75]\nWoman \u2192 [0.58, 0.24, 0.73]\n<\/code><\/pre>\n\n\n\n
\n\n\n\nUnderstanding Vector Spaces<\/h4>\n\n\n\n
Queen\n *\n |\n |\nKing *--------*\n |\n |\n *\n Woman\n<\/code><\/pre>\n\n\n\n\n
\n
Paris \u2192 France\nBerlin \u2192 Germany\n\nKing \u2192 Queen\nMan \u2192 Woman\n<\/code><\/pre>\n\n\n\n
\n\n\n\nWord2Vec<\/h4>\n\n\n\n
What is Word2Vec?<\/h5>\n\n\n\n
\n
The cat sits on the mat.\n\nThe dog sits on the mat.\n<\/code><\/pre>\n\n\n\n
from gensim.models import Word2Vec<\/p>\n\n\n\n
[“i”, “love”, “machine”, “learning”],
[“machine”, “learning”, “is”, “fun”],
[“i”, “love”, “python”]
]<\/p>\n\n\n\n
sentences,
vector_size=100,
window=5,
min_count=1,
workers=4
)<\/p>\n\n\n\n
<\/p>\n\n\n\n
\n\n\n\nCBOW (Continuous Bag of Words)<\/h5>\n\n\n\n
The ____ sits on the mat\n<\/code><\/pre>\n\n\n\nThe, sits, on, the, mat\n<\/code><\/pre>\n\n\n\ncat\n<\/code><\/pre>\n\n\n\nHow CBOW Works<\/h5>\n\n\n\n
\n
Advantages of CBOW<\/h5>\n\n\n\n
\n
Limitations of CBOW<\/h5>\n\n\n\n
\n
\n\n\n\nSkip-Gram<\/h5>\n\n\n\n
cat\n<\/code><\/pre>\n\n\n\nThe\nsits\non\nmat\n<\/code><\/pre>\n\n\n\nHow Skip-Gram Works<\/h5>\n\n\n\n
\n
Advantages of Skip-Gram<\/h5>\n\n\n\n
\n
Limitations of Skip-Gram<\/h5>\n\n\n\n
\n
\n\n\n\nAdvantages of Word2Vec<\/h5>\n\n\n\n
\n
King - Man + Woman \u2248 Queen\n<\/code><\/pre>\n\n\n\n
\n\n\n\nLimitations of Word2Vec<\/h5>\n\n\n\n
\n
Apple (fruit)\nApple (company)\n<\/code><\/pre>\n\n\n\n\n
\n\n\n\nGloVe (Global Vectors for Word Representation)<\/h4>\n\n\n\n
What is GloVe?<\/h4>\n\n\n\n
\n
\n\n\n\nHow GloVe Works<\/h5>\n\n\n\n
ice appears frequently with:\ncold\nwinter\nsnow\nfreeze\n\nsteam appears frequently with:\nhot\nwater\nheat\nboil\n<\/code><\/pre>\n\n\n\n
import gensim.downloader as api<\/p>\n\n\n\n
(‘machines’, 0.82),
(‘computer’, 0.79),
(‘technology’, 0.75)
]<\/p>\n\n\n\n
\n\n\n\nWord2Vec vs GloVe<\/h4>\n\n\n\n
Feature<\/th> Word2Vec<\/th> GloVe<\/th><\/tr><\/thead> Learning Method<\/td> Predictive<\/td> Count-Based<\/td><\/tr> Context Usage<\/td> Local Context<\/td> Global Context<\/td><\/tr> Training Speed<\/td> Fast<\/td> Fast<\/td><\/tr> Semantic Relationships<\/td> Strong<\/td> Strong<\/td><\/tr> Statistical Information<\/td> Limited<\/td> Extensive<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n <\/h3>\n\n\n\n
\n\n\n\n
\n\n\n\nConclusion<\/h1>\n\n\n\n