Category: NLP

NLP

Index:

  1. Mtech
  2. Deeplearning

26-July

Introduction & Real-World Relevance

Everyday NLP Applications:

Everyday tools relying on NLP:

Ranging from messaging apps (WhatsApp) to autocomplete in Gmail/search engines , and modern Generative AI tools (ChatGPT, Claude, Gemini, Copilot, Cursor, etc.).

Evolution of AI/NLP:

The rapid advance in NLP/AI is driven by three main factors:

  1. Massive availability of training data via the internet and social media.
  2. Massive increase in computing power (e.g., NVIDIA GPUs).
  3. Algorithmic breakthroughs.

Reference :

  1. Speech and Language Processing by Daniel Jurafsky & James H. Martin (Stanford University)

2-Aug

index

  1. History
  2. Fundamental Concepts
  3. Why NLP is Hard?

Real-World Relevance & Modern AI Landscape

  • Ubiquity of NLP: Everyday interactions rely heavily on NLP algorithms, including messaging auto-complete (WhatsApp), Gmail Smart Compose, and search engine suggestions.
  • Modern Generative AI Tools: Discussion of mainstream AI products and developer assistants widely used in corporate and startup environments (e.g., ChatGPT, Claude, Gemini, Copilot, Cursor, Windsurf, Codex).
  • Multi-modality: The shift from text-only models to multimodal architectures capable of seamlessly handling text, image, and audio inputs/outputs.

Historical Evolution of Natural Language Processing

the 80+ year historical trajectory of NLP research:

  1. 1950s–1960s (Rule-Based Era): Early work by Alan Turing (Turing Test), rule-based systems, and formal syntactic grammars.
  2. 1970s–1980s (Statistical Approaches): Shift toward statistical models and probabilistic methods.
  3. Early 2000s (Corpus & Neural Foundations): Growth of large text corpora via the internet; introduction of the first Neural Language Model in 2003.
  4. 2013 (Distributional Word Embeddings Breakthrough): Release of Google’s landmark paper on Word2Vec (Distributional Word Representations), introducing Skip-Gram and Continuous Bag-of-Words (CBOW) models, alongside Stanford’s GloVe (Global Vectors).
  5. 2017 (Transformer Architectural Shift): Publication of Google’s seminal paper “Attention Is All You Need”, introducing the Transformer Architecture and shifting the paradigm away from Recurrent Neural Networks (RNNs/LSTMs).
  6. 2022–Present (LLM & Agentic AI Era): Emergence of Large Language Models (LLMs), Small Language Models (SLMs), Agentic systems with memory mechanisms, and Multimodal models.

Foundations of Vector Semantics & Embeddings

  • The Core Premise: Computer hardware and algorithms operate strictly on numbers (0s and 1s) and cannot directly process categorical text or raw words.
  • Word Embeddings / Vector Semantics: The technique of converting words or categorical tokens into dense, continuous numerical vector space representations while preserving semantic meaning based on surrounding context.

Fundamental NLP Concepts & Mechanics

  1. Numerical Representation / Word Embeddings: Computers operate strictly on numbers (0s and 1s). Word embeddings / vector semantics convert categorical text into numerical vectors so machines can process language.
  2. Language Modeling: Predicting the probability of the next word given a preceding context. Real-world examples include Gmail smart compose and WhatsApp autocomplete predictions.
  3. NLP Pipeline & Tasks: Standard pipeline processing steps, including text chunking, Parts-of-Speech (POS) tagging, entity recognition, and sentence boundary detection.

Why NLP is Hard: Linguistic Ambiguity

human language is inherently complex due to ambiguity at multiple levels:

  1. Lexical Ambiguity (Homonymy & Polysemy): Words having multiple meanings based on context (e.g., “bank” as a financial institution vs. a river bank).
  2. Grammatical / POS Tagging Ambiguity: Words changing their part of speech depending on usage (e.g., “book” as a verb in “I will book tickets” vs. a noun in “I have a book”).
  3. Structural / Syntactic Ambiguity: Multiple structural interpretations of a single sentence (e.g., “I saw a man with a telescope on a hill”—who has the telescope, and who is on the hill?).
  4. Co-reference Resolution: Resolving pronouns/articles across multi-turn conversations or sentences (e.g., tracking what “there” or “it” refers to in a chatbot conversation).
  5. Sentence Truncation Ambiguity: Rules like splitting by periods (.) fail when encountering abbreviations (e.g., titles like “Dr. L. Naveen Kumar”).
Pages: Page 1, Page 2