diff --git a/src/content/docs/reference/preparation/ner.md b/src/content/docs/reference/preparation/ner.md index f674924..5c7e4b8 100644 --- a/src/content/docs/reference/preparation/ner.md +++ b/src/content/docs/reference/preparation/ner.md @@ -1,11 +1,36 @@ --- title: Named Entity Recognition -description: A guide in my new Starlight docs site. +description: A concise guide to Named Entity Recognition methods, applications, and challenges. --- -Guides lead a user through a specific task they want to accomplish, often with a sequence of steps. -Writing a good guide requires thinking about what your users are trying to do. +Named Entity Recognition (NER) is the process of automatically identifying and categorizing key elements in text such as names of people, organizations, locations, and more. +It is a foundational technique in natural language processing (NLP) that facilitates a range of applications including information extraction, content classification, and search optimization. + +## Traditional NER Methods + +Early NER systems relied on rule-based and statistical approaches. +Tools like [SpaCy](https://spacy.io/) implement a combination of hand-crafted rules and machine learning algorithms to efficiently recognize entities in text. +However, traditional NER models are typically effective only for a set of predefined entity types, limiting their flexibility. + +### GliNER: A Compact, Flexible Alternative + +In contrast to conventional systems, GliNER introduces a compact NER model designed to identify any type of entity. +Leveraging a bidirectional transformer encoder, GliNER facilitates parallel entity extraction—an advantage over the slow sequential token generation characteristic of many large language models (LLMs). +Comprehensive testing shows that GliNER outperforms both ChatGPT and fine-tuned LLMs in zero-shot evaluations on various NER benchmarks, addressing limitations in traditional models while maintaining resource efficiency. + +## NER with Large Language Models + +Modern large language models (LLMs) have introduced a paradigm shift in how NER tasks can be approached. +LLMs can perform NER in a zero-shot or few-shot learning setting, extracting arbitrary entities through natural language instructions. +This offers greater flexibility compared to traditional methods, although the size and cost of LLMs can be impractical in resource-limited scenarios. + +## Entity Linking and Entity Disambiguation + +Beyond merely identifying entities, many applications require associating them with specific, unique identifiers—a process known as entity linking. +This often involves resolving ambiguities where the same name might refer to multiple real-world entities, a challenge known as entity disambiguation. +High-level solutions typically integrate NER with external knowledge bases and context-aware algorithms to ensure that each recognized entity is accurately matched to its correct reference. ## Further reading -- Read [about how-to guides](https://diataxis.fr/how-to-guides/) in the Diátaxis framework +- [GliNER paper](https://arxiv.org/abs/2311.08526) +- [SpaCy](https://spacy.io/)