Categories

text preprocessing

Chunking

Chunking is the process of splitting longer content into smaller, meaningful parts so that an AI system can store, search, retrieve and use the right information more effectively. Instead of ...

Chunking is the process of splitting longer content into smaller, meaningful parts so that an AI system can store, search, retrieve and use the right information more effectively. Instead of Read article

RAG

RAG, short for Retrieval-Augmented Generation, is an AI architecture that combines retrieval of relevant information with answer generation. In other words, the model does not respond only from what it ...

RAG, short for Retrieval-Augmented Generation, is an AI architecture that combines retrieval of relevant information with answer generation. In other words, the model does not respond only from what it Read article

Token

A token is the basic unit of text or other input that a language model processes. It is not exactly the same as a word, a sentence or a character. ...

A token is the basic unit of text or other input that a language model processes. It is not exactly the same as a word, a sentence or a character. Read article