Posts

Visualising AI Embeddings in APEX

Image
" To deal with hyper-planes in a 14-dimensional space, visualize a 3-D space and say 'fourteen' to yourself very loudly. Everyone does it."     - Geoffrey Hinton , 2018 Turing Award winner. Within the wonderful world of  Generative AI , one concept that is all the rage is RAG, or  Retrieval Augmented Generation , which is an  AI framework that combines the strengths of traditional information retrieval systems (such as databases) with the capabilities of generative large language models (LLM s) . RAG's goal is to improve the accuracy, relevance, and timeliness of information  generation  - such as  documents, text and images -  by optimizing LLM output.  When creating a RAG system, it’s essential to store information in a format that a LLM can retrieve. This is where  data is converted into embeddings through a process that uses pre-trained m