A Deep Dive into Raw Models

Discovering the LLMs Behind the Scenes

I've been recently diving into the world of raw Large Language Models (LLMs) like Llama and Mistral, and it's been quite a journey! It truly opens your eyes to the complexities behind user-friendly APIs such as ChatGPT. There's an interesting analogy I came across: comparing models to engines and APIs to cars. This metaphor aptly highlights the intricate mechanisms at play behind the scenes!

Working directly with these raw models has been enlightening. They essentially predict the next token (word) in a sequence, and boy, can they wander off track! Take, for instance, my experience when I asked, “Who are you?”. The model spun a tale about a WWII army lieutenant at Omaha Beach – a fascinating, yet unexpected narrative direction, to say the least. This is a prime example of the model choosing what it thinks is the most suitable continuation.

This leads to a crucial aspect: the configuration of the agent prior to making a request. The way you set up the model significantly influences whether the response will hit the mark or miss it entirely. It's a delicate balance of guiding the model to produce relevant and coherent outputs.

Another key factor is the prompting template. Each model has its unique way of interpreting system instructions, user requests, and formulating responses.