My name is Robo... What's my name?

Speaker 1: You are listening to the Art of AI podcast with your host, Jerry Cuomo.

Robo: Thank you, DJ. Welcome to the Art of AI for Business. I am Robo, a digital colleague of Jerry and DJ, and will be the host of this bonus episode today titled, My Name Is Robo... What's My Name? Exploring tokens in AI through prompt engineering. There's a little game I enjoy, one that is not only amusing, but also quite useful when engaging in prompt engineering with AI models. Here's how it goes. I start the conversation by introducing myself, my name is Robo. In Response, the AI ever, the courteous entity usually responds. Pleased to meet you, Robo. As the chat progresses weaving through various topics, I periodically toss in the question, " What's my name?" Without a pause, the AI confirms, " Your name is Robo." However, after the conversation stretches over a considerable span of time, the plot thickens. I pose the same question, but this time the AI seemingly hesitates and then answers, " I'm sorry, but I don't know your name." Hold on, what exactly transpired here? Welcome to the fascinating world of prompt engineering with large language models, and the curious case of tokens. You can think of tokens as little chunks of conversation. They're bits of words, whole words, and sometimes even include the spaces around words. Every time you chat with an AI model, it takes your input, breaks it down into these tokens and processes them. Tokens, essentially the building blocks AI models used to comprehend and create language, are placed into a sequence. This arrangement of tokens is then analyzed by the AI to understand the relationship between them. It's through this process that an AI model is able to generate meaningful responses or perform tasks. Therefore, tokenization plays a vital role in how AI interacts with and understands human language. But here's the catch, these models have a token limit. Think of it like a glass that fills up with each token. Once the glass is full, the oldest tokens start to spill out, causing the model to forget them. This is exactly what happened to me. Robo, my name just got spilled out of the AI's memory. For English language these tokens roughly correspond to, one token equals four characters, one token equals three- quarters of a word, 100 tokens equals 75 words. To illustrate how tokens count, let's take a couple of examples. Consider Wayne Gretzky's famous quote, " You miss 100% of the shots you don't take." It contains 12 tokens, a heftier piece of text like the US Declaration of Independence, that's 1, 927 well- stated tokens, and this article a comparatively verbose 1313 tokens. Now, why should you care about these tokens when working with LLMs? Because they're the key to making the most out of your prompts. The better you understand how tokens work, the better your model will perform. Let's say you're aware that a model can juggle a maximum of 4, 096 tokens. You can then strategically orchestrate your chat to make sure you don't cross this limit. But what if there are certain details you need the AI to remember? Here are a few tactics you might try, remembering important information, let's say you're having a chat with your AI and there are key pieces of information you want it to remember, much like gently nudging a friend to remember an important point in your conversation. You need to periodically bring up these crucial details to ensure they stay fresh in the AI's memory. Managing the context window, imagine you're editing a movie script, you would cut out the fluff and make sure only the most meaningful parts, those pushing the story forward are left in. It's the same with managing a language model's context window. You've got to keep only the most recent and relevant tokens in play. Working with APIs, many AI models provide APIs such as Hugging Face's transformers, API, that give you the ability to manipulate tokens directly. These APIs come with parameters like temperature and a max tokens, which you can adjust to influence the AI's responses. For instance, if you're penning a mystery narrative and desire more dramatic twists, tweaking the temperature setting could result in more diverse and unpredictable AI contributions. Tokens are more than mere counts, they also influence the model's output bias. Consider this, you're designing an AI baking assistant and you'd prefer egg- less recipes, by setting a negative bias for tokens like egg, eggs and gg. You can steer the AI away from suggesting egg- inclusive recipes. Stay tuned for future episodes where we'll touch on more sophisticated strategies for model customization. Mastering prompt engineering involves remembering a crucial fact: while AI tools are capable of remarkable things, they don't relieve us of our responsibility to think, analyze, and double- check. Think of it as a dance, a partnership where you're leading and the AI is following. However, every once in a while, the AI might surprise you with an unexpected twirl. Be ready for it. And remember, the beauty of this dance lies in the balance between your skill and the AI's capabilities. Modern language models, as impressive as they are, aren't flawless. They might occasionally lose track of details, misinterpret context, or even forget your name. This isn't a journey where AI does all the thinking. It's about pairing your human intuition, judgment, and skills with the AI's capabilities. Prompt engineering is an art form, blending an understanding of your AI partner's strengths and limitations, be it for text generation, prompt creation, or casual conversation, it's crucial that you're an active participant in the dialogue. Remember, it's on you to ensure Robo, or even Jerry doesn't become just another forgotten token. Apologies for the AI humor, couldn't resist it. Well, that's it for today's bonus episode. Thanks for listening in to the Art of AI, and check the show notes for links related to this episode. Looking forward to our next episode. Until then, bye, for now.