Skip to main content

Prompting & Completion

Generally speaking, our models attempt to find the best continuation for a given input. In the world of Natural Language Processing (NLP) we call this a completion. To find the best completion we have to write good prompts, which is your input to the model. Our models continue the given prompt by choosing the most likely continuation from a probability distribution. Practically, this means that the model first recognizes the style of the prompt and then attempts to continue it accordingly. Depending on the task at hand, the structure and content of the prompt are essential to generating completions that match the input task.

Zero-shot Learning

For certain tasks, simply providing a natural language instruction to the model may be sufficient to have it complete a task. Completing a task like this is called zero-shot learning. Let’s illustrate this using an example.

Provide a short description of AI: AI is a set of techniques that use machine learning to make computer programs learn from data.

This worked well! However, the model’s best continuation is not necessarily aligned with your desired continuation. Therefore, it may not be enough to simply give the model a natural language instruction for its task.

Few-shot Learning

Especially for more complicated tasks, or those that require a specific format, you may have to explicitly show how to properly continue your prompt – by providing one or more examples. This is called few-shot learning (or one-shot learning in the case of just a single example). Let’s have a look at how this plays out in practice.

You’re thinking about where to go for your next vacation. Therefore, you’d like to have a list of possible travel destinations. You may ask the model to assist you in this matter. Before, it has proven effective to simply tell the model what to do. Let’s first try this zero-shot approach yet again:

I want to travel to a location where I can enjoy both beaches and mountains. Where could I go?

Think about how you would answer this query. You may get the gist and list a few possible locations that satisfy the given constraints. On the other hand, you could reply with a lengthy report of that one beautiful place you once visited. You could also say “I don’t know”. Anyhow, there are a dozen different ways in which you could answer this question, none of which are strictly right or wrong. Accordingly, the model outputs:

I’ve been to the Caribbean, and I’ve been to the Mediterranean, but I’ve never been to California.

Well, that’s not quite what we were hoping for. Let’s try to improve this prompt. To maximize our chances that the model outputs a list with cool vacation destinations, we must show examples of how to perform this task, i.e. use few-shot learning.

I want to travel to a location, where I can enjoy both beaches and mountains.
1. Lake Garda, Italy. This large Italian lake in the southern alps features gravel beaches and mountainside hiking trails.
2. Mallorca, Spain. This island is famous for its sandy beaches, turquoise water and hilly landscape.
3. Lake Tahoe, California. This famous lake in the Sierra Nevada mountains offers an amazing variety of outdoor activities.
4. Maui, Hawaii. This Hawaiian island boasts a spectacular coastline of untouched bays and beautiful volcanic peaks.

This is much better because the model understood the underlying structure of your task. To write good prompts use either instructions, examples or both to convey what you wish to do. Find the Jumpstart section for more detailed prompting advice for specific tasks.


Let’s go over some more tips and tricks for prompt design. In this case, we use a Q&A example to illustrate them. However, they are just as useful for all other kinds of prompting.

  1. Structure your prompt. For example, instead of asking:
    How many people live in New York City? How many people live in the United States?
    Q: How many people live in New York City?
    A: There are about 8.5 million people in the city.
    In case you are using few-shot learning, try to clearly separate the examples by using separators such as "###":
    Q: How long is the river Nile?
    A: It's roughly 6650 kilometers long.
    Q: How many people live in New York City?
    A: About 8.5 million people.
  2. You should not end your prompt on a space character. The model will most likely return nonsense.
    Q: How many people live in New York City?
    A: B:
  3. Check your prompts for mistakes. Spelling mistakes, grammatical errors or even double spaces may adversely affect your results.
    Q: How many people live in New York Cty?
    A: There are about 8.5 million people in the state of New York.
    In this example, the model did not recognize the word "city" correctly, thus it returned “state of New York” accompanied with the number of inhabitants of New York City. Thus, our answer is now wrong.