Minimal Conversational Agent

One of the most exciting potential application areas of Large Language Models are conversational agents. Such agents can assist with daily tasks and even access knowledge within databases. In this example use-case, we build a minimal conversational agent that can answer questions about sports documents, "remember" the conversation history, and even have a smalltalk conversation. First, we import the necessary dependencies to build the application and set up the client to interact with the Aleph Alpha API.

import os
from typing import List
from aleph_alpha_client import Client, CompletionRequest, Document, Prompt, QaRequest

# If you are using a Windows machine, you must install the python-dotenv package and run the two below lines as well.
# from dotenv import load_dotenv

# load_dotenv()

client = Client(token=os.getenv("AA_TOKEN"))

We collected some texts about football (soccer), tennis and chess.

documents = [
    "Modern football was reinvented in England with the founding of the Football Association, whose 1863 rules form the basis of today's sport. The governing body of football is the Fédération Internationale de Football Association, better known by its acronym FIFA. The most prestigious international football competition is the World Cup, organised by FIFA every four years. This event is the most famous and most watched event in the world and has twice as many spectators as the Olympic Games.",
    "The Grand Slam tournaments are played in a knockout system for men and women as singles, doubles and mixed competition within about two weeks. In the singles, 128 participants compete in both the men's and women's tournaments, so that seven rounds are played until the final. Compared to other tournaments, the players usually have a day off after each match. The men play their matches over three winning sets, the women over two.",
    "The forerunner of all games in the chess family, i.e. not only European chess but also Xiangqi, Shōgis or Makruks, probably originated in northern India. This primitive chess was called Chaturanga. Details of the development of the game are not known, which led to the formation of myths, especially the Sissa legend.",
]

Now we want to write a class with which we can configure our minimal chat agent and provide it with the documents. In the first iteration of our chat agent, it can only answer questions based on its document base. For this, we use the Q&A-endpoint.

class ChatAgent:
    def __init__(
        self,
        documents: List[str],
    ):
        self.document_base = documents

    def answer(self, query: str):
        params = {
            "query": query,
            "documents": [
                Document.from_text(document) for document in self.document_base
            ],
        }
        request = QaRequest(**params)
        response = client.qa(request=request)
        if response.answers:
            return response.answers[0].answer.strip()
        else:
            return "Sorry, I cannot answer your question based on the available documents."

This is pretty cool, but we want a more natural feel when working with our agent. So, let's give it a name and the ability to converse in smalltalk. To make this work, our agent also needs memory capabilities.

class ChatMemory:
    def __init__(self):
        self.user_messages = []
        self.agent_messages = []

    def memorize_user_message(self, msg: str):
        self.user_messages.append(msg)

    def memorize_agent_message(self, msg: str):
        self.agent_messages.append(msg)

    # this function takes the entire memory (itself) and returns it as a string
    def get_as_string(self, user_prefix: str, agent_prefix: str):
        user_strings = (f"{user_prefix}: {msg}" for msg in self.user_messages)
        agent_strings = (f"{agent_prefix}: {msg}" for msg in self.agent_messages)
        return "\n".join([f"{x}\n{y}" for x, y in zip(user_strings, agent_strings)])


class ChatAgent:
    def __init__(
        self,
        documents: List[str],
        name: str,
        enable_smalltalk: bool = True,
    ):
        self.document_base = documents
        self.name = name
        self.enable_smalltalk = enable_smalltalk
        self.memory = ChatMemory()

    def answer_smalltalk(self, query: str):
        if not self.enable_smalltalk:
            return "Sorry, I cannot answer your question based on the available documents."

        # building the prompt based on the prior conversation, a description of the chat agent and the most recent user query
        # this prompt will then be sent to our complete endpoint to generate a smalltalk response
        prompt = (
            f"""A chat agent called {self.name} is helping a user navigate the world of sports knowledge.
{self.name}: Hello, I'm {self.name}, your sports information agent."""
            + self.memory.get_as_string(agent_prefix="{self.name}", user_prefix="User")
            + f"""
User: {query}
{self.name}:"""
            )
        params = {
            "prompt": Prompt.from_text(prompt),
            "maximum_tokens": 64,
            "stop_sequences": ["\n"],
        }
        request = CompletionRequest(**params)
        response = client.complete(request=request, model="luminous-supreme")
        return response.completions[0].completion.strip()

    def memorize(self, query: str, reply: str):
        self.memory.memorize_user_message(msg=query)
        self.memory.memorize_agent_message(msg=reply)

    def answer(self, query: str):
        params = {
            "query": query,
            "documents": [
                Document.from_text(document) for document in self.document_base
            ],
        }
        request = QaRequest(**params)
        response = client.qa(request=request)
        if response.answers:
            reply = response.answers[0].answer.strip()
        else:
            reply = self.answer_smalltalk(query=query)
        self.memorize(query=query, reply=reply)
        return reply

Now we can run our chat agent to either query sports information or have a smalltalk conversation.

chat_agent = ChatAgent(
    documents=documents, name="Dave", enable_smalltalk=True
)

while True:
    user_message = input("Your message: ")
    agent_message = chat_agent.answer(query=user_message)
    print(f"Agent message: {agent_message}")