Sub-kanis (Agents)

When used in conjunction with Function Calling, kani can choose when to spawn “sub-kani” - self-contained “agents” capable of performing their own tasks, then reporting to the parent with their results.

For example, you might have the parent kani use a cheaper, faster model - but with the tradeoff that that model has a smaller context length. If you need it to perform a task that requires more context, you can spawn a sub-kani using a more expensive, slower model with a larger context.

Caution

Be careful when creating a new kani instance with an existing kani’s chat history! If you pass an old kani’s chat history to a new kani without copying it, the same list will be mutated.

Use newkani = Kani(..., chat_history=oldkani.chat_history.copy()) to pass a copy.

Index slicing (as shown in the example below) also creates a copy.

class KaniWithAISummarization(Kani):
    @ai_function()
    async def summarize_conversation(self):
        """Get the summary of the conversation so far."""
        # in this AI Function, we can spawn a sub-kani with a model that can handle
        # longer contexts, since the conversation may be longer than the fast model's
        # context window
        long_context_engine = OpenAIEngine(api_key, model="gpt-4-32k")
        # copy the parent's chat history to the child, except the last user message
        # and the function call ([:-2])
        sub_kani = Kani(long_context_engine, chat_history=self.chat_history[:-2])
        # then we ask it to summarize the whole thing, and return the result to the parent
        return await sub_kani.chat_round_str("Please summarize the conversation so far.")

Of course, the sub-kani you spawn doesn’t have to be a vanilla kani - you could imagine having multiple different kani types with different sets of functions or engines, each capable of performing their own specialized tasks.

>>> chat_in_terminal(KaniWithAISummarization(engine))
USER: Tell me about trains.
AI: Trains are a mode of long-distance transport...

[Multiple turns of conversation.]

USER: Summarize the conversation.
AI: Thinking (summarize_conversation)...
AI: Our conversation began with a general overview about trains, their history, types,
benefits, and how railway systems work around the world. We then moved onto discussing
the best high-speed rail lines around the world, with a specific focus on the Japanese
train system.

Your interest in trainspotting in Tokyo led to the sharing of some popular locations
in the city and a discussion about unique, non-standard train lines in Japan, including
lines that go through a tunnel of trees.

After exploring the topic of trains in Japan, we shifted to talk about lesser-known
attractions in Japan, which led to detailing a potential summer itinerary for a trip
starting in Tokyo and heading south.