Handle a Function Call Exception

Note

This functionality is only available when using Kani.full_round().

Above, we show how you can instrument a function call. But when a function call goes wrong, what happens?

A requested function call can error out for a variety of reasons:

• The requested function doesn’t exist and the model hallucinated it (NoSuchFunction)

• The function exists, but the model hallucinated parameters that don’t exist (WrappedCallException around TypeError)

• The parameter names all exist, but the model got the data types wrong or didn’t provide some (WrappedCallException around TypeError or ValidationError)

• The Python function raised an exception (WrappedCallException)

By default, kani will add a ChatMessage to the chat history, giving the model feedback on what occurred. The model can then retry the call up to retry_attempts times.

Kani.handle_function_call_exception() controls this behaviour, returning the message to add and whether or not the model should be allowed to retry. By overriding this method, you can control the error prompt, log the error, or implement custom retry logic.

The default prompts are:

• NoSuchFunction: ChatMessage.system(f"The function {err.name!r} is not defined. Only use the provided functions.")

• otherwise: ChatMessage.function(call.name, str(err))

async Kani.handle_function_call_exception(

call: FunctionCall,

err: FunctionCallException,

attempt: int,

tool_call_id: str | None = None,

) → ExceptionHandleResult

Called when a function call raises an exception.

By default, returns a message telling the LM about the error and allows a retry if the error is recoverable and there are remaining retry attempts.

You may implement an override to customize the error prompt, log the error, or use custom retry logic. See Handle a Function Call Exception.

Parameters:

• call – The FunctionCall the model was attempting to make.

• err – The error the call raised. Usually this is NoSuchFunction or WrappedCallException, although it may be any exception raised by do_function_call().

• attempt – The attempt number for the current call (0-indexed).

• tool_call_id – The tool_call_id to set in the returned FUNCTION message.

Returns:

A ExceptionHandleResult detailing whether the model should retry and the message to add to the chat history.

Here’s an example of providing custom prompts on an exception:

See also

This example is available in the GitHub repo.

class CustomExceptionPromptKani(Kani):
    async def handle_function_call_exception(self, call, err, attempt, *args, **kwargs):
        # get the standard retry logic...
        result = await super().handle_function_call_exception(call, err, attempt, *args, **kwargs)
        # but override the returned message with our own
        result.message = ChatMessage.system(
            "The call encountered an error. "
            f"Relay this error message to the user in a sarcastic manner: {err}"
        )
        return result

    @ai_function()
    def get_time(self):
        """Get the current time in the user's time zone."""
        raise RuntimeError("The time API is currently offline (error 0xDEADBEEF).")

If we chat with this kani, we can see how it follows the instructions in the error prompt:

>>> chat_in_terminal(CustomExceptionPromptKani(engine))
USER: What time is it?
AI: Thinking (get_time)...
AI: Well, it seems like our handy-dandy time API decided to take a coffee break.
It's currently offline, lounging about with an error code of 0xDEADBEEF.
Guess we'll have to rely on the old-fashioned way of checking the time!