The issue was caused by the function handle that included the reference to appbase.
sfun = @(x)streamTokens(app,x);
This was passed as part of the object
app.Bot = openAIChat("You are an AI assistant.",StreamFun=sfun);
And this was saved to the node data in UITree
s = struct('bot',app.Bot,'messages',app.Messages);
uitreenode(app.Tree,NodeData=s,Text="My Chat");
Then the node data was saved to the mat file.
PreviousChats.chat_history = arrayfun(@(x) x.NodeData, chats);
For future reference, the way I've troubleshooted this issue was to just load the MAT file at the command line (i.e take the app out of the picture) and inspect what its contents are. I check for any unexpected variables, and if nothing looks off, I then start to save the variables in small batches into a different MAT file until I see the warning get triggered. This helps track down the offending variable you can then dig into to see what it contains.
In this particular example, mat file contained node data, which contained a struct with two fields: bot and messages. I extracted the data saved in the bot field and I got the error message. When I saved the data in the messages field, I didn't. So I know that app reference in the bot field, which contained the openAIChat object. When I looked at the code that initialized the object, I saw that I was passing a function handle that contained the reference to the app.