For a recent client project, I kept ChatOpenAI.max_retries as 2 and Model Retry Middleware as 2. This meant if the LLM call failed, there were 9 retries available. Not a blocker in terms of latency or cost, so the 9 retries weren’t a problem.
But the issue I noticed was that ChatOpenAI retries weren’t showing in the trace. It showed only one model call including the three retries. With the SDK retries off and LangChain handling the retries, it showed all the three model calls.