As the name suggests, the error function for lsqcurvefit is the sum of squared differences between measured and predicted. For a polar plot, you might consider the theta values to be known, and the r values to be measured and predicted, or vice versa. In eithr case, the use of lsqcurvefit() is straightforward.
If r and theta are both functions of a third variable, such as frequency, then you probably want to define the error function to be the distance on the Cartesian plane between the measured and predictred points. This is the standard approach when fitting frequency response functions.
Provide example data and a candidate function for fitting, if you want more detailed suggestions.
Good luck!
