Roundoff Errors in the Evaluation of the Cost Function in Sine Wave Based ADC Testing

Balázs Renczes, István Kollár

Abstract

In this paper sine fitting algorithms are investigated for the purpose of ADC testing. The aim is to decide whether the minimum of the cost function (CF) has been reached. For this, two different types of algorithms, the conventional Levenberg-Marquardt and the genetic-type Differential Evolution methods are investigated in order to compare their optima. It is shown that due to roundoff errors the bottom of the cost function is fairly uneven for conventional number representations for the Maximum Likelihood method, hence the minimum can only be determined with decreased precision. Finally, a band is calculated in which solutions can be considered equivalent, since their CF difference is smaller than roundoff errors.

Keywords

Sine wave fit, Cost Function Evaluation, Maximum Likelihood, ADC testing

Download

IMEKO-TC4-2014-214.pdf

DOI

-

IMEKO TC

TC4 - Measurement of Electrical Quantities

Back to the proceedings