Live Chat with Tek representatives. Available 9:30 AM - 5:30 PM IST

Call

Call us at

Available 9:30 AM – 5:30 PM (IST) Business Days

MODEL or KEYWORD

Feedback

# How do I calculate the resistance accuracy of the Model 2450 on the 2 Ohm and greater than 200M ohm range?

### Question :

How do I calculate the resistance accuracy of the Model 2450 on the 2 Ohm and greater than 200M ohm range?

Below are the numbers to use for calculating the accuracy of 2 ohm and 200M ohms ranges.
Since these particular ranges are only realized by using MANUAL OHMS, the below source values and measure values were used.
In manual ohms the user can choose what source function and value to use to make the resistance measurement.
On the 2 ohm range = 1A source with 1V measure.
On the 200M ohm range = 1uA source with 200V measure.

2 ohm range = 0.697%
200M ohm range = 0.0529%

Sample calculation on 2 ohm range for 1 ohm measurement
1 A source accuracy (source readback accuracy) = 0.03% + 500uA
This is 300uA + 500uA = 800uA
This is 0.0008A / 1A = 0.08%
Now to figure out the voltage measurement accuracy (use measure accuracy).
1V measurement accuracy = 0.012% + 300uV
This is 120uV + 300uV = 420uV.
This is 0.42mV / 1V = 0.042%
Current source = 0.08%
Voltage measure = 0.042%
Together we have 0.122% for the 2 ohm range using a 1A source and 1V measurement.

This FAQ Applies to:

Product Series:

Product:

FAQ ID 470646

View all FAQs »