How do I calculate the resistance accuracy of the Model 2450 on the 2 Ohm and greater than 200M ohm range?
Below are the numbers to use for calculating the accuracy of 2 ohm and 200M ohms ranges.
Since these particular ranges are only realized by using MANUAL OHMS, the below source values and measure values were used.
In manual ohms the user can choose what source function and value to use to make the resistance measurement.
On the 2 ohm range = 1A source with 1V measure.
On the 200M ohm range = 1uA source with 200V measure.
2 ohm range = 0.697%
200M ohm range = 0.0529%
Sample calculation on 2 ohm range for 1 ohm measurement
1 A source accuracy (source readback accuracy) = 0.03% + 500uA
This is 300uA + 500uA = 800uA
This is 0.0008A / 1A = 0.08%
Now to figure out the voltage measurement accuracy (use measure accuracy).
1V measurement accuracy = 0.012% + 300uV
This is 120uV + 300uV = 420uV.
This is 0.42mV / 1V = 0.042%
Now add up each accuracy
Current source = 0.08%
Voltage measure = 0.042%
Together we have 0.122% for the 2 ohm range using a 1A source and 1V measurement.
This FAQ Applies to:
Product Series: 2400 Graphical Series SMU
FAQ ID 470646View all FAQs »