Accuracy, Precision & Resolution – Complete Guide for Instrument Technicians

1. Introduction

Accuracy in instrumentation is essential for reliable industrial measurement. Accuracy, precision and resolution are three important characteristics used to evaluate the performance of measuring instruments.

For example, a pressure transmitter installed in a boiler may measure pressure continuously. If the transmitter provides incorrect readings, it can affect process control, product quality and plant safety.

To understand the performance of any measuring instrument, three important characteristics must be understood:

• Accuracy
• Precision
• Resolution

These three terms are related to measurement quality, but they have different meanings.

In this chapter, we will understand these concepts with practical industrial examples.

2. What is Accuracy?

Accuracy is the closeness of a measured value to the true or reference value.

In simple words, accuracy tells us how close an instrument’s reading is to the actual value.

Example:

Suppose the actual pressure in a pipeline is 10 bar.

Three pressure gauges show:

• Gauge A = 9.9 bar
• Gauge B = 8.5 bar
• Gauge C = 10.0 bar

Gauge C provides the most accurate reading because its measurement is equal to the reference value.

Accuracy Formula:

Measurement Error = Measured Value − Reference Value

Absolute Error = |Measured Value − Reference Value|

Percentage Error:

Percentage Error = (Absolute Error / Reference Value) × 100

Example Calculation:

Reference pressure = 10 bar

Measured pressure = 9.8 bar

Absolute Error = |9.8 − 10|

Absolute Error = 0.2 bar

Percentage Error = (0.2 / 10) × 100

Percentage Error = 2%

Therefore, the measurement error is 2%.

Note: Accuracy is often specified by manufacturers as a percentage of full scale or percentage of reading. Always check the instrument datasheet to understand the stated accuracy specification.

3. What is Precision?

Precision refers to the closeness of repeated measurements to one another under the same measurement conditions.

In simple words, precision tells us how consistently an instrument gives the same reading when measuring the same quantity repeatedly.

Example:

Suppose a temperature transmitter measures the same temperature five times.

Readings:

• 100.1°C
• 100.2°C
• 100.1°C
• 100.2°C
• 100.1°C

These readings are very close to one another.

Therefore, the instrument demonstrates good precision.

However, if the actual temperature is 105°C, these readings are not accurate.

This means an instrument can be precise but not accurate.

4. Difference Between Accuracy and Precision

Accuracy and precision are two different characteristics of a measuring instrument.

Accuracy	Precision
Closeness to the true value	Closeness of repeated readings to each other
Indicates correctness	Indicates consistency
Related to measurement error	Related to repeatability and measurement variation
Can be evaluated against a reference standard	Can be evaluated using repeated measurements
Calibration helps identify and correct systematic error	Stable measurement conditions and instrument performance support repeatability

Industrial Example:

A pressure transmitter has a reference pressure of 10 bar.

Case 1: Readings are 9.9, 10.0 and 10.1 bar.

These readings are close to the reference value and to one another. The instrument demonstrates good accuracy and precision.

Case 2: Readings are 8.0, 8.1 and 8.0 bar.

These readings are close to one another but far from the reference value. The instrument is precise but inaccurate.

Case 3: Readings are 9.0, 10.5 and 11.0 bar.

These readings vary significantly. The instrument demonstrates poor precision.

5. What is Resolution?

Resolution is the smallest change in an input quantity that an instrument can detect or display.

In simple words, resolution tells us the smallest measurement difference an instrument can show.

Example 1: Digital Temperature Indicator

Suppose a digital temperature indicator displays:

• 25.1°C
• 25.2°C
• 25.3°C

The smallest displayed increment is 0.1°C.

Therefore, its display resolution is 0.1°C.

Example 2: Pressure Transmitter

Suppose a digital pressure transmitter displays pressure up to two decimal places:

10.01 bar, 10.02 bar, 10.03 bar.

Its display increment is 0.01 bar.

Therefore, its display resolution is 0.01 bar.

Important: Display resolution does not automatically mean measurement accuracy. An instrument may display many decimal places but still have a significant measurement error.

6. Difference Between Accuracy, Precision and Resolution

Parameter	Meaning	What It Tells Us
Accuracy	Closeness to true value	How correct is the reading?
Precision	Closeness of repeated readings	How consistent are the readings?
Resolution	Smallest detectable or displayed change	How small a change can be observed?

Remember:

• Accuracy = Correctness
• Precision = Consistency
• Resolution = Smallest detectable or displayed change

7. What is Repeatability?

Repeatability is the ability of an instrument to produce similar readings when the same measurement is repeated under the same conditions.

These conditions generally include:

• Same measuring instrument
• Same operator
• Same measurement method
• Same location
• Same environmental conditions
• Short time interval

Example:

A pressure gauge is tested at 5 bar five times.

Readings:

5.01 bar, 5.02 bar, 5.01 bar, 5.02 bar, 5.01 bar.

The readings are very close to one another, indicating good repeatability.

Repeatability is an important factor in evaluating precision.

8. What is Reproducibility?

Reproducibility refers to the closeness of measurement results when measurement conditions change, such as:

• Different operators
• Different instruments
• Different laboratories
• Different measurement locations
• Different measurement times

Example:

Two technicians independently measure the same pressure using different calibrated pressure gauges.

If their results are close to each other, the measurements demonstrate good reproducibility.

9. What is Sensitivity?

Sensitivity is the ratio of the change in an instrument’s output to the corresponding change in its input.

It indicates how much the output changes when the input changes.

Formula:

Sensitivity = Change in Output / Change in Input

Example:

A temperature transmitter produces an output change of 2 mA when the temperature changes by 10°C.

Sensitivity = 2 / 10

Sensitivity = 0.2 mA/°C

Therefore, the sensitivity is 0.2 mA per °C over this measurement interval.

10. What is Linearity?

Linearity describes how closely an instrument’s actual input-output relationship follows a specified straight-line relationship.

An instrument with good linearity produces an output that changes proportionally with its input over the specified operating range.

Example:

A pressure transmitter has:

• Range: 0–10 bar
• Output: 4–20 mA

Ideally:

Pressure	Output
0 bar	4 mA
2.5 bar	8 mA
5 bar	12 mA
7.5 bar	16 mA
10 bar	20 mA

These are ideal linear output values.

Actual transmitter performance may differ slightly within its specified accuracy and linearity limits.

11. Practical Industrial Example – Pressure Transmitter

Consider a pressure transmitter installed in an industrial pipeline.

Transmitter details:

• Measuring Range: 0–10 bar
• Output Signal: 4–20 mA
• Reference Pressure: 6 bar
• Measured Pressure: 5.9 bar

Step 1: Calculate the measurement error.

Error = Measured Value − Reference Value

Error = 5.9 − 6

Error = −0.1 bar

Step 2: Calculate the absolute error.

Absolute Error = 0.1 bar

Step 3: Calculate the percentage error.

Percentage Error = (0.1 / 6) × 100

Percentage Error = 1.67% approximately.

The transmitter reading is 0.1 bar below the reference pressure.

This example shows how measurement accuracy can be evaluated using a reference standard.

12. Why Are Accuracy, Precision and Resolution Important in Industry?

These characteristics are important for reliable industrial measurement and process control.

1. Process Control

Accurate measurements help PLC, DCS and control systems maintain the required process conditions.

2. Product Quality

Incorrect temperature, pressure or flow measurements can affect product quality.

3. Plant Safety

Reliable instrument measurements help operators monitor potentially unsafe process conditions.

4. Calibration

Instrument technicians use reference standards to check instrument performance and identify measurement errors.

5. Troubleshooting

Understanding precision, accuracy and resolution helps technicians identify measurement-related problems.

13. Common Mistakes Made by Instrument Technicians

1. Assuming that a digital display with more decimal places is always more accurate.
2. Confusing precision with accuracy.
3. Ignoring the instrument’s specified measurement range.
4. Using an unsuitable reference standard during calibration.
5. Ignoring environmental conditions during measurement.
6. Assuming that calibration automatically guarantees perfect accuracy.
7. Not checking the instrument datasheet for accuracy specifications.

14. Instrument Technician Interview Questions and Answers

Q1. What is accuracy in instrumentation?

Answer: Accuracy is the closeness of a measured value to the true or reference value.

Q2. What is precision?

Answer: Precision is the closeness of repeated measurements to one another under the same measurement conditions.

Q3. What is resolution?

Answer: Resolution is the smallest change in input that an instrument can detect or display.

Q4. Can an instrument be precise but not accurate?

Answer: Yes. An instrument can produce consistent readings that are far away from the true value.

Q5. What is repeatability?

Answer: Repeatability is the ability of an instrument to produce similar results under the same measurement conditions.

Q6. What is reproducibility?

Answer: Reproducibility is the closeness of measurement results when conditions such as operators, instruments or locations change.

Q7. What is sensitivity?

Answer: Sensitivity is the ratio of change in output to the corresponding change in input.

Q8. What is linearity?

Answer: Linearity describes how closely the actual input-output relationship follows a specified straight-line relationship.

Q9. Does higher resolution mean higher accuracy?

Answer: No. Resolution and accuracy are different measurement characteristics.

Q10. Why is accuracy important in industrial instrumentation?

Answer: Accuracy helps ensure reliable process measurement, process control, product quality and safe plant operation.

15. Multiple Choice Questions (MCQs)

Q1. Accuracy refers to:

A. Closeness to true value
B. Closeness of repeated readings
C. Smallest display increment
D. Instrument range

Correct Answer: A. Closeness to true value

Q2. Precision refers to:

A. Measurement range
B. Consistency of repeated readings
C. True value
D. Instrument output

Correct Answer: B. Consistency of repeated readings

Q3. Resolution is the:

A. Maximum measurement value
B. Minimum measurement value
C. Smallest detectable or displayed change
D. Measurement error

Correct Answer: C. Smallest detectable or displayed change

Q4. An instrument can be precise but not:

A. Digital
B. Accurate
C. Electrical
D. Calibrated

Correct Answer: B. Accurate

Q5. Which characteristic describes closeness to the true value?

A. Sensitivity
B. Precision
C. Accuracy
D. Resolution

Correct Answer: C. Accuracy

Q6. Repeatability means:

A. Similar results under the same conditions
B. Different results every time
C. Maximum output
D. Minimum range

Correct Answer: A. Similar results under the same conditions

Q7. Sensitivity is calculated as:

A. Input / Output
B. Change in Output / Change in Input
C. Output × Input
D. Error / Range

Correct Answer: B. Change in Output / Change in Input

Q8. A digital indicator displays 10.1°C, 10.2°C and 10.3°C. Its display increment is:

A. 1°C
B. 0.1°C
C. 0.01°C
D. 10°C

Correct Answer: B. 0.1°C

Q9. Which instrument characteristic is most directly associated with consistency of repeated measurements?

A. Repeatability
B. Range
C. Span
D. Resolution

Correct Answer: A. Repeatability

Q10. Does a display showing more decimal places guarantee better accuracy?

A. Yes
B. No
C. Always
D. Only for digital instruments

Correct Answer: B. No

16. Chapter Summary

In this chapter, we learned the important measurement characteristics used in industrial instrumentation.

Key points:

• Accuracy indicates closeness to the true or reference value.
• Precision indicates consistency between repeated measurements.
• Resolution indicates the smallest detectable or displayed change.
• Repeatability means similar results under the same conditions.
• Reproducibility means similar results when measurement conditions change.
• Sensitivity describes output change relative to input change.
• Linearity describes how closely the input-output relationship follows a specified straight line.

Understanding these concepts helps Instrument Technicians select, test, calibrate and troubleshoot industrial measuring instruments.

17. What Is Next?

CHAPTER 04: Range, Span, LRV & URV

In the next chapter, we will learn:

• What is Instrument Range?
• What is Span?
• What is LRV (Lower Range Value)?
• What is URV (Upper Range Value)?
• Difference between Range and Span
• Practical examples of pressure transmitters
• Interview Questions and MCQs

Continue learning with THE ENGINEERS GROUP – Your Free Industrial Instrumentation Learning Platform.

Learn. Practice. Improve. Grow.