This looks like a lot to go through but its not, in the end its one line of code. Here I reference a 16bit A/D but it doesn't matter. Could be 8 bit or 24

The purpose of this tip is an easy way to linearize the A/D output and change it into engineering units at the same time. A 16 ibt A/D’s output when read is 0 to 65535 or 0x00 to 0xffff. The analog input is going to be some voltage within the A/D’s input range. For example lets say we have a 16bit A/D in the range of 0-5vdc. The sensor for this example is a pressure sensor 0-75 psig with an output of 0-5vdc. So a pressure reading of 0 is 0 volts and 0 A/D reading. A pressure of 75 psig is 5 volts and a A/D reading of 0xffff or 65535. This is great in the ideal but never happens in the real world. There is always some non linear part and an offset. In this case 0 psig may be .002 volts and the A/D would be some value above 0. The same goes for the high end (or anyplace for that matter) at 75 psig you may get 4.96 volts. So how do we correct this?

Y=MX+B is used to calculate the slope and offset. We don’t have to figure that out but the following is based on it.

We need to calculate a Slope and offset to be used as the Calibration factors. The KEY thing here to have a accurate measuring standard. Accurate is to whatever you deem necessary. It may be just a simple \$25 meter (DMM) or a \$20000 HP Scope calibrated to NIST specs. The point is the Calibration will only be as good as the standard used. So lets begain!!

# Terms used:

HighMeter = this is the reading from the cal standard at the highest point

High_AD = this is the A/D reading at HighMeter

LowMeter = this is the reading from the cal standard at the lowest point

Low_AD = this is the A/D reading at LowMeter

In this example I’ll use a analog pressure gauge as my standard. We’ll be calibrating the A/D to a Pressure gauge that has a 0-5vdc output and a pressure range of 0-75psi.

# The Calculations:

Slope = (HighMeter – LowMeter) / (High_AD – Low_AD)

Offset = LowMeter - ( Slope * Low_AD)

Val = ADraw * Slope + Offset

1. Set the pressure close to the high end of the range say 70psi as read on the standard this is the HighMeter value. You don't want to use that MAX end of the scale such as 75

3. Set the pressure close to the low end of the range say 5 psi as read on the standard this is the LowMeter value. You don't want to use that Min end of the scale such as 0

So lets assume at the following:

HighMeter =0x46 (70)

High_AD = 0xf230 (62000)

LowMeter =0x05 (5)

Low_AD = 0x1162 (4450)

Slope = (HighMeter – LowMeter) / (High_AD – Low_AD)

Slope = (70-5) / (6200-4450)

Slope = 65 / 57550

Slope = .001294

Offset = LowMeter - ( Slope * Low_AD)

Offset = 5 - (.001294 * 4450)

Offset = 5 - 5.7583

Offset = -0.7583

Now we have our Slope and Offset we can apply it to the A/D reading and the result will not only be in engineering units, psi in this case, but will be linear

In practice we get:

answer = 35375(AD) * .001294(slope) + (-.7583(offset))
answer = 45.0169 psi

In Proton PDS:
```Symbol slope = .001294
Symbol offset = -.7583
Dim AD as float 'must be right justified since only 10 bits I think that is default in PDS