What exactly does ε represent in numerical computing
显示 更早的评论
I often see ε (epsilon or flipped 3, whatever you call it) in different contexts and I'm not sure I fully grasp it. Sometimes it's the machine epsilon (eps in MATLAB), sometimes it's just a small positive number in proofs and it's occasionally used in optimization tolerances.
Is machine epsilon specifically the smallest number such that 1 + ε > 1 in floating point? And when people set solver tolerances like 1e-6, is that related to machine epsilon or just an arbitrary choice? Just wanted to hear how people actually think about it 'practically'!
回答（2 个）
Epsilon is just a letter. Like all letters, epsilon can mean lots of different things, just like the letters "alpha" and "gamma" have lots of different meaning depending on the field, the context, the author, the target audience, etc.
"Is machine epsilon specifically the smallest number such that 1 + ε > 1 in floating point?"
That is one common definition, yes. But note that there is no one "floating point", there are many standard (and some esoteric) floating point number systems, all(?) of them have some concept of machine epsilon with different actual values:
"And when people set solver tolerances like 1e-6, is that related to machine epsilon or just an arbitrary choice?"
Arbitrary choice: they selected a letter and defined it to mean something. You can do that too.
Some letters are used by convention for particular meanings, e.g. pi. But even pi, famously used for the transcendental constant, is also used with plenty of other meanings.
John D'Errico
about 12 hours 前
编辑：John D'Errico
about 8 hours 前
Just to add some more info...
For example, pi(x) is known as the prime counting function to some, the number of primes less than or equal to x.
Just one of the various alternatives pi can be used to represent. There are too few greek letters for all the things one might want to represent. As a name, eps also has many various meanings, though the most common one in my experience is to represent a small thing. (But then, I've always been primarily a numerical animalist.)
A solver tolerance of 1e-6 is just a relatively small number. One hopes it is small enough that you are sufficiently close to a "solution". One part in a million seems small to many. But depending on whether that tolerance is an absolute or a relative tolerance can be a hugely different thing.
And for SOME (approximately quadratic) problems, one might want to use a tolerance that is at least close to sqrt(eps), where here eps is machine epsilon for the given precision you are working in. As a double, we have
eps('double')
sqrt(eps('double'))
This is at least reasonably close to the arbitrary 1e-6 tolerance many use.
Finally, is eps the smallest number you can add to 1, and see a different result? I think this is a common misperception.
hexnums = ['3ff0000000000000'
'3ff0000000000001'];
These two numbers are represented in hex, different by one bit, down in the least significant bit in the vicinity of 1.
dubnums = hex2num(hexnums);
num2str(dubnums,55)
You should see the first is EXACTLY 1. The second is different by eps('double'), or 2.2204e-16.
diff(dubnums)
So the stride between two consecutive doubles near 1, is eps. A 1 bit difference in the LSB. But is that actually the smallest number we could have added to 1, and still get a different result?
delta = 1;
x = 1;
xhat = x + delta;
d = 1023/1024;
while x ~= xhat
delta = d*delta;
xhat = x + delta;
end
delta = delta/d
As you can see, delta here is SMALLER than eps. If I went any smaller, then we had x==x+delta.
x == x+delta
x == x + delta*d
A better way to think of eps is as the relative "stride" between two consecutive floating point numbers.
What happens when we added delta to 1? We get the number 1+1.1103e-16, which is then ROUNDED to the closest double precision number in that vicinity, that is...
num2str(1+1.1103e-16,55)
So you should now understand that delta should be eps/2. delta as I created it is 1/2 of the stride between consecutive floating point numbers, and eps is that stride between consecutive floats.
eps('double')/2
For larger numbers of course, that stride is itself larger in absolute value. For example, 1024 is a double precision number. I would expect eps(1024) to be proportionally larger, as it is:
eps(1024)
1024*eps
Of course, in a proof, where epsilon is just used as a small number, in a limit where epsilon goes to zero, etc., there it is just some tiny number. And this has essentially nothing to do with the computationally rigorous definition of eps, in terms of the stride between floating point numbers.
Ask in a comment if I can clarify this anymore.
3 个评论
If eps(x) is the stride between from x to xp, where xp is the next larger floating point number from x, then what is the stride from xn to x, where xn is the next smaller floating point number from x? Is that just eps(x)/2?
For example, with x = 1, we have
[x,xp,xn] = deal('3ff0000000000000','3ff0000000000001','3fefffffffffffff');
x = hex2num(x); xp = hex2num(xp); xn = hex2num(xn);
[isequal(eps(x),xp-x), isequal(eps(x)/2,x-xn)]
Does the relationship x - xn == eps(x)/2 hold in general?
Good question, but no. Not quite. Consider x0 here.
x0 = 1.25;
hx0 = num2hex(x0)
Now I'll construct the pair of numbers just 1 LSB on either side of x0 as respectively
hxlo = '3ff3ffffffffffff';
hxhi = '3ff4000000000001';
xlo = hex2num(hxlo); % convert to doubles
xhi = hex2num(hxhi);
x = [xlo,x0,xhi]
format long g
diff(x)
As you see, the lower and upper strides were identical. At some rare locations, the stride below a point and the stride above are different, as you showed. In fact, the locations where the stride changes must be rare. In fact, I think we can list the set of locations where the stride below changes from the stride above. If I had to guess, it would be all integer powers of 2. For example...
X = 2.^(-5:5);
hX = num2hex(X)
Now, construct the pars of doubles just above and below.
hXhi = num2hex(X + eps(X))
Clearly all of those numbers were 1 LSB above. But as I said, this is where the break point appears, the stride just below those powers of 2 is only eps(X)/2.
hXlo = num2hex(X - eps(X)/2)
However, I believe the stride does not change at any other locations besides powers of 2. This is something that deserves better proof than I have offered, but you get what you pay for. ;-) As I showed above, the sub and super strides are the same at almost all points on the number line.
Proof by Wikipedia: Double-precision floating-point format - Wikipedia
"The spacing as a fraction of the numbers in the range from 2^n to 2^(n+1) is 2^(n−52)."
Trying it for a couple of values
isequal(eps(4.5),2^(2-52))
isequal(eps(2^10 + 51),2^(10-52))
isequal(eps(2^-8 + 1e-4),2^(-8-52))
But the the wikipedia page doesn't explain how it works for negative numbers, i.e., does eps(x), with x < 0 take us to the next number away from or towards zero? According to eps it's the former (I think).
Also there must be a separate rule for very small numbers, e.g.
format long
eps(0)
format hex
eps(0)
2^-1023 * 2^-51
类别
在 帮助中心 和 File Exchange 中查找有关 Functions 的更多信息
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
Translated by 
