Skip to content

Commit

Permalink
* numeric.c (flo_round): Avoid overflow by optimizing for trivial cases
Browse files Browse the repository at this point in the history
  [Bug ruby#5227]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33140 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
  • Loading branch information
marcandre committed Aug 31, 2011
1 parent 800081d commit 64e0cae
Show file tree
Hide file tree
Showing 2 changed files with 33 additions and 6 deletions.
5 changes: 5 additions & 0 deletions ChangeLog
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
Wed Aug 31 13:09:41 2011 Marc-Andre Lafortune <[email protected]>

* numeric.c (flo_round): Avoid overflow by optimizing for trivial
cases [Bug #5227]

Wed Aug 31 00:50:01 2011 NAKAMURA Usaku <[email protected]>

* win32/win32.c (rb_w32_select_with_thread): and my typo. we all must
Expand Down
34 changes: 28 additions & 6 deletions numeric.c
Original file line number Diff line number Diff line change
Expand Up @@ -1491,18 +1491,40 @@ flo_round(int argc, VALUE *argv, VALUE num)
VALUE nd;
double number, f;
int ndigits = 0;
int binexp;
long val;

if (argc > 0 && rb_scan_args(argc, argv, "01", &nd) == 1) {
ndigits = NUM2INT(nd);
}
number = RFLOAT_VALUE(num);
f = pow(10, abs(ndigits));

if (isinf(f)) {
if (ndigits < 0) number = 0;
}
else {
frexp (number , &binexp);

/* Let `exp` be such that `number` is written as:"0.#{digits}e#{exp}",
i.e. such that 10 ** (exp - 1) <= |number| < 10 ** exp
Recall that up to 17 digits can be needed to represent a double,
so if ndigits + exp >= 17, the intermediate value (number * 10 ** ndigits)
will be an integer and thus the result is the original number.
If ndigits + exp <= 0, the result is 0 or "1e#{exp}", so
if ndigits + exp < 0, the result is 0.
We have:
2 ** (binexp-1) <= |number| < 2 ** binexp
10 ** ((binexp-1)/log_2(10)) <= |number| < 10 ** (binexp/log_2(10))
If binexp >= 0, and since log_2(10) = 3.322259:
10 ** (binexp/4 - 1) < |number| < 10 ** (binexp/3)
binexp/4 <= exp <= binexp/3
If binexp <= 0, swap the /4 and the /3
So if ndigits + binexp/(4 or 3) >= 17, the result is number
If ndigits + binexp/(3 or 4) < 0 the result is 0
*/
if (isinf(number) || isnan(number)) {
/* Do nothing */
}
else if ((long)ndigits * (4 - (binexp > 0)) + binexp < 0) {
number = 0;
}
else if (((long)ndigits - 17) * (3 + (binexp > 0)) + binexp < 0) {
f = pow(10, abs(ndigits));
if (ndigits < 0) {
double absnum = fabs(number);
if (absnum < f) return INT2FIX(0);
Expand Down

0 comments on commit 64e0cae

Please sign in to comment.