Casting from long double to unsigned long long appears broken in the MSVC C++ compiler

Question

Consider the following code:

#include <iostream>

using namespace std;

int main(int argc, char *argv[])
{
    long double test = 0xFFFFFFFFFFFFFFFF;
    cout << "1: " << test << endl;
    unsigned long long test2 = test;
    cout << "2: " << test2 << endl;
    cout << "3: " << (unsigned long long)test << endl;
    return 0;
}

Compiling this code with GCC g++ (7.5.0) and running produces the following output as expected:

1: 1.84467e+19
2: 18446744073709551615
3: 18446744073709551615

However compiling this with the Microsoft Visual C++ compiler (16.8.31019.35, both 64-bit and 32-bit) and running produces the following output:

1: 1.84467e+19
2: 9223372036854775808
3: 9223372036854775808

When casting a value to an unsigned long long, the MSVC compiler won't give a value lager than the max of a (signed) long long.

Am I doing something wrong? 

Am I running into a compiler limitation that I do not know about?

Does anyone know of a possible workaround to this problem?

Answer 1

Because a MSVC long double is really just a double (as pointed out by @drescherjm in the comments), it does not have enough precision to contain the exact value of 0xFFFFFFFFFFFFFFFF. When this value is stored in the long double it gets "rounded" to a value that is lager than 0xFFFFFFFFFFFFFFFF. This then causes undefined behaviour when converting to an unsigned long long.

Answer 2

You are seeing undefined behaviour because, as pointed out in the comments, a long double is the same as a double in MSVC and the 'converted' value of your 0xFFFFFFFFFFFFFFFF (or ULLONG_MAX) actually gets 'rounded' to a slightly (but significantly) larger value, as can be seen in the following code:

int main(int argc, char* argv[])
{
    long double test = 0xFFFFFFFFFFFFFFFF;
    cout << 0xFFFFFFFFFFFFFFFFuLL << endl;
    cout << fixed << setprecision(16) << endl;
    cout << test << endl;
    return 0;
}

Output:

18446744073709551615
18446744073709551616.0000000000000000

Thus, when converting that floating-point value back to an unsigned long long, you are falling foul of the conversion rules specified in this Microsoft document:

• For conversion to unsigned long or unsigned long long, the result of converting an out-of-range value may be some value other than the highest or lowest representable value. Whether the result is a sentinel or saturated value or not depends on the compiler options and target architecture. Future compiler releases may return a saturated or sentinel value instead.

This UB can be further 'verified' (for want of a better term) by switching to the clang-cl compiler that can be used from within Visual Studio. For your original code, this then gives 0 for the values on both the "2" and "3" output lines.

Assuming that the clang (LLVM) compiler is not bound by the aforementioned "Microsoft Rules," we can, instead, fall back on the C++ Standard:

7.10 Floating-integral conversions      [conv.fpint]

¹     A prvalue of a floating-point type can be converted to a prvalue of an integer type. The conversion truncates; that is, the fractional part is discarded. The behavior is undefined if the truncated value cannot be represented in the destination type.