Is the "__half" a native C++ type?

Question

I am looking at https://github.com/NVIDIA/CUDALibrarySamples/blob/master/cuSPARSELt/spmma/spmma_example.cpp#L115-L117, and I see the __half type, which I believe corresponds to half precision float. I am wondering if this is a native C++ type? If not, is there a native float16 type?

Answer 1

While there is a IEEE-754 definition of 16-bit floating point, there's no C++ portable type for it as of yet. GGC uses _Float16, CUDA uses __half, etc.

There are also some 16-bit floating point variants like bfloat16.

For DirectXMath, I currently define a using HALF = uint16_t; which is not as type-safe as I'd like.

Answer 2

Confusingly, __half is not, but what you probably mean by half (fp16) is.

Essentially, __half is not a native type, nor is it in a sense a reserved keyword (beyond the standard nature of __ being reserved for compiler vendor implementations).

What you're referring to as "half" is a colloquial term for the IEEE-754:2008 Binary16 type (otherwise known as FP16) which is codified as a new addition into both the C and C++ 23 specifications.

Because of the nature of C++, you will be able to access the type via its C naming convention of _Float16, or its C++ naming convention of std::float16_t

On the topic of "Half". The naming comes from IEEE 32 bit floating points being regularly known as "single" precision, and their 64 bit siblings being known as "doubles". Invert the nomenclature and you have "half" precision types.

For details you can "C" the specifications here:

C23: https://open-std.org/JTC1/SC22/WG14/www/docs/n3054.pdf (see page 562)

IEEE-754:2008 (or newer): https://irem.univ-reunion.fr/IMG/pdf/ieee-754-2008.pdf

In usage: Work in progress for GCC implementation of Complex "Half": https://gcc.gnu.org/bugzilla/show_bug.cgi?id=106652

Depending on how up to date your compiler is, you may already have support for the type. From my own testing (and current work) _Float16 is available in Clang-14, GCC-12, ICC 2021.2 and ICX 2021.2.

Fortran has had the type since 2018