Draft:HiFloat8
8-bit floating-point data format
From Wikipedia, the free encyclopedia
The HiFloat8 (HiF8) is an 8‑bit tapered‑precision floating‑point format proposed for the training and inference of deep learning models.[1][2][3] Under an 8‑bit encoding budget, HiF8 introduces an additional prefix‑coded dot field that selects the exponent field width (and a denormal mode), which allows the available mantissa bits to vary with the encoded exponent range and better balance numerical precision and dynamic range.[4]
 | Review waiting, please be patient.
This may take 8 weeks or more, since drafts are reviewed in no specific order. There are 2,916 pending submissions waiting for review.
Where to get help
How to improve a draft
You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Editor resources
Reviewer tools
|
Definition
HiF8 is an 8-bit floating-point format designed for deep learning workloads. As illustrated in Fig. 1, in addition to the conventional sign, exponent, and mantissa fields (with the exponent using an implicit leading magnitude bit, shown in red), HiF8 introduces a variable-length dot field encoded with prefix codes. The dot field directly indicates the storage width of the exponent field and carries a denormal flag, thereby enabling dynamic allocation of exponent and mantissa bit widths within the 8-bit encoding budget.[4]
Normal encoding mode
In the normal mode, the dot field encodes five values (0–4), corresponding to stored exponent widths from 0 to 4 bits. Given the sign bit, the dot field, and the selected exponent width, the mantissa width is determined by the 8-bit budget. Let Dot denote the decoded value of the dot field. When Dot = 0, no exponent bits are stored and the exponent is defined as E = 0. When Dot = 1–4, HiF8 encodes the exponent in sign-magnitude form and treats the most significant magnitude bit as an implicit 1 (shown in red in Fig. 1) and therefore does not store it. Specifically, with Dot = 1, the only possible exponents are E ∈ {−1, +1}, since E = 0 is represented by Dot = 0; when Dot = 2, the only possible exponents are E ∈ {−3, −2, +2, +3} (excluding {−1, +1} already covered by Dot = 1). By extension, the five cases Dot = 0–4 jointly represent all exponents in [-15, 15] without duplication.[4]
Denormal encoding mode
In the denormal mode, HiF8 further extends its dynamic range. As shown in Fig. 1, when the dot field indicates the denormal mode, no exponent field is stored; instead, the remaining mantissa bits are used to encode additional low-magnitude exponent values. Specifically, the denormal mode extends the 31 exponents [-15, 15] available in normal mode by seven additional exponents [-22, -16], with one additional pattern reserved for special values, yielding 38 distinct exponents in total over [-22, 15].[4]
Special values
In addition to regular numeric encodings, HiF8 defines four special values: Zero, NaN, positive and negative infinity. HiF8 does not distinguish between +0 and −0, using a single bit pattern to represent zero. Together, these special values cover common exceptional and boundary cases encountered in floating-point computation.[4]
Properties
Tapered precision
As illustrated in Fig. 3, fixed-field 8-bit floating-point formats such as FP8-E4M3 and FP8-E5M2 use constant exponent and mantissa widths, which can be difficult to simultaneously meet the dynamic-range and numerical precision requirements of deep learning training and inference within an 8-bit budget. In contrast, HiF8 exhibits a tapered-precision profile in which the effective precision varies with magnitude: it allocates more mantissa bits for values with magnitude near 1 while allocating more bits to the exponent toward the tails.[4]
Non-redundant encoding
HiF8 represents the exponent using sign-magnitude encoding and fixes the most significant magnitude bit to a known implicit value that is not stored. This construction ensures that the exponent ranges associated with different stored bit-widths do not overlap, resulting in a non-redundant encoding.[4]
Wide dynamic range
With the denormal mode, HiF8 extends the representable exponent range from [-15, 15] to [-22, 15], yielding 38 distinct exponent values in total—close to FP16’s 40 exponents.[4]