WTF-8

Wobby Transformat Format 8 (WTF-8) is an encoding scheme that preserves lone-encoded surrogates, which is generally not allowed in streams composed purely of Unicode Scalar Values.

constexpr wtf8_t ztd::text::wtf8 = {}

An instance of the WTF-8 type for ease of use.

using ztd::text::wtf8_t = basic_wtf8<uchar8_t>

A “Wobbly Transformation Format 8” (WTF-8) Encoding that traffics in char8_t. See ztd::text::basic_wtf8 for more details.

Base Template

template<typename _CodeUnit, typename _CodePoint = unicode_code_point>
class basic_wtf8 : public __utf8_with<basic_wtf8<_CodeUnit, _CodePoint>, _CodeUnit, _CodePoint, __txt_detail::__empty_state, __txt_detail::__empty_state, false, true, false>

A “Wobbly Transformation Format 8” (WTF-8) Encoding that traffics in, specifically, the desired code unit type provided as a template argument.

Remark

This type as a maximum of 4 input code points and a maximum of 1 output code point. Unpaired surrogates are allowed in this type, which may be useful for dealing with legacy storage and implementations of the Windows Filesystem (modern Windows no longer lets non-Unicode filenames through). For a strict, Unicode-compliant UTF-8 Encoding, see ztd::text::basic_utf8 .

Template Parameters
  • _CodeUnit – The code unit type to use.

  • _CodePoint – The code point type to use.

Public Types

using is_unicode_encoding = ::std::true_type

Whether or not this encoding that can encode all of Unicode.

using decode_state = __txt_detail::__empty_state

The state that can be used between calls to the encoder and decoder. It is normally an empty struct because there is no shift state to preserve between complete units of encoded information.

using encode_state = __txt_detail::__empty_state

The state that can be used between calls to the encoder and decoder. It is normally an empty struct because there is no shift state to preserve between complete units of encoded information.

using code_unit = _CodeUnit

The individual units that result from an encode operation or are used as input to a decode operation. For UTF-8 formats, this is usually char8_t, but this can change (see ztd::text::basic_utf8).

using code_point = _CodePoint

The individual units that result from a decode operation or as used as input to an encode operation. For most encodings, this is going to be a Unicode Code Point or a Unicode Scalar Value.

using is_decode_injective = ::std::true_type

Whether or not the decode operation can process all forms of input into code point values. Thsi is true for all Unicode Transformation Formats (UTFs), which can encode and decode without a loss of information from a valid collection of code units.

using is_encode_injective = ::std::true_type

Whether or not the encode operation can process all forms of input into code unit values. This is true for all Unicode Transformation Formats (UTFs), which can encode and decode without loss of information from a valid input code point.

Public Static Functions

static inline constexpr auto encode_one(_InputRange &&__input, _OutputRange &&__output, _ErrorHandler &&__error_handler, encode_state &__s)

Encodes a single complete unit of information as code units and produces a result with the input and output ranges moved past what was successfully read and written; or, produces an error and returns the input and output ranges untouched.

Remark

To the best ability of the implementation, the iterators will be returned untouched (e.g., the input models at least a view and a forward_range). If it is not possible, returned ranges may be incremented even if an error occurs due to the semantics of any view that models an input_range.

Parameters
  • __input – [in] The input view to read code points from.

  • __output – [in] The output view to write code units into.

  • __error_handler – [in] The error handler to invoke if encoding fails.

  • __s – [inout] The necessary state information. For this encoding, the state is empty and means very little.

Returns

A ztd::text::encode_result object that contains the reconstructed input range, reconstructed output range, error handler, and a reference to the passed-in state.

static inline constexpr auto decode_one(_InputRange &&__input, _OutputRange &&__output, _ErrorHandler &&__error_handler, decode_state &__s)

Decodes a single complete unit of information as code points and produces a result with the input and output ranges moved past what was successfully read and written; or, produces an error and returns the input and output ranges untouched.

Remark

To the best ability of the implementation, the iterators will be returned untouched (e.g., the input models at least a view and a forward_range). If it is not possible, returned ranges may be incremented even if an error occurs due to the semantics of any view that models an input_range.

Parameters
  • __input – [in] The input view to read code uunits from.

  • __output – [in] The output view to write code points into.

  • __error_handler – [in] The error handler to invoke if encoding fails.

  • __s – [inout] The necessary state information. For this encoding, the state is empty and means very little.

Returns

A ztd::text::decode_result object that contains the reconstructed input range, reconstructed output range, error handler, and a reference to the passed-in state.

Public Static Attributes

static constexpr ::std::size_t max_code_points

The maximum number of code points a single complete operation of decoding can produce. This is 1 for all Unicode Transformation Format (UTF) encodings.

static constexpr ::std::size_t max_code_units

The maximum code units a single complete operation of encoding can produce. If overlong sequence allowed, this is 6: otherwise, this is 4.