Count as Decoded
Counting code units is the action of finding out how many code points will result from a given sequence of encoded information. Essentially, we run the decoding algorithm loop, but instead of giving the end user the decoded values, we instead simply provide the count for running that bulk operation.
Thusly, we use the algorithm as below to do the work. Given an
code_units with an
encoding, an initial
count set at 0, and any necessary additional
state, we can generically predict how many code units will result from a decoding operation by running the following loop:
⏩ Is the
inputvalue empty? Return the current results with the the empty
state, everything is okay ✅! Otherwise,
Set up an
intermediatestorage location of
code_points, using the
max_code_pointsof the input encoding, for the next operations.
end()) into the
code_pointstorage location, saving the returned
🛑 If it failed, return with the current
input(unmodified from before this iteration, if possible), current
Compute the difference between the
begin(intermediate)from the original step, and the
decode_one; add that difference to the current
begin()value to point to after what was read by the
⤴️ Go back to the start.
This involves a single encoding type, and so does not need any cooperation to count the
code_units. Note that this algorithm doesn’t show what the error handler does; if the error handler “erases” the failure by setting the result type’s
.error_code == ztd::text::encoding_error::ok, then the algorithm will keep going. This is useful to, for example, detect the maximum size of an operation even if it errors and would result in replacement characters being inserted (e.g., from ztd::text::replacement_handler_t).
Check out the API documentation for ztd::text::count_as_decoded to learn more.