Validate Decodable

Validation is the way to verify a given sequence of input can have a specific action performed on it. Particularly, we check here if the input of code units can be turned into code points of the given encoding. The way it does this, however, is two-fold:

  • it first decodes the input code units, to see if it can do the transformation without loss of information; then,

  • it encodes the output from the last step.

The algorithm for this is as follows:

  • ⏩ Is the input value empty? If so, is the state finished and have nothing to output? If both are true, return the current results with the the empty input, valid set to true, and states, everything is okay ✅!

  • ⏩ Otherwise,

    1. Set up an intermediate storage location of code_points, using the max_code_points of the input encoding, for the next operations.

    2. Set up an intermediate_checked_output storage location of code_units, using the max_code_units of the output encoding, for the next operations.

    3. Do the decode_one step from input (using its begin() and end()) into the intermediate code_point storage location.

      • 🛑 If it failed, return with the current input (unmodified from before this iteration, if possible), valid set to false, and states.

    4. Do the encode_one step from the intermediate into the intermediate_checked_output.

      • 🛑 If it failed, return with the current input (unmodified from before this iteration, if possible), valid set to false, and states.

  • ⏩ Update input‘s begin() value to point to after what was read by the decode_one step.

  • ⤴️ Go back to the start.

This fundamental process works for all encoding objects, provided they implement the basic Lucky 7. Unlike the encode and decode validation functions, this one does not have anything to compare its output to. By virtue of converting from the source to the destination, it is transcodable. Whether or not it can be round-tripped in the other direction isn’t particularly of concern, just that it can do so without error. This is the more general purpose forms of the encode or decode operations.

There are extension points used in the API that allow certain encodings to get around the limitation of having to do both the decode_one step and the encode_one step, giving individual encodings control over the verification of a single unit of input and of bulk validation as well.

Check out the API documentation for ztd::text::validate_transcodable_as to learn more.