Types
Internally, the Text type is represented as an array of Word8
UTF-8 code units. The offset and length fields in the constructor
are in these units, not units of Char.
Invariants that all functions must maintain:
Since the Text type uses UTF-8 internally, it cannot represent
characters in the reserved surrogate code point range U+D800 to
U+DFFF. To maintain this invariant, the safe function maps
Char values in this range to the replacement character (U+FFFD,
'�').
A space efficient, packed, unboxed Unicode text type.
Text :: A.Array -> Int -> Int -> Text
A.Arraybytearray encoded as UTF-8
Intoffset in bytes (not in Char!), pointing to a start of UTF-8 sequence
Intlength in bytes (not in Char!), pointing to an end of UTF-8 sequence
Type synonym for the strict flavour of Text.
Construction
text :: A.Array -> Int -> Int -> Text
Construct a Text without invisibly pinning its byte array in
memory if its length has dwindled to zero.
It ensures that empty Text values are shared.
A.Arraybytearray encoded as UTF-8
Intoffset in bytes (not in Char!), pointing to a start of UTF-8 sequence
Intlength in bytes (not in Char!), pointing to an end of UTF-8 sequence
textP :: A.Array -> Int -> Int -> Text
Use text instead
Safety
safe :: Char -> Char
Map a Char to a Text-safe value.
Unicode Surrogate code points are not included in the set of Unicode
scalar values, but are unfortunately admitted as valid Char
values by Haskell. They cannot be represented in a Text. This
function remaps those code points to the Unicode replacement
character (U+FFFD, '�'), and leaves other code points
unchanged.
Code that must be here for accessibility
append :: Text -> Text -> Text
O(n) Appends one Text to the other by copying both of them
into a new Text.
Utilities
firstf :: (a -> c) -> Maybe (a, b) -> Maybe (c, b)
Apply a function to the first element of an optional pair.
Checked multiplication
mul :: Int -> Int -> Int
Checked multiplication. Calls error if the result would
overflow.
mul32 :: Int32 -> Int32 -> Int32
Checked multiplication. Calls error if the result would
overflow.
mul64 :: Int64 -> Int64 -> Int64
Checked multiplication. Calls error if the result would
overflow.
Debugging
showText :: Text -> String
A useful show-like function for debugging purposes.
Conversions
pack :: String -> Text
O(n) Convert a String into a Text.
Performs replacement on invalid scalar values, so unpack . pack is not id:
>>> Data.Text.unpack (pack "\55555")
"\65533"
- Copyright
- (c) 2008, 2009 Tom Harper, (c) 2009, 2010 Bryan O'Sullivan, (c) 2009 Duncan Coutts