newtype Builder
Builders denote sequences of bytes.
They are Monoids where
mempty is the zero-length sequence and
mappend is concatenation, which runs in O(1).
Builders are used to efficiently construct sequences of bytes from
smaller parts.
Typically,
such a construction is part of the implementation of an encoding, i.e.,
a function for converting Haskell values to sequences of bytes.
Examples of encodings are the generation of the sequence of bytes
representing a HTML document to be sent in a HTTP response by a
web application or the serialization of a Haskell value using
a fixed binary format.
For an efficient implementation of an encoding,
it is important that (a) little time is spent on converting
the Haskell values to the resulting sequence of bytes and
(b) that the representation of the resulting sequence
is such that it can be consumed efficiently.
Builders support (a) by providing an O(1) concatentation operation
and efficient implementations of basic encodings for Chars, Ints,
and other standard Haskell values.
They support (b) by providing their result as a LazyByteString,
which is internally just a linked list of pointers to chunks
of consecutive raw memory.
LazyByteStrings can be efficiently consumed by functions that
write them to a file or send them over a network socket.
Note that each chunk boundary incurs expensive extra work (e.g., a system call)
that must be amortized over the work spent on consuming the chunk body.
Builders therefore take special care to ensure that the
average chunk size is large enough.
The precise meaning of large enough is application dependent.
The current implementation is tuned
for an average chunk size between 4kb and 32kb,
which should suit most applications.
As a simple example of an encoding implementation, we show how to efficiently convert the following representation of mixed-data tables to an UTF-8 encoded Comma-Separated-Values (CSV) table.
data Cell = StringC String
| IntC Int
deriving( Eq, Ord, Show )
type Row = [Cell]
type Table = [Row]We use the following imports.
import qualified "Data.ByteString.Lazy" as L
import "Data.ByteString.Builder"
import Data.List ('Data.List.intersperse')
CSV is a character-based representation of tables. For maximal modularity,
we could first render Tables as Strings and then encode this String
using some Unicode character encoding. However, this sacrifices performance
due to the intermediate String representation being built and thrown away
right afterwards. We get rid of this intermediate String representation by
fixing the character encoding to UTF-8 and using Builders to convert
Tables directly to UTF-8 encoded CSV tables represented as
LazyByteStrings.
encodeUtf8CSV :: Table -> L.LazyByteString
encodeUtf8CSV = 'toLazyByteString' . renderTable
renderTable :: Table -> Builder
renderTable rs = 'mconcat' [renderRow r '<>' 'charUtf8' \'\\n\' | r <- rs]
renderRow :: Row -> Builder
renderRow [] = 'mempty'
renderRow (c:cs) =
renderCell c \<\> mconcat [ charUtf8 \',\' \<\> renderCell c\' | c\' <- cs ]
renderCell :: Cell -> Builder
renderCell (StringC cs) = renderString cs
renderCell (IntC i) = 'intDec' i
renderString :: String -> Builder
renderString cs = charUtf8 \'\"\' \<\> 'foldMap' escape cs \<\> charUtf8 \'\"\'
where
escape \'\\\\\' = charUtf8 \'\\\\\' \<\> charUtf8 \'\\\\\'
escape \'\\\"\' = charUtf8 \'\\\\\' \<\> charUtf8 \'\\\"\'
escape c = charUtf8 c
Note that the ASCII encoding is a subset of the UTF-8 encoding,
which is why we can use the optimized function intDec to
encode an Int as a decimal number with UTF-8 encoded digits.
Using intDec is more efficient than ,
as it avoids constructing an intermediate stringUtf8 . showString.
Avoiding this intermediate data structure significantly improves
performance because encoding Cells is the core operation
for rendering CSV-tables.
See Data.ByteString.Builder.Prim for further
information on how to improve the performance of renderString.
We demonstrate our UTF-8 CSV encoding function on the following table.
strings :: [String]
strings = [\"hello\", \"\\\"1\\\"\", \"λ-wörld\"]
table :: Table
table = [map StringC strings, map IntC [-3..3]]
The expression encodeUtf8CSV table results in the following lazy
LazyByteString.
Chunk "\"hello\",\"\\\"1\\\"\",\"\206\187-w\195\182rld\"\n-3,-2,-1,0,1,2,3\n" EmptyWe can clearly see that we are converting to a binary format. The 'λ' and 'ö' characters, which have a Unicode codepoint above 127, are expanded to their corresponding UTF-8 multi-byte representation.
We use the criterion library (http://hackage.haskell.org/package/criterion)
to benchmark the efficiency of our encoding function on the following table.
import Criterion.Main -- add this import to the ones above
maxiTable :: Table
maxiTable = take 1000 $ cycle table
main :: IO ()
main = defaultMain
[ bench "encodeUtf8CSV maxiTable (original)" $
whnf (L.length . encodeUtf8CSV) maxiTable
]On a Core2 Duo 2.20GHz on a 32-bit Linux,
the above code takes 1ms to generate the 22'500 bytes long LazyByteString.
Looking again at the definitions above,
we see that we took care to avoid intermediate data structures,
as otherwise we would sacrifice performance.
For example,
the following (arguably simpler) definition of renderRow is about 20% slower.
renderRow :: Row -> Builder
renderRow = mconcat . intersperse (charUtf8 ',') . map renderCellSimilarly, using O(n) concatentations like ++ or the equivalent concat
operations on strict and LazyByteStrings should be avoided.
The following definition of renderString is also about 20% slower.
renderString :: String -> Builder
renderString cs = charUtf8 $ "\"" ++ concatMap escape cs ++ "\""
where
escape '\\' = "\\"
escape '\"' = "\\\""
escape c = return cApart from removing intermediate data-structures,
encodings can be optimized further by fine-tuning their execution
parameters using the functions in Data.ByteString.Builder.Extra and
their "inner loops" using the functions in
Data.ByteString.Builder.Prim.
newtype BuilderBuilders denote sequences of bytes.
They are Monoids where
mempty is the zero-length sequence and
mappend is concatenation, which runs in O(1).
Builders are buffer-filling functions. They are
executed by a driver that provides them with an actual buffer to
fill. Once called with a buffer, a Builder fills it and returns a
signal to the driver telling it that it is either done, has filled the
current buffer, or wants to directly insert a reference to a chunk of
memory. In the last two cases, the Builder also returns a
continuation Builder that the driver can call to fill the next
buffer. Here, we provide the two drivers that satisfy almost all use
cases. See Data.ByteString.Builder.Extra, for information
about fine-tuning them.toLazyByteString :: Builder -> L.LazyByteStringExecute a Builder and return the generated chunks as a LazyByteString.
The work is performed lazy, i.e., only when a chunk of the LazyByteString
is forced.
hPutBuilder :: Handle -> Builder -> IO ()Output a Builder to a Handle.
The Builder is executed directly on the buffer of the Handle. If the
buffer is too small (or not present), then it is replaced with a large
enough buffer.
It is recommended that the Handle is set to binary and
BlockBuffering mode. See hSetBinaryMode and
hSetBuffering.
This function is more efficient than hPut . because in
many cases no buffer allocation has to be done. Moreover, the results of
several executions of short toLazyByteStringBuilders are concatenated in the Handles
buffer, therefore avoiding unnecessary buffer flushes.
writeFile :: FilePath -> Builder -> IO ()Write a Builder to a file.
Similarly to hPutBuilder, this function is more efficient than
using hPut . toLazyByteString with a file handle.
byteString :: S.StrictByteString -> BuilderCreate a Builder denoting the same sequence of bytes as a
StrictByteString.
The Builder inserts large StrictByteStrings directly, but copies small ones
to ensure that the generated chunks are large on average.
lazyByteString :: L.LazyByteString -> BuilderCreate a Builder denoting the same sequence of bytes as a lazy
LazyByteString.
The Builder inserts large chunks of the LazyByteString directly,
but copies small ones to ensure that the generated chunks are large on
average.
shortByteString :: Sh.ShortByteString -> BuilderConstruct a Builder that copies the ShortByteString.
int8 :: Int8 -> BuilderEncode a single signed byte as-is.
word8 :: Word8 -> BuilderEncode a single unsigned byte as-is.
int16BE :: Int16 -> BuilderEncode an Int16 in big endian format.
int32BE :: Int32 -> BuilderEncode an Int32 in big endian format.
int64BE :: Int64 -> BuilderEncode an Int64 in big endian format.
word16BE :: Word16 -> BuilderEncode a Word16 in big endian format.
word32BE :: Word32 -> BuilderEncode a Word32 in big endian format.
word64BE :: Word64 -> BuilderEncode a Word64 in big endian format.
floatBE :: Float -> BuilderEncode a Float in big endian format.
doubleBE :: Double -> BuilderEncode a Double in big endian format.
int16LE :: Int16 -> BuilderEncode an Int16 in little endian format.
int32LE :: Int32 -> BuilderEncode an Int32 in little endian format.
int64LE :: Int64 -> BuilderEncode an Int64 in little endian format.
word16LE :: Word16 -> BuilderEncode a Word16 in little endian format.
word32LE :: Word32 -> BuilderEncode a Word32 in little endian format.
word64LE :: Word64 -> BuilderEncode a Word64 in little endian format.
floatLE :: Float -> BuilderEncode a Float in little endian format.
doubleLE :: Double -> BuilderEncode a Double in little endian format.
Char and String into Builders in various encodings.char7 :: Char -> BuilderChar7 encode a Char.
string7 :: String -> BuilderChar7 encode a String.
char8 :: Char -> BuilderChar8 encode a Char.
string8 :: String -> BuilderChar8 encode a String.
Chars and Strings unless an application
really requires another encoding.charUtf8 :: Char -> BuilderUTF-8 encode a Char.
stringUtf8 :: String -> BuilderUTF-8 encode a String.
Note that stringUtf8 performs no codepoint validation and consequently may
emit invalid UTF-8 if asked (e.g. single surrogates).
byteStringHex :: S.StrictByteString -> BuilderEncode each byte of a StrictByteString using its fixed-width hex encoding.
doubleDec :: Double -> BuilderReturns a rendered Double. Matches `show` in displaying in standard or scientific notation
doubleDec = 'formatDouble' 'generic'
doubleHexFixed :: Double -> BuilderEncode an IEEE Double using 16 nibbles.
floatDec :: Float -> BuilderReturns a rendered Float. Matches `show` in displaying in standard or scientific notation
floatDec = 'formatFloat' 'generic'
floatHexFixed :: Float -> BuilderEncode an IEEE Float using 8 nibbles.
int16Dec :: Int16 -> BuilderDecimal encoding of an Int16 using the ASCII digits.
int16HexFixed :: Int16 -> BuilderEncode a Int16 using 4 nibbles.
int32Dec :: Int32 -> BuilderDecimal encoding of an Int32 using the ASCII digits.
int32HexFixed :: Int32 -> BuilderEncode a Int32 using 8 nibbles.
int64Dec :: Int64 -> BuilderDecimal encoding of an Int64 using the ASCII digits.
int64HexFixed :: Int64 -> BuilderEncode a Int64 using 16 nibbles.
int8Dec :: Int8 -> BuilderDecimal encoding of an Int8 using the ASCII digits.
e.g.
toLazyByteString (int8Dec 42) = "42"
toLazyByteString (int8Dec (-1)) = "-1"int8HexFixed :: Int8 -> BuilderEncode a Int8 using 2 nibbles (hexadecimal digits).
intDec :: Int -> BuilderDecimal encoding of an Int using the ASCII digits.
integerDec :: Integer -> BuilderDecimal encoding of an Integer using the ASCII digits.
lazyByteStringHex :: L.LazyByteString -> BuilderEncode each byte of a LazyByteString using its fixed-width hex encoding.
word16Dec :: Word16 -> BuilderDecimal encoding of a Word16 using the ASCII digits.
word16Hex :: Word16 -> BuilderShortest hexadecimal encoding of a Word16 using lower-case characters.
word16HexFixed :: Word16 -> BuilderEncode a Word16 using 4 nibbles.
word32Dec :: Word32 -> BuilderDecimal encoding of a Word32 using the ASCII digits.
word32Hex :: Word32 -> BuilderShortest hexadecimal encoding of a Word32 using lower-case characters.
word32HexFixed :: Word32 -> BuilderEncode a Word32 using 8 nibbles.
word64Dec :: Word64 -> BuilderDecimal encoding of a Word64 using the ASCII digits.
word64Hex :: Word64 -> BuilderShortest hexadecimal encoding of a Word64 using lower-case characters.
word64HexFixed :: Word64 -> BuilderEncode a Word64 using 16 nibbles.
word8Dec :: Word8 -> BuilderDecimal encoding of a Word8 using the ASCII digits.
word8Hex :: Word8 -> BuilderShortest hexadecimal encoding of a Word8 using lower-case characters.
word8HexFixed :: Word8 -> BuilderEncode a Word8 using 2 nibbles (hexadecimal digits).
wordDec :: Word -> BuilderDecimal encoding of a Word using the ASCII digits.
wordHex :: Word -> BuilderShortest hexadecimal encoding of a Word using lower-case characters.
Data.ByteString.Builder.RealFloat bytestring-0.12.2.0 · 7 declarationsformatDouble :: FloatFormat -> Double -> BuilderReturns a rendered Double. Returns the 'shortest' representation in scientific notation and takes an optional precision argument in standard notation. Also see `doubleDec`.
With standard notation, the precision argument is used to truncate (or extend with 0s) the 'shortest' rendered Float. The 'default precision' does no such modifications and will return as many decimal places as the representation demands.
e.g
>>> formatDouble (standard 1) 1.2345e-2
"0.0"
>>> formatDouble (standard 10) 1.2345e-2
"0.0123450000"
>>> formatDouble standardDefaultPrecision 1.2345e-2
"0.01234"
>>> formatDouble scientific 12.345
"1.2345e1"
>>> formatDouble generic 12.345
"12.345"formatFloat :: FloatFormat -> Float -> BuilderReturns a rendered Float. Returns the 'shortest' representation in scientific notation and takes an optional precision argument in standard notation. Also see `floatDec`.
With standard notation, the precision argument is used to truncate (or extend with 0s) the 'shortest' rendered Float. The 'default precision' does no such modifications and will return as many decimal places as the representation demands.
e.g
>>> formatFloat (standard 1) 1.2345e-2
"0.0"
>>> formatFloat (standard 10) 1.2345e-2
"0.0123450000"
>>> formatFloat standardDefaultPrecision 1.2345e-2
"0.01234"
>>> formatFloat scientific 12.345
"1.2345e1"
>>> formatFloat generic 12.345
"12.345"generic :: FloatFormatStandard or scientific notation depending on the exponent. Matches `show`
scientific :: FloatFormatScientific notation with 'default precision' (decimal places matching `show`)
standard :: Int -> FloatFormatStandard notation with `n` decimal places
standardDefaultPrecision :: FloatFormatStandard notation with the 'default precision' (decimal places matching `show`)
data FloatFormatFormat type for use with `formatFloat` and `formatDouble`.
instance IsString Builderinstance Show BuilderCommunity contribution. The service has not verified this documentation. The generator reported 11 diagnostics.
Target: Linux on x86-64. Some inferred types and resolved links are not available.
Generator: aihc-haddock-aa6debbd-web-3
Metadata snapshot: 35f1f56d3dce7f0c7af322ec05f2fba4d1a04564930537462a4ba21b64b756fa
Open the dependency plan · Download the documentation model