data Buffer
A Buffer together with the BufferRange of free bytes. The filled
space starts at offset 0 and ends at the first free byte.
*Warning:* this module is internal. If you find that you need it then please contact the maintainers and explain what you are trying to do and discuss what you would need in the public API. It is important that you do this as the module may not be exposed at all in future releases.
Core types and functions for the Builder monoid and its generalization,
the Put monad.
The design of the Builder monoid is optimized such that
buffers of arbitrary size can be filled as efficiently as possible and
sequencing of Builders is as cheap as possible.
We achieve (1) by completely handing over control over writing to the buffer
to the BuildStep implementing the Builder. This BuildStep is just told
the start and the end of the buffer (represented as a BufferRange). Then,
the BuildStep can write to as big a prefix of this BufferRange in any
way it desires. If the BuildStep is done, the BufferRange is full, or a
long sequence of bytes should be inserted directly, then the BuildStep
signals this to its caller using a BuildSignal.
We achieve (2) by requiring that every Builder is implemented by a
BuildStep that takes a continuation BuildStep, which it calls with the
updated BufferRange after it is done. Therefore, only two pointers have
to be passed in a function call to implement concatenation of Builders.
Moreover, many Builders are completely inlined, which enables the compiler
to sequence them without a function call and with no boxing at all.
This design gives the implementation of a Builder full access to the IO
monad. Therefore, utmost care has to be taken to not overwrite anything
outside the given BufferRanges. Moreover, further care has to be taken to
ensure that Builders and Puts are referentially transparent. See the
comments of the builder and put functions for further information.
Note that there are no safety belts at all, when implementing a Builder
using an IO action: you are writing code that might enable the next
buffer-overflow attack on a Haskell server!
data BufferA Buffer together with the BufferRange of free bytes. The filled
space starts at offset 0 and ends at the first free byte.
Buffer :: (ForeignPtr Word8) -> BufferRange -> Bufferdata BufferRangeA range of bytes in a buffer represented by the pointer to the first byte of the range and the pointer to the first byte after the range.
BufferRange :: (Ptr Word8) -> (Ptr Word8) -> BufferRangenewBuffer :: Int -> IO BufferAllocate a new buffer of the given size.
bufferSize :: Buffer -> IntCombined size of the filled and free space in the buffer.
byteStringFromBuffer :: Buffer -> S.StrictByteStringConvert the filled part of a Buffer to a StrictByteString.
data ChunkIOStream aA stream of chunks that are constructed in the IO monad.
This datatype serves as the common interface for the buffer-by-buffer
execution of a BuildStep by buildStepToCIOS. Typical users of this
interface are ciosToLazyByteString or iteratee-style libraries like
enumerator.
Finished :: Buffer -> a -> ChunkIOStream aaThe partially filled last buffer together with the result.
Yield1 :: S.StrictByteString -> (IO (ChunkIOStream a)) -> ChunkIOStream a(IO (ChunkIOStream a))Yield a non-empty StrictByteString.
buildStepToCIOS :: forall a. AllocationStrategy -> BuildStep a -> IO (ChunkIOStream a)Convert a BuildStep to a ChunkIOStream stream by executing it on
Buffers allocated according to the given AllocationStrategy.
BuildStep to execute
AllocationStrategyBuffer allocation strategy to use
ciosUnitToLazyByteString :: AllocationStrategy -> L.LazyByteString -> ChunkIOStream () -> L.LazyByteStringConvert a to a ChunkIOStream ()LazyByteString using
unsafeDupablePerformIO.
ciosToLazyByteString :: AllocationStrategy -> (a -> (b, L.LazyByteString)) -> ChunkIOStream a -> (b, L.LazyByteString)Convert a ChunkIOStream to a lazy tuple of the result and the written
LazyByteString using unsafeDupablePerformIO.
data BuildSignal aBuildSignals abstract signals to the caller of a BuildStep. There are
three signals: done, bufferFull, or 'insertChunks signals
type BuildStep a = BufferRange -> IO (BuildSignal a)BuildSteps may be called *multiple times* and they must not rise an
async. exception.
finalBuildStep :: BuildStep ()The final build step that returns the done signal.
done :: Ptr Word8 -> a -> BuildSignal aSignal that the current BuildStep is done and has computed a value.
Next free byte in current BufferRange
aComputed value
bufferFull :: Int -> Ptr Word8 -> BuildStep a -> BuildSignal aSignal that the current buffer is full.
Next free byte in current BufferRange.
BuildStep to run on the next BufferRange. This BuildStep
may assume that it is called with a BufferRange of at least the
required minimal size; i.e., the caller of this BuildStep must
guarantee this.
IntMinimal size of next BufferRange.
insertChunk :: Ptr Word8 -> S.StrictByteString -> BuildStep a -> BuildSignal aSignal that a StrictByteString chunk should be inserted directly.
Next free byte in current BufferRange
BuildStep to run on next BufferRange
S.StrictByteStringChunk to insert.
fillWithBuildStep :: BuildStep a -> (Ptr Word8 -> a -> IO b) -> (Ptr Word8 -> Int -> BuildStep a -> IO b) -> (Ptr Word8 -> S.StrictByteString -> BuildStep a -> IO b) -> BufferRange -> IO bFill a BufferRange using a BuildStep.
Build step to use for filling the BufferRange.
Value computed while filling this BufferRange.
(Ptr Word8 -> a -> IO b)Handling the done signal
(Ptr Word8 -> Int -> BuildStep a -> IO b)Handling the bufferFull signal
(Ptr Word8 -> S.StrictByteString -> BuildStep a -> IO b)Handling the insertChunk signal
BufferRangeBuffer range to fill.
newtype BuilderBuilders denote sequences of bytes.
They are Monoids where
mempty is the zero-length sequence and
mappend is concatenation, which runs in O(1).
builder :: (forall r. BuildStep r -> BuildStep r) -> BuilderConstruct a Builder. In contrast to BuildSteps, Builders are
referentially transparent.
(forall r. BuildStep r -> BuildStep r)A function that fills a BufferRange, calls the continuation with
the updated BufferRange once its done, and signals its caller how
to proceed using done, bufferFull, or insertChunk.
This function must be referentially transparent; i.e., calling it
multiple times with equally sized BufferRanges must result in the
same sequence of bytes being written. If you need mutable state,
then you must allocate it anew upon each call of this function.
Moreover, this function must call the continuation once its done.
Otherwise, concatenation of Builders does not work. Finally, this
function must write to all bytes that it claims it has written.
Otherwise, the resulting Builder is not guaranteed to be
referentially transparent and sensitive data might leak.
runBuilder :: Builder -> BuildStep ()Run a Builder with the finalBuildStep.
BuildStep that writes the byte stream of this
Builder and signals done upon completion.
BuilderBuilder to run
runBuilderWith :: Builder -> BuildStep a -> BuildStep aRun a Builder.
Continuation BuildStep
BuilderBuilder to run
empty :: BuilderThe Builder denoting a zero-length sequence of bytes. This function is
only exported for use in rewriting rules. Use mempty otherwise.
append :: Builder -> Builder -> BuilderConcatenate two Builders. This function is only exported for use in rewriting
rules. Use mappend otherwise.
flush :: BuilderFlush the current buffer. This introduces a chunk boundary.
ensureFree :: Int -> Builder ensures that there are at least ensureFree nn free bytes
for the following Builder.
byteStringCopy :: S.StrictByteString -> BuilderConstruct a Builder that copies the StrictByteString.
Use this function to create Builders from smallish (<= 4kb)
StrictByteStrings or if you need to guarantee that the StrictByteString is not
shared with the chunks generated by the Builder.
byteStringInsert :: S.StrictByteString -> BuilderConstruct a Builder that always inserts the StrictByteString
directly as a chunk.
This implies flushing the output buffer, even if it contains just
a single byte. You should therefore use byteStringInsert only for large
(> 8kb) StrictByteStrings. Otherwise, the generated chunks are too
fragmented to be processed efficiently afterwards.
byteStringThreshold :: Int -> S.StrictByteString -> BuilderConstruct a Builder that copies the StrictByteStrings, if it is
smaller than the treshold, and inserts it directly otherwise.
For example, byteStringThreshold 1024 copies StrictByteStrings whose size
is less or equal to 1kb, and inserts them directly otherwise. This implies
that the average chunk-size of the generated LazyByteString may be as
low as 513 bytes, as there could always be just a single byte between the
directly inserted 1025 byte, StrictByteStrings.
lazyByteStringCopy :: L.LazyByteString -> BuilderConstruct a Builder that copies the LazyByteString.
lazyByteStringInsert :: L.LazyByteString -> BuilderConstruct a Builder that inserts all chunks of the LazyByteString
directly.
lazyByteStringThreshold :: Int -> L.LazyByteString -> BuilderConstruct a Builder that uses the thresholding strategy of byteStringThreshold
for each chunk of the LazyByteString.
shortByteString :: Sh.ShortByteString -> BuilderConstruct a Builder that copies the ShortByteString.
maximalCopySize :: IntThe maximal size of a StrictByteString that is copied.
2 * to guarantee that on average a chunk is of
smallChunkSizesmallChunkSize.
byteString :: S.StrictByteString -> BuilderCreate a Builder denoting the same sequence of bytes as a
StrictByteString.
The Builder inserts large StrictByteStrings directly, but copies small ones
to ensure that the generated chunks are large on average.
lazyByteString :: L.LazyByteString -> BuilderCreate a Builder denoting the same sequence of bytes as a lazy
LazyByteString.
The Builder inserts large chunks of the LazyByteString directly,
but copies small ones to ensure that the generated chunks are large on
average.
toLazyByteString :: Builder -> L.LazyByteStringExecute a Builder and return the generated chunks as a LazyByteString.
The work is performed lazy, i.e., only when a chunk of the LazyByteString
is forced.
toLazyByteStringWith :: AllocationStrategy -> L.LazyByteString -> Builder -> L.LazyByteStringHeavy inlining. Execute a Builder with custom execution parameters.
This function is inlined despite its heavy code-size to allow fusing with
the allocation strategy. For example, the default Builder execution
function toLazyByteString is defined as follows.
{-\# NOINLINE toLazyByteString \#-}
toLazyByteString =
toLazyByteStringWith ('safeStrategy' 'L.smallChunkSize' 'L.defaultChunkSize') L.Empty
where L.Empty is the zero-length LazyByteString.
In most cases, the parameters used by toLazyByteString give good
performance. A sub-performing case of toLazyByteString is executing short
(<128 bytes) Builders. In this case, the allocation overhead for the first
4kb buffer and the trimming cost dominate the cost of executing the
Builder. You can avoid this problem using
toLazyByteStringWith (safeStrategy 128 smallChunkSize) L.EmptyThis reduces the allocation and trimming overhead, as all generated
LazyByteStrings fit into the first buffer and there is no trimming
required, if more than 64 bytes and less than 128 bytes are written.
AllocationStrategyBuffer allocation strategy to use
L.LazyByteStringLazyByteString to use as the tail of the generated lazy
LazyByteString
BuilderBuilder to execute
L.LazyByteStringResulting LazyByteString
data AllocationStrategyA buffer allocation strategy for executing Builders.
safeStrategy :: Int -> Int -> AllocationStrategyUse this strategy for generating LazyByteStrings whose chunks are
likely to survive one garbage collection. This strategy trims buffers
that are filled less than half in order to avoid spilling too much memory.
IntSize of first buffer
IntSize of successive buffers
AllocationStrategyAn allocation strategy that guarantees that at least half of the allocated memory is used for live data
untrimmedStrategy :: Int -> Int -> AllocationStrategyUse this strategy for generating LazyByteStrings whose chunks are
discarded right after they are generated. For example, if you just generate
them to write them to a network socket.
IntSize of the first buffer
IntSize of successive buffers
AllocationStrategyAn allocation strategy that does not trim any of the filled buffers before converting it to a chunk
customStrategy :: (Maybe (Buffer, Int) -> IO Buffer) -> Int -> (Int -> Int -> Bool) -> AllocationStrategyCreate a custom allocation strategy. See the code for safeStrategy and
untrimmedStrategy for examples.
(Maybe (Buffer, Int) -> IO Buffer)Buffer allocation function.
If Nothing is given, then a new first buffer should be allocated.
If is given, then a buffer with minimal
size Just (oldBuf, minSize)minSize must be returned. The strategy may reuse oldBuf only if
oldBuf is large enough and the consumer can guarantee that this will
not result in a violation of referential transparency.
Warning: for multithreaded programs, it is generally unsafe to reuse
buffers when using the consumers of Builder in this package. For
example, if toLazyByteStringWith is called with an
AllocationStrategy that reuses buffers, evaluating the result by
multiple threads simultaneously may lead to corrupted output.
IntDefault buffer size.
(Int -> Int -> Bool)A predicate trim used allocated returning True, if the buffer
should be trimmed before it is returned.
smallChunkSize :: IntThe recommended chunk size. Currently set to 4k, less the memory management overhead
defaultChunkSize :: IntThe chunk size used for I/O. Currently set to 32k, less the memory management overhead
chunkOverhead :: IntThe memory management overhead. Currently this is tuned for GHC only.
newtype Put aA Put action denotes a computation of a value that writes a stream of
bytes as a side-effect. Puts are strict in their side-effect; i.e., the
stream of bytes will always be written before the computed value is
returned.
Puts are a generalization of Builders. The typical use case is the
implementation of an encoding that might fail (e.g., an interface to the
zlib
compression library or the conversion from Base64 encoded data to
8-bit data). For a Builder, the only way to handle and report such a
failure is ignore it or call error. In contrast, Put actions are
expressive enough to allow reporting and handling such a failure in a pure
fashion.
actions are isomorphic to Put ()Builders. The functions putBuilder
and fromPut convert between these two types. Where possible, you should
use Builders, as sequencing them is slightly cheaper than sequencing
Puts because they do not carry around a computed value.
put :: (forall r. (a -> BuildStep r) -> BuildStep r) -> Put aConstruct a Put action. In contrast to BuildSteps, Puts are
referentially transparent in the sense that sequencing the same Put
multiple times yields every time the same value with the same side-effect.
(forall r. (a -> BuildStep r) -> BuildStep r)A function that fills a BufferRange, calls the continuation with
the updated BufferRange and its computed value once its done, and
signals its caller how to proceed using done, bufferFull, or
insertChunk signals.
This function must be referentially transparent; i.e., calling it
multiple times with equally sized BufferRanges must result in the
same sequence of bytes being written and the same value being
computed. If you need mutable state, then you must allocate it anew
upon each call of this function. Moreover, this function must call
the continuation once its done. Otherwise, monadic sequencing of
Puts does not work. Finally, this function must write to all bytes
that it claims it has written. Otherwise, the resulting Put is
not guaranteed to be referentially transparent and sensitive data
might leak.
runPut :: Put a -> BuildStep aRun a Put.
Put to run
BuildStep that first writes the byte stream of
this Put and then yields the computed value using
the done signal.
putToLazyByteString :: Put a -> (a, L.LazyByteString)Execute a Put and return the computed result and the bytes
written during the computation as a LazyByteString.
This function is strict in the computed result and lazy in the writing of the bytes. For example, given
infinitePut = sequence_ (repeat (putBuilder (word8 1))) >> return 0
evaluating the expression
fst $ putToLazyByteString infinitePut
does not terminate, while evaluating the expression
L.head $ snd $ putToLazyByteString infinitePut
does terminate and yields the value 1 :: Word8.
An illustrative example for these strictness properties is the implementation of Base64 decoding (http://en.wikipedia.org/wiki/Base64).
type DecodingState = ...
decodeBase64 :: 'S.StrictByteString' -> DecodingState -> 'Put' (Maybe DecodingState)
decodeBase64 = ...
The above function takes a StrictByteString supposed to represent
Base64 encoded data and the current decoding state.
It writes the decoded bytes as the side-effect of the Put and returns the
new decoding state, if the decoding of all data in the StrictByteString was
successful. The checking if the StrictByteString represents Base64
encoded data and the actual decoding are fused. This makes the common case,
where all data represents Base64 encoded data, more efficient. It also
implies that all data must be decoded before the final decoding
state can be returned. Puts are intended for implementing such fused
checking and decoding/encoding, which is reflected in their strictness
properties.
Put to execute
(a, L.LazyByteString)Result and LazyByteString
written as its side-effect
putToLazyByteStringWith :: AllocationStrategy -> (a -> (b, L.LazyByteString)) -> Put a -> (b, L.LazyByteString)Execute a Put with a buffer-allocation strategy and a continuation. For
example, putToLazyByteString is implemented as follows.
putToLazyByteString = 'putToLazyByteStringWith'
('safeStrategy' 'L.smallChunkSize' 'L.defaultChunkSize') (\x -> (x, L.empty))
Put to execute
AllocationStrategyBuffer allocation strategy to use
(a -> (b, L.LazyByteString))Continuation to use for computing the final result and the tail of its side-effect (the written bytes).
(b, L.LazyByteString)Resulting LazyByteString
hPut :: forall a. Handle -> Put a -> IO aRun a Put action redirecting the produced output to a Handle.
The output is buffered using the Handles associated buffer. If this
buffer is too small to execute one step of the Put action, then
it is replaced with a large enough buffer.
putBuilder :: Builder -> Put ()Run a Builder as a side-effect of a action.Put ()
fromPut :: Put () -> BuilderConvert a action to a Put ()Builder.
instance NFData BufferRangeinstance NFData BufferLike the NFData instance for StrictByteString,
this does not force the ForeignPtrContents field
of the underlying ForeignPtr.
instance Semigroup Builderinstance Monoid Builderinstance IsList BuilderFor long or infinite lists use fromList because it uses LazyByteString otherwise use fromListN which uses StrictByteString.
instance Functor Putinstance Applicative Putinstance Monad PutCommunity contribution. The service has not verified this documentation. The generator reported 11 diagnostics.
Target: Linux on x86-64. Some inferred types and resolved links are not available.
Generator: aihc-haddock-aa6debbd-web-3
Metadata snapshot: 35f1f56d3dce7f0c7af322ec05f2fba4d1a04564930537462a4ba21b64b756fa
Open the dependency plan · Download the documentation model