| [ Web Proxy ] |
| Viewing: https://arrow.apache.org/docs/python/generated/../integration/../generated/../../cpp/api/io.html | [Back] [Original] |
Subclassed by arrow::io::InputStream, arrow::io::OutputStream
Public Functions
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Close the stream asynchronously.
By default, this will just submit the synchronous Close() to the default I/O thread pool. Subclasses may implement this in a more efficient manner.
Close the stream abruptly.
This method does not guarantee that any pending data is flushed. It merely releases any underlying resource used by the stream for its operation.
After Abort() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Subclassed by arrow::io::InputStream
Public Functions
Read data from current file position.
Read at most nbytes from the current file position into out. The number of bytes read is returned.
Read data from current file position.
Read at most nbytes from the current file position. Less bytes may be read if EOF is reached. This method updates the current file position.
In some cases (e.g. a memory-mapped file), this method may avoid a memory copy.
EXPERIMENTAL: The IOContext associated with this file.
By default, this is the same as default_io_context(), but it may be overridden by subclasses.
Subclassed by arrow::io::RandomAccessFile, arrow::io::WritableFile
Subclassed by arrow::io::OutputStream
Public Functions
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
Write the given data to the stream.
Since the Buffer owns its memory, this method can avoid a copy if buffering is required. See Write(const void*, int64_t) for details.
Subclassed by arrow::io::SlowInputStreamBase< InputStream >, arrow::io::internal::InputStreamConcurrencyWrapper< BufferedInputStream >, arrow::io::internal::InputStreamConcurrencyWrapper< CompressedInputStream >, arrow::io::RandomAccessFile, arrow::io::StdinStream, arrow::io::TransformInputStream, arrow::io::internal::InputStreamConcurrencyWrapper< Derived >
Public Functions
Advance or skip stream indicated number of bytes.
nbytes [in] the number to move forward
Return zero-copy string_view to upcoming bytes.
Do not modify the stream position. The view becomes invalid after any operation on the stream. May trigger buffering if the requested size is larger than the number of buffered bytes.
May return NotImplemented on streams that dont support it.
nbytes [in] the maximum number of bytes to see
Return true if InputStream is capable of zero copy Buffer reads.
Zero copy reads imply the use of Buffer-returning Read() overloads.
Read and return stream metadata.
If the stream implementation doesnt support metadata, empty metadata is returned. Note that it is allowed to return a null pointer rather than an allocated empty metadata.
Read stream metadata asynchronously.
Subclassed by arrow::io::SlowInputStreamBase< RandomAccessFile >, arrow::io::internal::RandomAccessFileConcurrencyWrapper< CudaBufferReader >, arrow::io::internal::RandomAccessFileConcurrencyWrapper< BufferReader >, arrow::io::internal::RandomAccessFileConcurrencyWrapper< ReadableFile >, arrow::io::HdfsReadableFile, arrow::io::ReadWriteFileInterface, arrow::io::internal::RandomAccessFileConcurrencyWrapper< Derived >
Public Functions
Necessary because we hold a std::unique_ptr.
Return the total file size in bytes.
This method does not read or move the current file position, so is safe to call concurrently with e.g. ReadAt().
Read data from given file position.
At most nbytes bytes are read. The number of bytes read is returned. If allow_short_read is true, the number of bytes read can be less than nbytes if EOF is reached, otherwise an error is returned.
This method can be safely called from multiple threads concurrently. It is unspecified whether this method updates the file position or not.
The default RandomAccessFile-provided implementation uses Seek() and Read(), but subclasses may override it with a more efficient implementation that doesnt depend on implicit file positioning.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
allow_short_read [in] Whether to allow reading less than nbytes
out [out] The buffer to read bytes into
The number of bytes read, or an error
Read data from given file position.
Like ReadAt(position, nbytes, allow_short_read, out) with allow_short_read set to true.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
out [out] The buffer to read bytes into
The number of bytes read, or an error
Read data from given file position.
At most nbytes bytes are read. If allow_short_read is true, the number of bytes read can be less than nbytes if EOF is reached, otherwise an error is returned.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
allow_short_read [in] Whether to allow reading less than nbytes
A buffer containing the bytes read, or an error
Read data from given file position.
Like ReadAt(position, nbytes, allow_short_read) with allow_short_read set to true.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
A buffer containing the bytes read, or an error
EXPERIMENTAL: Read data asynchronously.
EXPERIMENTAL: Read data asynchronously, using the files IOContext.
EXPERIMENTAL: Explicit multi-read.
Request multiple reads at once
The underlying filesystem may optimize these reads by coalescing small reads into large reads or by breaking up large reads into multiple parallel smaller reads. The reads should be issued in parallel if it makes sense for the filesystem.
One future will be returned for each input read range. Multiple returned futures may correspond to a single read. Or, a single returned future may be a combined result of several individual reads.
ranges [in] The ranges to read
A future that will complete with the data from the requested range is available
EXPERIMENTAL: Explicit multi-read, using the files IOContext.
EXPERIMENTAL: Inform that the given ranges may be read soon.
Some implementations might arrange to prefetch some of the data. However, no guarantee is made and the default implementation does nothing. For robust prefetching, use ReadAt() or ReadAsync().
Public Static Functions
Create an isolated InputStream that reads a segment of a RandomAccessFile.
Multiple such stream can be created and used independently without interference
file [in] a file instance
file_offset [in] the starting position in the file
nbytes [in] the extent of bytes to read. The file should have sufficient bytes available
Subclassed by arrow::io::BufferOutputStream, arrow::io::BufferedOutputStream, arrow::io::CompressedOutputStream, arrow::io::FileOutputStream, arrow::io::HdfsOutputStream, arrow::io::MockOutputStream, arrow::io::StderrStream, arrow::io::StdoutStream, arrow::io::WritableFile
Subclassed by arrow::io::MemoryMappedFile
Random access zero-copy reads on an arrow::Buffer.
Public Functions
Instantiate from std::shared_ptr<Buffer>.
This is a zero-copy constructor.
Return whether the stream is closed.
Return true if InputStream is capable of zero copy Buffer reads.
Zero copy reads imply the use of Buffer-returning Read() overloads.
EXPERIMENTAL: Read data asynchronously.
EXPERIMENTAL: Inform that the given ranges may be read soon.
Some implementations might arrange to prefetch some of the data. However, no guarantee is made and the default implementation does nothing. For robust prefetching, use ReadAt() or ReadAsync().
Public Static Functions
Instantiate from std::string. Owns data.
A helper class to track the size of allocations.
Writes to this stream do not copy or retain any data, they just bump a size counter that can be later used to know exactly which data size needs to be allocated for actual writing.
Public Functions
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
An output stream that writes to a resizable buffer.
Public Functions
Close the stream, preserving the buffer (retrieve it with Finish()).
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
Initialize state of OutputStream with newly allocated memory and set position to 0.
initial_capacity [in] the starting allocated capacity
pool [inout] the memory pool to use for allocations
Public Static Functions
Create in-memory output stream with indicated capacity using a memory pool.
initial_capacity [in] the initial allocated internal capacity of the OutputStream
pool [inout] a MemoryPool to use for allocations
the created stream
An output stream that writes into a fixed-size mutable buffer.
Public Functions
Input buffer must be mutable, will abort if not.
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
An operating system file open in read-only mode.
Reads through this implementation are unbuffered. If many small reads need to be issued, it is recommended to use a buffering layer for good performance.
Public Functions
Return whether the stream is closed.
EXPERIMENTAL: Inform that the given ranges may be read soon.
Some implementations might arrange to prefetch some of the data. However, no guarantee is made and the default implementation does nothing. For robust prefetching, use ReadAt() or ReadAsync().
Public Static Functions
Open a local file for reading.
path [in] with UTF8 encoding
pool [in] a MemoryPool for memory allocations
ReadableFile instance
Open a local file for reading.
The file descriptor becomes owned by the ReadableFile, and will be closed on Close() or destruction.
fd [in] file descriptor
pool [in] a MemoryPool for memory allocations
ReadableFile instance
An operating system file open in write-only mode.
Public Functions
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
Public Static Functions
Open a local file for writing, truncating any existing file.
When opening a new file, any existing file with the indicated path is truncated to 0 bytes, deleting any existing data
path [in] with UTF8 encoding
append [in] append to existing file, otherwise truncate to 0 bytes
an open FileOutputStream
Open a file descriptor for writing.
The underlying file isnt truncated.
The file descriptor becomes owned by the
OutputStream, and will be closed on Close() or destruction.fd [in] file descriptor
an open FileOutputStream
A file interface that uses memory-mapped files for memory interactions.
This implementation supports zero-copy reads. The same class is used for both reading and writing.
If opening a file in a writable mode, it is not truncated first as with FileOutputStream.
Public Functions
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Read data from current file position.
Read at most nbytes from the current file position into out. The number of bytes read is returned.
Read data from current file position.
Read at most nbytes from the current file position. Less bytes may be read if EOF is reached. This method updates the current file position.
In some cases (e.g. a memory-mapped file), this method may avoid a memory copy.
Read data from given file position.
Like ReadAt(position, nbytes, allow_short_read) with allow_short_read set to true.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
A buffer containing the bytes read, or an error
Read data from given file position.
Like ReadAt(position, nbytes, allow_short_read, out) with allow_short_read set to true.
position [in] Where to read bytes from
nbytes [in] The number of bytes to read
out [out] The buffer to read bytes into
The number of bytes read, or an error
EXPERIMENTAL: Inform that the given ranges may be read soon.
Some implementations might arrange to prefetch some of the data. However, no guarantee is made and the default implementation does nothing. For robust prefetching, use ReadAt() or ReadAsync().
Return true if InputStream is capable of zero copy Buffer reads.
Zero copy reads imply the use of Buffer-returning Read() overloads.
Write data at the current position in the file. Thread-safe.
Public Static Functions
Create new file with indicated size, return in read/write mode.
An InputStream that performs buffered reads from an unbuffered InputStream, which can mitigate the overhead of many small reads in some cases.
Public Functions
Resize internal read buffer; calls to Read() will read at least this many bytes from the raw InputStream if possible.
new_buffer_size [in] the new read buffer size
Return the number of remaining bytes in the read buffer.
Return the current size of the internal buffer.
Release the raw InputStream.
Any data buffered will be discarded. Further operations on this object are invalid
raw the underlying InputStream
Return the unbuffered InputStream.
Return whether the stream is closed.
Read and return stream metadata.
If the stream implementation doesnt support metadata, empty metadata is returned. Note that it is allowed to return a null pointer rather than an allocated empty metadata.
Read stream metadata asynchronously.
Public Static Functions
Create a BufferedInputStream from a raw InputStream.
buffer_size [in] the size of the temporary read buffer
pool [in] a MemoryPool to use for allocations
raw [in] a raw InputStream
raw_read_bound [in] a bound on the maximum number of bytes to read from the raw input stream. The default -1 indicates that it is unbounded
the created BufferedInputStream
Public Functions
Resize internal buffer.
new_buffer_size [in] the new buffer size
Return the current size of the internal buffer.
Return the number of remaining bytes that have not been flushed to the raw OutputStream.
Flush any buffered writes and release the raw OutputStream.
Further operations on this object are invalid
the underlying OutputStream
Close the buffered output stream.
This implicitly closes the underlying raw output stream.
Close the stream abruptly.
This method does not guarantee that any pending data is flushed. It merely releases any underlying resource used by the stream for its operation.
After Abort() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
Write the given data to the stream.
Since the Buffer owns its memory, this method can avoid a copy if buffering is required. See Write(const void*, int64_t) for details.
Return the underlying raw output stream.
Public Static Functions
Create a buffered output stream wrapping the given output stream.
buffer_size [in] the size of the temporary write buffer
pool [in] a MemoryPool to use for allocations
raw [in] another OutputStream
the created BufferedOutputStream
Public Functions
Return whether the stream is closed.
Read and return stream metadata.
If the stream implementation doesnt support metadata, empty metadata is returned. Note that it is allowed to return a null pointer rather than an allocated empty metadata.
Read stream metadata asynchronously.
Return the underlying raw input stream.
Public Static Functions
Create a compressed input stream wrapping the given input stream.
The codec must be capable of streaming decompression. Some codecs, like Snappy, are not able to do so.
Public Functions
Close the compressed output stream.
This implicitly closes the underlying raw output stream.
Close the stream abruptly.
This method does not guarantee that any pending data is flushed. It merely releases any underlying resource used by the stream for its operation.
After Abort() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Write the given data to the stream.
This method always processes the bytes in full. Depending on the semantics of the stream, the data may be written out immediately, held in a buffer, or written asynchronously. In the case where the stream buffers the data, it will be copied. To avoid potentially large copies, use the Write variant that takes an owned Buffer.
Return the underlying raw output stream.
Public Static Functions
Create a compressed output stream wrapping the given output stream.
The codec must be capable of streaming compression. Some codecs, like Snappy, are not able to do so.
Public Functions
Close the stream cleanly.
For writable streams, this will attempt to flush any pending data before releasing the underlying resource.
After Close() is called, closed() returns true and the stream is not available for further operations.
Close the stream abruptly.
This method does not guarantee that any pending data is flushed. It merely releases any underlying resource used by the stream for its operation.
After Abort() is called, closed() returns true and the stream is not available for further operations.
Return whether the stream is closed.
Read data from current file position.
Read at most nbytes from the current file position into out. The number of bytes read is returned.
Read data from current file position.
Read at most nbytes from the current file position. Less bytes may be read if EOF is reached. This method updates the current file position.
In some cases (e.g. a memory-mapped file), this method may avoid a memory copy.
Read and return stream metadata.
If the stream implementation doesnt support metadata, empty metadata is returned. Note that it is allowed to return a null pointer rather than an allocated empty metadata.
Read stream metadata asynchronously.
| Web Proxy Viewer | New URL | Original Page |
![[Apache Arrow v25.0.1 - Home]](https://arrow.apache.org/docs/python/generated/../integration/../generated/../../cpp/api/../../_static/arrow.png){kind=link}
![[Apache Arrow v25.0.1 - Home]](https://arrow.apache.org/docs/python/generated/../integration/../generated/../../cpp/api/../../_static/arrow-dark.png){kind=link}