IO#

Interface#

enum class paimon::ByteOrder : int8_t#

Values:

enumerator PAIMON_BIG_ENDIAN#
enumerator PAIMON_LITTLE_ENDIAN#
constexpr ByteOrder paimon::SystemByteOrder()#

Get the byte order of the system.

class BufferedInputStream : public paimon::InputStream#

A buffered input stream that wraps another InputStream to provide buffering capabilities.

BufferedInputStream improves I/O performance by reducing the number of system calls through internal buffering. It reads data from the underlying stream in larger chunks and serves subsequent read requests from the internal buffer when possible.

Public Functions

BufferedInputStream(const std::shared_ptr<InputStream> &in, int32_t buffer_size, MemoryPool *pool)#

Creates a new buffered input stream that wraps the provided input stream.

The buffer is allocated from the specified memory pool.

Parameters:

  • in – The underlying input stream to wrap.

  • buffer_size – Size of the internal buffer in bytes.

  • pool – Memory pool for buffer allocation.

~BufferedInputStream() noexcept override#
virtual Status Seek(int64_t offset, SeekOrigin origin) override#

Seek to a specified position in the input stream.

Parameters:
Returns:

Status indicating success (OK) or failure with appropriate error information.

virtual Result<int64_t> GetPos() const override#

Get the current position in the input stream.

Returns:

Current position in the input stream.

Returns:

IOError returned if an I/O error occurred in the underlying stream. implementation while accessing the stream’s position.

virtual Result<int32_t> Read(char *buffer, uint32_t size) override#

Read data from the current position in the stream.

Note

The stream position advances by the number of bytes actually read.

Parameters:

  • buffer – [out] Pointer to the buffer where read data will be stored.

  • size – Maximum number of bytes to read.

Returns:

Result containing the actual number of bytes read on success, or an error status on failure.

virtual Result<int32_t> Read(char *buffer, uint32_t size, uint64_t offset) override#

Read data from given position in the stream.

Read with offset performs like pread() function, which will not change the position in the input stream.

Parameters:

  • buffer – [out] The buffer to store the read content.

  • size – The number of bytes to read.

  • offset – The position in the stream to read from.

virtual void ReadAsync(char *buffer, uint32_t size, uint64_t offset, std::function<void(Status)> &&callback) override#

Asynchronously read data from the input stream.

This function initiates an asynchronous read operation. The specified number of bytes will be read from the stream starting at the given offset and stored in the provided buffer. Once the read operation is complete, the provided callback function will be invoked with the status of the read operation.

Parameters:

  • buffer – [out] The buffer to store the read content.

  • size – The number of bytes to read.

  • offset – The position in the stream to read from.

  • callback – The callback function to be invoked upon completion of the read operation. The callback will receive a Status object indicating the success or failure of the read operation.

virtual Result<uint64_t> Length() const override#

Get the total length of the file in bytes.

virtual Status Close() override#

Close the stream.

virtual Result<std::string> GetUri() const override#

Get an identifier that uniquely identify the underlying content.

Returns:

An uri if the underlying content can be uniquely identified.

Returns:

Empty string if the underlying content cannot be uniquely identified.

Public Static Attributes

static constexpr int32_t DEFAULT_BUFFER_SIZE = 8192#
class ByteArrayInputStream : public paimon::InputStream#

Input stream for memory buffer, inherits from InputStream.

Public Functions

ByteArrayInputStream(const char *buffer, uint64_t length)#
~ByteArrayInputStream() override = default#
const char *GetRawData() const#
Returns:

The raw data pointer of current pos.

virtual Status Seek(int64_t offset, SeekOrigin origin) override#

Seek to a specified position in the input stream.

Parameters:
Returns:

Status indicating success (OK) or failure with appropriate error information.

inline virtual Result<int64_t> GetPos() const override#

Get the current position in the input stream.

Returns:

Current position in the input stream.

Returns:

IOError returned if an I/O error occurred in the underlying stream. implementation while accessing the stream’s position.

virtual Result<int32_t> Read(char *buffer, uint32_t size) override#

Read data from the current position in the stream.

Note

The stream position advances by the number of bytes actually read.

Parameters:

  • buffer – [out] Pointer to the buffer where read data will be stored.

  • size – Maximum number of bytes to read.

Returns:

Result containing the actual number of bytes read on success, or an error status on failure.

virtual Result<int32_t> Read(char *buffer, uint32_t size, uint64_t offset) override#

Read data from given position in the stream.

Read with offset performs like pread() function, which will not change the position in the input stream.

Parameters:

  • buffer – [out] The buffer to store the read content.

  • size – The number of bytes to read.

  • offset – The position in the stream to read from.

virtual void ReadAsync(char *buffer, uint32_t size, uint64_t offset, std::function<void(Status)> &&callback) override#

Asynchronously read data from the input stream.

This function initiates an asynchronous read operation. The specified number of bytes will be read from the stream starting at the given offset and stored in the provided buffer. Once the read operation is complete, the provided callback function will be invoked with the status of the read operation.

Parameters:

  • buffer – [out] The buffer to store the read content.

  • size – The number of bytes to read.

  • offset – The position in the stream to read from.

  • callback – The callback function to be invoked upon completion of the read operation. The callback will receive a Status object indicating the success or failure of the read operation.

inline virtual Result<uint64_t> Length() const override#

Get the total length of the file in bytes.

virtual Status Close() override#

Close the stream.

virtual Result<std::string> GetUri() const override#

Get an identifier that uniquely identify the underlying content.

Returns:

An uri if the underlying content can be uniquely identified.

Returns:

Empty string if the underlying content cannot be uniquely identified.

class DataInputStream#

DataInputStream provides a convenient wrapper around InputStream for reading typed data.

Note

The default byte order is big-endian to maintain compatibility with the Java implementation.

Public Functions

explicit DataInputStream(const std::shared_ptr<InputStream> &input_stream)#

Constructs a DataInputStream wrapping the given InputStream.

Parameters:

input_stream – The underlying input stream to read from.

Status Seek(int64_t offset) const#

Seek to a specific position in the underlying input stream.

Parameters:

offset – The absolute byte offset to seek to.

template<typename T>
Result<T> ReadValue() const#

Read a typed value from the stream.

Returns:

Result containing the read value or an error status.

Status ReadBytes(Bytes *bytes) const#

Read some bytes to a Bytes object from the stream.

The length of bytes is the number of bytes read from the stream.

Parameters:

bytes – Buffer to store the read bytes.

Status Read(char *data, uint32_t size) const#

Read raw data of specified size from the stream.

Parameters:

  • data – Buffer to store the read data.

  • size – Number of bytes to read.

Result<std::string> ReadString() const#

Read string from the stream.

Note

First read length (int16), then read string bytes.

Result<int64_t> GetPos() const#

Get the current position in the underlying input stream.

Result<uint64_t> Length() const#

Get the total length of the underlying input stream.

inline void SetOrder(ByteOrder order)#

Set the byte order for endianness conversion.

Parameters:

order – The byte order to use PAIMON_BIG_ENDIAN or PAIMON_LITTLE_ENDIAN.