User Guide# Catalog Filesystem Catalog Schema Example Compatibility DataField Update Schema Snapshot Directory Layout Snapshot File Manifest Manifest List Manifest Data Manifest Index Manifest Appendix SimpleStats BinaryRow Data Types Primary Key Table Bucket LSM Trees Sorted Runs Append Only Table Write And Prepare Commit Bucketing Modes RecordBatch Construction Prepare Commit Compatibility Goals Interface Design in Paimon C++ CommitMessage Contract Serialization and Deserialization Operational Flow Commit Commit Process Overview Detailed Process Collect File Changes Compact (Merge) Manifest Files Full Compaction Minor Compaction Compaction Rules Generate the Base Manifest List Generate New Manifest Files and the Delta Manifest List Generate the Snapshot and HINT File Read and Data Evolution Schema Evolution Scope and Compatibility Per-File Schema via Field IDs Overflow Behavior Disclaimer Type Change Support Matrix Implementation Guidance Data Cleanup Orphan File Cleanup Description Detailed Steps Performance Considerations Expiring Partitions Description Detailed Steps Expiring Snapshots Description Detailed Steps Performance Considerations Prefetch Read Range Splitting Strategy Parquet Memory Format Why Apache Arrow Versioning and Dependency Concerns Adopting the Arrow C Data Interface Design Principles Implementation Outline Schema and buffers Memory management Nullability and validity Conversion boundaries