Skip to content
Icepack

Maintenance Actions

Icepack executes maintenance actions in the enum order defined by icepack.models.MaintenanceAction. The generated SQL examples come from icepack.spark.SparkQueryEngine.maintenance_sql.

Execution order

1. expire_snapshots

Trigger: The table has old snapshots that should be dereferenced before physical cleanup and compaction.

Expires old snapshots while retaining the configured minimum snapshot count.

/* {"app": "icepack", "table": "my_database.my_table", "action": "EXPIRE_SNAPSHOTS"} */
CALL lakehouse_dev.system.expire_snapshots(
    table => 'my_database.my_table',
    older_than => TIMESTAMP '2026-04-20 00:00:00.000',
    retain_last => 3,
    stream_results => true
)

2. remove_orphan_files

Trigger: Runs after snapshot expiration and before compaction so stale physical files are removed first.

Deletes files under the table location that no active snapshot references.

/* {"app": "icepack", "table": "my_database.my_table", "action": "REMOVE_ORPHAN_FILES"} */
CALL lakehouse_dev.system.remove_orphan_files(
    table => 'my_database.my_table',
    older_than => TIMESTAMP '2026-04-22 00:00:00.000',
    location => 's3a://example-bucket/my_database/my_table'
)

3. rewrite_data_files

Trigger: The cleaned table still has enough undersized data files or delete-file pressure to trigger compaction.

Compacts active small data files and rewrites delete-heavy data files using Iceberg’s binpack strategy.

/* {"app": "icepack", "table": "my_database.my_table", "action": "REWRITE_DATA_FILES"} */
CALL lakehouse_dev.system.rewrite_data_files(
    table => 'my_database.my_table',
    strategy => 'binpack',
    options => map(
        'target-file-size-bytes', '536870912',
        'min-file-size-bytes', '402653184',
        'max-file-size-bytes', '966367641',
        'min-input-files', '5',
        'delete-file-threshold', '2',
        'delete-ratio-threshold', '0.1',
        'partial-progress.enabled', 'true',
        'partial-progress.max-commits', '10'
    )
)

4. rewrite_position_delete_files

Trigger: The table has position-delete files above the configured threshold after data-file compaction.

Compacts remaining position-delete files and removes dangling delete records after data-file rewrites.

/* {"app": "icepack", "table": "my_database.my_table", "action": "REWRITE_POSITION_DELETE_FILES"} */
CALL lakehouse_dev.system.rewrite_position_delete_files(
    table => 'my_database.my_table',
    options => map(
        'partial-progress.enabled', 'true',
        'partial-progress.max-commits', '10'
    )
)

5. rewrite_manifests

Trigger: The table has enough manifest files to justify metadata consolidation.

Consolidates manifests after compaction so manifests match the final file layout.

/* {"app": "icepack", "table": "my_database.my_table", "action": "REWRITE_MANIFESTS"} */
CALL lakehouse_dev.system.rewrite_manifests(table => 'my_database.my_table')