Skip to content

Configuration

Icepack is configured through environment variables. Defaults are read from icepack.config.CompactionConfig unless a component has a separate runtime default.

API

VariableConfig pathDefaultRequiredDescription
ICEPACK_MAX_CONCURRENT_JOBSmax_workers2noMaximum concurrent in-process maintenance workers per API replica.
ICEPACK_TABLE_CACHE_REFRESH_SECONDStable_cache_refresh_seconds300noTable inventory refresh cadence in seconds.

API / Worker

VariableConfig pathDefaultRequiredDescription
ICEPACK_DRAIN_MODEdrain_modefalsenoWhen true/on, the API rejects new maintenance submissions and the worker avoids stale-claim recovery.
ICEPACK_POSTGRES_HEALTH_CHECK_TIMEOUT_SECONDSpostgres.health_check_timeout_seconds2.0noMaximum seconds readiness waits for a pool connection and SELECT 1 probe.
ICEPACK_POSTGRES_MAX_CONNECTIONSpostgres.max_connections20noMaximum PostgreSQL pool connections.
ICEPACK_POSTGRES_MIN_CONNECTIONSpostgres.min_connections4noMinimum PostgreSQL pool connections.
ICEPACK_SOCKET_TIMEOUT_MSspark.socket_timeout_ms7200000noSpark Thrift socket read timeout in milliseconds.
ICEPACK_TABLE_CACHE_STALENESS_SECONDStable_cache_staleness_seconds2x ICEPACK_TABLE_CACHE_REFRESH_SECONDSnoMaximum table-cache age before readiness fails.

API / Worker / CLI

VariableConfig pathDefaultRequiredDescription
ICEPACK_CATALOGspark.cataloglakehouse_devnoIceberg catalog name used in Spark SQL statements.
ICEPACK_SPARK_HOSTspark.hostkyuubi-maintenance.data-platform.us-east-1.test-dataops.fetchrewards.comnoSpark Thrift Server / Kyuubi hostname.
ICEPACK_SPARK_PORTspark.port10009noSpark Thrift Server / Kyuubi port.

API / Worker / Health Sync

VariableConfig pathDefaultRequiredDescription
AWS_REGIONaws_regionus-east-1noAWS region used for Glue and S3 API calls.
ICEPACK_GLUE_CATALOGglue_catalog_namelakehouse_devnoGlue catalog name used by metadata inspectors for direct Iceberg metadata access.
ICEPACK_ICEBERG_GO_INSPECTOR_PATHiceberg_go_inspector_path/usr/local/bin/icepack-iceberg-inspectornoPath to the bundled iceberg-go metadata inspector helper.
ICEPACK_ICEBERG_GO_INSPECTOR_TIMEOUT_SECONDSiceberg_go_inspector_timeout_seconds120noMaximum seconds to wait for the iceberg-go metadata inspector helper.
ICEPACK_METADATA_INSPECTORmetadata_inspectorpyicebergnoMetadata inspector backend for table health: pyiceberg or iceberg-go.
ICEPACK_POLARIS_CATALOGpolaris.catalog_nameglue_devnoPolaris REST catalog name.
ICEPACK_POLARIS_CLIENT_IDpolaris.client_idunsetnoOAuth2 client ID for Polaris REST catalog access.
ICEPACK_POLARIS_CLIENT_SECRETpolaris.client_secretunsetyesOAuth2 client secret for Polaris REST catalog access.
ICEPACK_POLARIS_URIpolaris.uriunsetnoPolaris REST catalog URI. Empty falls back to Glue.
ICEPACK_POSTGRES_DATABASEpostgres.databaseicepacknoPostgreSQL database name.
ICEPACK_POSTGRES_HOSTpostgres.hostlocalhostnoPostgreSQL hostname.
ICEPACK_POSTGRES_PASSWORDpostgres.passwordrequiredyesPostgreSQL password.
ICEPACK_POSTGRES_PORTpostgres.port5432noPostgreSQL port.
ICEPACK_POSTGRES_USERpostgres.usericepacknoPostgreSQL username.

API / Worker / Orchestrator

VariableConfig pathDefaultRequiredDescription
ICEPACK_ENVIRONMENT-unknownnoDeployment environment label attached to Icepack operational metrics.

Health Sync

VariableConfig pathDefaultRequiredDescription
ICEPACK_HEALTH_SYNC_CONCURRENCYhealth_sync_concurrency10noMaximum concurrent table metadata inspections for health sync.
ICEPACK_HEALTH_SYNC_DATABASEShealth_sync_databasesallnoComma-separated databases to sync health for. Empty means all databases.
ICEPACK_MIMIR_ENDPOINTmimir_endpointunsetnoOTLP gRPC endpoint for Mimir metrics push. Empty disables push.

Orchestrator

VariableConfig pathDefaultRequiredDescription
ICEPACK_API_URL-http://icepack-api.icepack.svc.cluster.local:443noIcepack API base URL used for table health checks and job submission.
ICEPACK_DRY_RUN-falsenoWhen true, logs orchestration decisions without submitting maintenance jobs.
ICEPACK_HEALTH_CONCURRENCY-10noMaximum concurrent health check requests during orchestration.
ICEPACK_MAX_SUBMIT-3noMaximum total maintenance jobs submitted by one orchestrator run.
ICEPACK_POLL_INTERVAL-30noSeconds between job status polls while the orchestrator waits for submitted jobs.
ICEPACK_SLACK_WEBHOOK_URL-unsetnoSlack incoming webhook URL for orchestrator summaries. Optional.

Orchestrator, API

VariableConfig pathDefaultRequiredDescription
ICEPACK_INCLUDE_DATABASESinclude_databasesunsetnoComma-separated database allowlist. The orchestrator maintains only these databases; the API also surfaces it on GET /tables so the UI greys out databases outside the allowlist.
ICEPACK_MAINTENANCE_CADENCE_HOURS-24noGlobal minimum hours between automated maintenance runs for one table.
ICEPACK_OPT_IN_MODE-falsenoOrchestrator enrollment mode. When false (opt-out, the default), every table in the database allowlist is maintained unless it sets icepack.maintenance_enabled=false or compaction_skip=true. When true (opt-in), only tables with icepack.maintenance_enabled=true are maintained.

Worker / Orchestrator

VariableConfig pathDefaultRequiredDescription
OTEL_EXPORTER_OTLP_ENDPOINT-unsetnoOTLP endpoint used by ephemeral worker and orchestrator pods for push metrics. Leave unset to disable.