
# PIP-444: Rate limit for deleting ledger to alleviate the zk pressure.

# Background knowledge

Apache Pulsar uses Apache BookKeeper as its storage layer. 
Each topic consists of multiple ledgers, which are created periodically based on the configuration.
Each ledger corresponds to a ZooKeeper (ZK) node for metadata storage. When a ledger is deleted, the corresponding ZK node will also be deleted.
Retention policies determine how long messages are retained in a topic. When the retention period is reached, the corresponding ledgers will be deleted.

# Motivation

When the retention of a large topic is reduced, a significant number of ledgers need to be deleted.
Since this deletion operation is not rate-limited, it could results in ZooKeeper (ZK) latency of several minutes,
which threatens the stability of the entire Pulsar cluster.

# Goals

Add rate limit feature for deleting ledgers.

# Detailed Design

Introduce a new configuration parameter:
```
    @FieldContext(
            category = CATEGORY_STORAGE_ML,
            doc = "Max number of concurrent requests for deleting ledgers at broker level"
    )
    private int managedLedgerDeleteMaxConcurrentRequests = 1000;
```
These parameters can be configured in the `broker.conf` file.

When the `managedLedgerDeleteMaxConcurrentRequests` parameter is set to a value greater than 0,
the concurrency of ledger deletions will be limited to the specified value.

# Backward & Forward Compatibility

Fully backward compatible and forward compatible.

## Upgrade

New feature is enabled by default with a rate limit of 1000 deletions per second and concurrency of 1.

## Downgrade / Rollback

To downgrade or rollback, simply revert the configuration changes in the `broker.conf` file and restart the broker.


# Links

<!--
Updated afterwards
-->
* Mailing List discussion thread: https://lists.apache.org/thread/ppmvm4noowlrhr229zf65mc44xqhj8k1
* Mailing List voting thread: https://lists.apache.org/thread/2dop6kyvrpt6ztz9pvylyl8o2syvh9kw
