Skip to content

Performance Tuning

PII Eraser is designed for high throughput, but client-side implementation details significantly impact total processing speed.

1. Concurrency is Key

The benchmarks (see Benchmarks) demonstrate that PII Eraser scales linearly with concurrency up to the limit of the CPU.

  • Do not send requests serially (one after another).
  • Do use asynchronous requests or multi-threading in your application.

Recommendation: If you have a cluster of PII Eraser instances behind a load balancer, ensure your client application maintains 4 * [Number of Cores] concurrent requests to fully saturate the hardware.

2. Batching vs. Single Requests

The API accepts lists of strings ["text1", "text2"]. * Batching: Sending 10 short strings in one request reduces HTTP overhead (latency). * Throughput: For pure token throughput (tokens/second), batching has a negligible effect compared to concurrency.

3. Keep-Alive Connections

Ensure your HTTP client (e.g., requests.Session in Python or http.Agent in Node.js) uses persistent connections (Keep-Alive). Re-establishing TCP/TLS handshakes for every API call adds significant latency.