22/02/2020

[Kafka] Exactly once delivery

Among the various configuration settings that you set for your producers and consumers, there is a specific set that is very important if you care about EOD (Exactly Once Delivery).

Producers:
  • enable.idempotence = true
  • max.in.flight.requests.per.connection <= 5
  • retries > 0
  • acks = "all"
Consumers:
  • isolation.level = "read_committed"

The consumer configuration is NOT strictly necessary, since messages that are not part of a transaction are not affected by it, but in case your producer would send messages as part of a transaction, you might want to avoid reading aborted ones.

No comments:

Post a Comment

With great power comes great responsibility