Skip to content

Partitioning

Running one instance for everything works fine until you have multiple tenants or multiple target APIs sharing it, and then one tenant’s burst, or one target API’s own rate limit, ends up affecting everyone else on that same instance. Two ways to split traffic apart so that doesn’t happen, not mutually exclusive:

Static partitioning: decided once, at deploy time. Dedicate one instance to a single protected resource (a CI runner, a database, a GPU, or a rate-limited external API you want to be nice to) so that resource only ever sees traffic paced the way you configured, up to whatever it can actually bear. Multiple tenants can safely share that same instance: turn on account-queue isolation and each tenant gets their own independently-paced queue, so one tenant’s burst doesn’t starve another’s, and the resource itself never sees more aggregate load than it’s rated for. The mistake to avoid: pointing multiple instances at the same resource instead of routing everyone through this one pacing checkpoint, which just multiplies your total request rate against it. Same rule for pools: a given pool_id should belong to exactly one instance, since pool state isn’t shared across instances.

Optional regional cluster routing supports either a static rendezvous member list or Valkey-coordinated, load-aware rendezvous. The Valkey mode discovers Aquifer instances, keeps active users sticky while capacity is available, and releases idle assignments without requiring another control plane. See API.md for failure semantics and configuration.

Dynamic partitioning (drain mode): off by default, for a more specific shape. Instead of deciding every assignment up front, an instance gets handed to one tenant at a time, absorbs and drains whatever burst that tenant sends, then frees itself up to be handed to a different tenant next, useful when you want dedicated capacity per user without hand-assigning it at deploy time. Aquifer can stream completed/failed job ledger events in acknowledged batches to a webhook or Valkey, and when idle for AQUIFER_DRAIN_TIMER_SECONDS, it flushes anything remaining before clearing local state and moving through an active → draining → unassigned state machine visible via GET /health. See DRAIN_MODE.md for the full state machine, env vars, and payload shape.

The two combine: a fleet can partition statically by target API domain, while individual instances within a partition cycle through tenants dynamically via drain mode.

External registration: off by default, and orthogonal to the above. AQUIFER_REGISTRY_URL makes an instance periodically report its own listening port to an external control plane (deciding tenant assignment, scaling, etc. is entirely that service’s job, not Aquifer’s). See REGISTRATION.md for the env vars and ping payload shape.

None of this needs instances sharing state centrally, and that’s the normal shape for a load balancer, not a gap unique to Aquifer: nginx and HAProxy make local decisions the same way. Pure central rate limiting is a gateway-layer concern that composes in front of Aquifer if you want it, not something Aquifer needs to reinvent. What an instance does need is to be safe to hand off without knowing anything about the rest of the fleet, which is what slow start is for: a freshly-assigned instance starts below its configured ceiling and creeps up, rather than assuming the ramp some other instance already earned.