Waiting in-between RPCs increase latency

[Stuart0l]

My client is written like this:

bool complete = false;
void rpc_cont_func(void *context, void *_tag) { complete = true; }
void client_thread() {
  while (true) {
    // timer start
    rpc_->enqueue_request(session_num_, REQ, &req_, &resp_, rpc_cont_func, nullptr);
    while (!complete)
      rpc_->run_event_loop(1000);
    complete = false;
    // timer stop

    sleep(period); // rate limiter wait
  }
}

I will issue an RPC request and poll for a response before issuing the next one. After each request, I may sleep (or busy waiting) for a while (due to a rate limiter). However, as the waiting time becomes longer, the RPC request latency also increases, which is counterintuitive. Because the longer the wait time, the lighter the load, and the latency should not increase. What could be the root cause of this? And is there a way to fix it? Thanks!

[Stuart0l]

mention @shreesha00 here.

[anujkaliaiitd]

Hey! Does it happen even if you replace sleep with a busy-polling sleep? sleep usually does a context switch out of the process. If the sleep duration is long and the system is idle, DVFS might kick in and reduce CPU frequency. That might explain higher latency for requests after the sleep.

[Funlxy]

same question, have you found the problem and solved it?

[jiangxiaosheng]

I met exactly the same issue before, see #104

One possible workaround, as @anujkaliaiitd mentioned above, is using a busy loop instead of a sleep, although being inconvenient. I suspect it's because of some scheduling things but not very sure.