Running client/server apps with daemon makes rpc stuck

[mfdj2002]

Hi Anuj,

environment: ubuntu 2004, rdma-core v4 stable, dpdk 21.11, cloudlab c6525-25g (ethernet, no ib)

I'm trying to run multiple eRPC processes on one node and the only way to do this seems to start an erpc daemon. So I tried running a client/server app simulating the scenario I wanted (I basically changed the large_rpc_tput into a client/server app
Archive.zip
like the masstree analytics app). However, the client end is stuck and giving me the following output:
 
Rpc 0 stuck in rte_eth_tx_burst16:389292 WARNG: Rpc 2 stuck in rte_eth_tx_burst16:391622 WARNG: Rpc 3 stuck in rte_eth_tx_burst16:422230 WARNG: Rpc 7 stuck in rte_eth_tx_burst16:577255 ...

Any help is greatly appreciated!

[anujkaliaiitd]

The output suggests that the code isn't able to transmit packets, which suggests something wrong with how eRPC is using DPDK. I assume it works without the daemon.

The daemon stuff uses DPDK's secondary process feature. I had tested it on only a few NICs (e.g., Azure mlx5), and that too very long ago. So it's very possible that the DPDK-related code doesn't work for cloudlab c6525-25g.

It shouldn't be too hard to fix though, if you're interested in trying. The usual way to fix this is to try to get some other known-good process code (e.g., a DPDK sample) secondary working, and bisect to where the difference lies.

[mfdj2002]

Hi Anuj,
thank you for responding! Yes, I can look into the issue more when I have a minute and hopefully submit a fix!