Skip to content

Conversation

@hwdef
Copy link
Contributor

@hwdef hwdef commented Nov 19, 2025

Ref: #18667 #20545

This PR adds a benchmark for txn_put_get, specifically tailored for issue #18667.

Regarding the original issue, there are some existing solutions. However, we are concerned that they might impact performance. The challenge is that we currently lack a dedicated benchmark case to quantify the extent of this potential impact. Some discussions in #20545

The key to this benchmark case is to perform both the PUT and GET within a single transaction while utilizing a prefix query. This way, when there is ahrtr@76ac23f, the GET/RANGE requests can not be skipped, which degraded performance, and vice versa.

I'll attach the benchmark results shortly.

cc @siyuanfoundation @serathius @ahrtr

@k8s-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: hwdef
Once this PR has been reviewed and has the lgtm label, please assign siyuanfoundation for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov
Copy link

codecov bot commented Nov 19, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.11%. Comparing base (bc12c94) to head (aef7d12).
⚠️ Report is 30 commits behind head on main.

Additional details and impacted files

see 22 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #20953      +/-   ##
==========================================
- Coverage   69.20%   69.11%   -0.09%     
==========================================
  Files         422      422              
  Lines       34841    34841              
==========================================
- Hits        24110    24082      -28     
- Misses       9330     9355      +25     
- Partials     1401     1404       +3     

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc12c94...aef7d12. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hwdef
Copy link
Contributor Author

hwdef commented Nov 20, 2025

/retest

@hwdef
Copy link
Contributor Author

hwdef commented Nov 20, 2025

/test pull-etcd-unit-test-arm64

@hwdef
Copy link
Contributor Author

hwdef commented Dec 3, 2025

original:

benchmark --endpoints http://127.0.0.1:2379,http://127.0.0.1:22379,http://127.0.0.1:32379 --conns 3 --clients 3 txn-put-get
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #3] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #3] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:41:02 INFO: [core] original dial target is: "etcd-endpoints://0xc0001fc600/127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #4] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #4] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #2] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel created for target "etcd-endpoints://0xc0001fc600/127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #5] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"etcd-endpoints", Opaque:"", User:(*url.Userinfo)(nil), Host:"0xc0001fc600", Path:"/127.0.0.1:2379", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel authority set to "127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #5] Resolver state updated: {
  "Addresses": null,
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:2379",
          "ServerName": "127.0.0.1:2379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:22379",
          "ServerName": "127.0.0.1:22379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:32379",
          "ServerName": "127.0.0.1:32379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": {
    "Config": {
      "Config": null,
      "Methods": {}
    },
    "Err": null
  },
  "Attributes": null
} (service config updated)
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #3] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #1] Channel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel switches to new LB policy "round_robin"
2025/12/03 15:41:02 INFO: [roundrobin] [0xc000254990] Created
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #9] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #1 SubChannel #4] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #10] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #11] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #10] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #11] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #9] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #10] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #9] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #11] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel exiting idle mode
2025/12/03 15:41:02 INFO: [core] original dial target is: "etcd-endpoints://0xc0000ee400/127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel created for target "etcd-endpoints://0xc0000ee400/127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #14] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"etcd-endpoints", Opaque:"", User:(*url.Userinfo)(nil), Host:"0xc0000ee400", Path:"/127.0.0.1:2379", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel authority set to "127.0.0.1:2379"
2025/12/03 15:41:02 INFO: [core] [Channel #14] Resolver state updated: {
  "Addresses": null,
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:2379",
          "ServerName": "127.0.0.1:2379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:22379",
          "ServerName": "127.0.0.1:22379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:32379",
          "ServerName": "127.0.0.1:32379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": {
    "Config": {
      "Config": null,
      "Methods": {}
    },
    "Err": null
  },
  "Attributes": null
} (service config updated)
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #10] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel switches to new LB policy "round_robin"
2025/12/03 15:41:02 INFO: [roundrobin] [0xc00058bb30] Created
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #9] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #5] Channel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #16] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #5 SubChannel #11] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #17] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #18] Subchannel created
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #16] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #18] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel exiting idle mode
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #17] Subchannel Connectivity change to CONNECTING
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #16] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #18] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #17] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #16] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #17] Subchannel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14] Channel Connectivity change to READY
2025/12/03 15:41:02 INFO: [core] [Channel #14 SubChannel #18] Subchannel Connectivity change to READY
10000 / 10000 [---------------------------------------------] 100.00% 220 p/s

Summary:
  Total:        45.6897 secs.
  Slowest:      0.2107 secs.
  Fastest:      0.0022 secs.
  Average:      0.0137 secs.
  Stddev:       0.0066 secs.
  Requests/sec: 218.8677

Response time histogram:
  0.0022 [1]    |
  0.0231 [9607] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.0439 [382]  |∎
  0.0648 [1]    |
  0.0856 [0]    |
  0.1065 [0]    |
  0.1273 [3]    |
  0.1482 [0]    |
  0.1690 [0]    |
  0.1899 [2]    |
  0.2107 [4]    |

Latency distribution:
  10% in 0.0093 secs.
  25% in 0.0105 secs.
  50% in 0.0126 secs.
  75% in 0.0159 secs.
  90% in 0.0194 secs.
  95% in 0.0220 secs.
  99% in 0.0281 secs.
  99.9% in 0.0481 secs.

modified:

$ benchmark --endpoints http://127.0.0.1:2379,http://127.0.0.1:22379,http://127.0.0.1:32379 --conns 3 --clients 3 txn-put-get
2025/12/03 15:51:24 INFO: [core] original dial target is: "etcd-endpoints://0xc000436400/127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel created for target "etcd-endpoints://0xc000436400/127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #3] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #2] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #3] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #5] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"etcd-endpoints", Opaque:"", User:(*url.Userinfo)(nil), Host:"0xc000436400", Path:"/127.0.0.1:2379", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel authority set to "127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #4] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #1] Channel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #5] Resolver state updated: {
  "Addresses": null,
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:2379",
          "ServerName": "127.0.0.1:2379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:22379",
          "ServerName": "127.0.0.1:22379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:32379",
          "ServerName": "127.0.0.1:32379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": {
    "Config": {
      "Config": null,
      "Methods": {}
    },
    "Err": null
  },
  "Attributes": null
} (service config updated)
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #4] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel switches to new LB policy "round_robin"
2025/12/03 15:51:24 INFO: [roundrobin] [0xc0001fc150] Created
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #4] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #9] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #10] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #11] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel exiting idle mode
2025/12/03 15:51:24 INFO: [core] [Channel #1 SubChannel #3] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] original dial target is: "etcd-endpoints://0xc000436600/127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #10] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel created for target "etcd-endpoints://0xc000436600/127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #11] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #11] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #12] parsed dial target is: resolver.Target{URL:url.URL{Scheme:"etcd-endpoints", Opaque:"", User:(*url.Userinfo)(nil), Host:"0xc000436600", Path:"/127.0.0.1:2379", RawPath:"", OmitHost:false, ForceQuery:false, RawQuery:"", Fragment:"", RawFragment:""}}
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel authority set to "127.0.0.1:2379"
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #10] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #9] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #9] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #12] Resolver state updated: {
  "Addresses": null,
  "Endpoints": [
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:2379",
          "ServerName": "127.0.0.1:2379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:22379",
          "ServerName": "127.0.0.1:22379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    },
    {
      "Addresses": [
        {
          "Addr": "127.0.0.1:32379",
          "ServerName": "127.0.0.1:32379",
          "Attributes": null,
          "BalancerAttributes": null,
          "Metadata": null
        }
      ],
      "Attributes": null
    }
  ],
  "ServiceConfig": {
    "Config": {
      "Config": null,
      "Methods": {}
    },
    "Err": null
  },
  "Attributes": null
} (service config updated)
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel switches to new LB policy "round_robin"
2025/12/03 15:51:24 INFO: [roundrobin] [0xc0001634a0] Created
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #13] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #17] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #18] Subchannel created
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #13] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel exiting idle mode
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #9] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #5] Channel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #17] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #10] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #18] Subchannel Connectivity change to CONNECTING
2025/12/03 15:51:24 INFO: [core] [Channel #5 SubChannel #11] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #18] Subchannel picks a new address "127.0.0.1:22379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #17] Subchannel picks a new address "127.0.0.1:2379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #13] Subchannel picks a new address "127.0.0.1:32379" to connect
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #17] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12] Channel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #18] Subchannel Connectivity change to READY
2025/12/03 15:51:24 INFO: [core] [Channel #12 SubChannel #13] Subchannel Connectivity change to READY
10000 / 10000 [---------------------------------------------] 100.00% 220 p/s

Summary:
  Total:        45.7123 secs.
  Slowest:      0.2121 secs.
  Fastest:      0.0022 secs.
  Average:      0.0137 secs.
  Stddev:       0.0058 secs.
  Requests/sec: 218.7596

Response time histogram:
  0.0022 [1]    |
  0.0232 [9536] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  0.0442 [457]  |∎
  0.0652 [3]    |
  0.0862 [0]    |
  0.1072 [0]    |
  0.1282 [0]    |
  0.1491 [0]    |
  0.1701 [0]    |
  0.1911 [0]    |
  0.2121 [3]    |

Latency distribution:
  10% in 0.0094 secs.
  25% in 0.0104 secs.
  50% in 0.0124 secs.
  75% in 0.0159 secs.
  90% in 0.0198 secs.
  95% in 0.0226 secs.
  99% in 0.0292 secs.
  99.9% in 0.0390 secs.

I think the design of this benchmark case is fine, but the test results still show that the original and modified versions are the same.
Perhaps this indicates that removeNeedlessRangeReqs() has little impact on performance? :)

@siyuanfoundation
Copy link
Contributor

/cc @ahrtr

@k8s-ci-robot k8s-ci-robot requested a review from ahrtr December 4, 2025 15:33
Signed-off-by: hwdef <hwdefcom@outlook.com>
@hwdef hwdef force-pushed the add-txn-benchmark branch from 1833b61 to aef7d12 Compare December 4, 2025 16:35
@k8s-ci-robot
Copy link

@hwdef: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-etcd-grpcproxy-integration-amd64 aef7d12 link true /test pull-etcd-grpcproxy-integration-amd64
pull-etcd-coverage-report aef7d12 link true /test pull-etcd-coverage-report

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants