Skip to content

Conversation

@guo-shaoge
Copy link
Contributor

@guo-shaoge guo-shaoge commented Mar 18, 2025

What problem does this PR solve?

Issue Number: close #9901

Problem Summary:

workload: tpch50g
2 tiflash(16c) on benchbot
queries:

explain analyze select count(distinct L_SUPPKEY) from lineitem;
explain analyze select count(distinct L_ORDERKEY) from lineitem;
explain analyze select count(distinct L_EXTENDEDPRICE, L_QUANTITY) from lineitem;
explain analyze select count(distinct L_ORDERKEY, L_TAX) from lineitem;
explain analyze select count(distinct L_SHIPINSTRUCT) from lineitem;
explain analyze select count(distinct L_COMMENT) from lineitem;
HashTable Key nightly opt-no_prefetch opt-prefetch_16 rate-opt-no_prefetch rate-opt-prefetch_16
key_int64 (unary) 0.5 0.49 0.5 2.00% 0.00%
1.45 1.34 1.34 7.59% 7.59%
key_string (variadic-2 Int64) 1.65 1.61 1.63 2.42% 1.21%
6.00 5.28 4.59 12.00% 23.50%
key_string (unary) 0.58 0.56 0.56 3.45% 3.45%
8.91 8.71 8.32 2.24% 6.62%

What is changed and how it works?

  1. add addBatchSinglePlace() for AggregateFunctionUniq and AggregateFunctionUniqVariadic
  2. updateHashWithValues support compute hash from specific row instead of the whole column

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

None

Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot ti-chi-bot bot added the release-note-none Denotes a PR that doesn't merit a release note. label Mar 18, 2025
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented Mar 18, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from guo-shaoge, ensuring that each of them provides their approval before proceeding. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot ti-chi-bot bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Mar 18, 2025
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@guo-shaoge guo-shaoge force-pushed the hashagg_prefetch_count_distinct branch from 8ea181f to 6241dcc Compare March 18, 2025 06:44
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
This reverts commit f00d272.
This reverts commit 9304379.
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot ti-chi-bot bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 20, 2025
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Signed-off-by: guo-shaoge <shaoge1994@163.com>
@ti-chi-bot
Copy link
Contributor

ti-chi-bot bot commented May 9, 2025

@guo-shaoge: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-unit-test a47a9a7 link true /test pull-unit-test

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support prefetch HashSet for count(distinct)

1 participant