HBASE-29729 Add per-region table descriptor hash to regionServer JMX Metrics #7481

ajkh88 · 2025-11-25T18:05:35Z

When table descriptors are modified (e.g., changing compression settings, TTL, or column family configurations), region servers need to reopen regions to pick up these changes. However, there's currently no reliable way to detect which regions are running with stale table descriptors versus the current configuration. This makes it difficult to verify that descriptor changes have propagated correctly across all regions, increasing the risk of configuration drift and making deployment automation more challenging.

This PR adds a tableDescriptorHash field to the per-region JMX metrics exposed by region servers. The hash is a SHA-256 digest of the serialized TableDescriptor protobuf, computed once when the region is opened and exposed through the MetricsRegionWrapper interface. Regions opened with identical table descriptors will produce identical hashes, while any configuration change (compression, TTL, block size, etc.) will produce a different hash. This enables monitoring systems and deployment automation to quickly identify regions that need reopening after table descriptor changes, without parsing and comparing complex nested configuration objects.

…metrics

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java

...e-hadoop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapper.java

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java

...t/java/org/apache/hadoop/hbase/regionserver/TestMetricsRegionWrapperTableDescriptorHash.java

Kota-SH · 2025-12-12T15:36:19Z

hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptor.java

+      crc32.update(byteBuffer);
+      return Long.toHexString(crc32.getValue());
+    } catch (Exception e) {
+      Logger log = LoggerFactory.getLogger(TableDescriptor.class);


Can we move this log variable to the class level to be consistent?

Apache-HBase · 2025-12-12T16:37:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 0s		Docker mode activated.
-1 ❌	docker	0m 2s		Docker failed to build run-specific yetus/hbase:tp-31159}.

Subsystem	Report/Notes
GITHUB PR	#7481
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/12/console
versions	git=2.17.1
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-12-12T21:10:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 33s		Maven dependency ordering for branch
+1 💚	mvninstall	4m 13s		master passed
+1 💚	compile	1m 48s		master passed
+1 💚	javadoc	1m 4s		master passed
+1 💚	shadedjars	6m 51s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 40s		the patch passed
+1 💚	compile	1m 50s		the patch passed
+1 💚	javac	1m 50s		the patch passed
+1 💚	javadoc	1m 10s		the patch passed
+1 💚	shadedjars	6m 45s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	0m 48s		hbase-hadoop-compat in the patch passed.
+1 💚	unit	1m 35s		hbase-client in the patch passed.
+1 💚	unit	236m 23s		hbase-server in the patch passed.
		273m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/12/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7481
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 4f4630c20126 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `5e7d08a`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/12/testReport/
Max. process+thread count	3984 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/12/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

rmdmattingly · 2025-12-12T22:47:05Z

Going to kick off a rebuild.

Apache-HBase · 2025-12-12T23:21:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 11s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	2m 34s		master passed
+1 💚	compile	3m 27s		master passed
+1 💚	checkstyle	1m 4s		master passed
+1 💚	spotbugs	2m 2s		master passed
+1 💚	spotless	0m 38s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s		the patch passed
+1 💚	compile	3m 22s		the patch passed
+1 💚	javac	3m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 3s		the patch passed
+1 💚	spotbugs	2m 14s		the patch passed
+1 💚	hadoopcheck	8m 23s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	0m 32s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 21s		The patch does not generate ASF License warnings.
		34m 2s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/13/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7481
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux acfb87a74478 6.8.0-1024-aws #26~22.04.1-Ubuntu SMP Wed Feb 19 06:54:57 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `5e7d08a`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	84 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/13/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-12-13T02:58:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	5m 2s		master passed
+1 💚	compile	2m 32s		master passed
+1 💚	javadoc	1m 55s		master passed
+1 💚	shadedjars	6m 46s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 16s		the patch passed
+1 💚	compile	1m 35s		the patch passed
+1 💚	javac	1m 35s		the patch passed
+1 💚	javadoc	0m 57s		the patch passed
+1 💚	shadedjars	6m 12s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	0m 36s		hbase-hadoop-compat in the patch passed.
+1 💚	unit	1m 31s		hbase-client in the patch passed.
+1 💚	unit	212m 25s		hbase-server in the patch passed.
		249m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/13/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7481
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 29c2ebb2a812 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `5e7d08a`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/13/testReport/
Max. process+thread count	3873 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7481/13/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Kota-SH

LGTM

HBASE-29729 Add per-region table descriptor hash to regionServer JMX …

9d4cb4d

…metrics

This comment has been minimized.

Sign in to view

HBASE-29729 Add integration test

ff75f81

This comment has been minimized.

Sign in to view

Trigger build

d263efe

This comment has been minimized.

Sign in to view

rmdmattingly requested review from charlesconnell and rmdmattingly November 26, 2025 16:01

charlesconnell reviewed Nov 26, 2025

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionWrapperImpl.java Outdated Show resolved Hide resolved

HBASE-29729 import instead of qualify class names

e6d3bc8

This comment has been minimized.

Sign in to view

HBASE-29729 fix checkstyle

2333c04

This comment has been minimized.

Sign in to view

rmdmattingly requested a review from ndimiduk November 28, 2025 13:16

This comment has been minimized.

Sign in to view

ndimiduk reviewed Dec 1, 2025

View reviewed changes

Alex Hughes added 5 commits December 2, 2025 17:07

Merge branch 'master' into HBASE-29729/add-table-descriptor-hash-metric

7a10dcf

HBASE-29729 Move hash to TableDescriptor

60fa1d2

HBASE-29729 Use try-with-resources

7329edf

HBASE-29729 Move logger construction

971efc1

HBASE-29729 Spotless

0937d3c

This comment has been minimized.

Sign in to view

HBASE-29729 Use CRC32 hash

cd205de

This comment has been minimized.

Sign in to view

charlesconnell approved these changes Dec 4, 2025

View reviewed changes

Trigger build

87572d4

This comment has been minimized.

Sign in to view

Alex Hughes added 2 commits December 10, 2025 13:33

HBASE-29729 Add tag and use byte buffer

d22379a

HBASE-29729 Spotless

76d0079

This comment has been minimized.

Sign in to view

Kota-SH reviewed Dec 12, 2025

View reviewed changes

HBASE-29729 Move logger to interface level

5e7d08a

rmdmattingly approved these changes Dec 13, 2025

View reviewed changes

Kota-SH approved these changes Dec 15, 2025

View reviewed changes

HBASE-29729 Add per-region table descriptor hash to regionServer JMX Metrics #7481

Are you sure you want to change the base?

HBASE-29729 Add per-region table descriptor hash to regionServer JMX Metrics #7481

Conversation

ajkh88 commented Nov 25, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Kota-SH Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Apache-HBase commented Dec 12, 2025

Uh oh!

Apache-HBase commented Dec 12, 2025

Uh oh!

rmdmattingly commented Dec 12, 2025

Uh oh!

Apache-HBase commented Dec 12, 2025

Uh oh!

Apache-HBase commented Dec 13, 2025

Uh oh!

Kota-SH left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants