Skip to content

TEZ-4677: Upgrades Tez to compile and run with JDK 25#482

Open
maheshrajus wants to merge 1 commit into
apache:masterfrom
maheshrajus:TEZ-4677
Open

TEZ-4677: Upgrades Tez to compile and run with JDK 25#482
maheshrajus wants to merge 1 commit into
apache:masterfrom
maheshrajus:TEZ-4677

Conversation

@maheshrajus

@maheshrajus maheshrajus commented Apr 15, 2026

Copy link
Copy Markdown
Contributor
  • Upgrades Tez source compilation target to JDK 25
  • Updates Yetus CI Docker image to JDK 25 for pre-commit testing

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/1/console in case of problems.

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 4m 51s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for branch
+1 💚 mvninstall 2m 34s master passed
+1 💚 compile 1m 24s master passed
+1 💚 javadoc 1m 14s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 2m 22s the patch passed
+1 💚 codespell 0m 49s No new issues.
+1 💚 compile 1m 26s the patch passed
-1 ❌ javac 1m 6s /results-compile-javac-root.txt root generated 4 new + 17 unchanged - 17 fixed = 21 total (was 34)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 hadolint 0m 2s No new issues.
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 javadoc 1m 10s the patch passed
_ Other Tests _
-1 ❌ unit 0m 24s /patch-unit-tez-plugins_tez-aux-services.txt tez-aux-services in the patch passed.
-1 ❌ unit 8m 36s /patch-unit-root.txt root in the patch passed.
+1 💚 asflicense 0m 46s The patch does not generate ASF License warnings.
27m 20s
Reason Tests
Failed junit tests tez.auxservices.TestIndexCache
tez.auxservices.TestShuffleHandler
tez.auxservices.TestShuffleHandlerJobs
tez.runtime.metrics.TestFileSystemStatisticUpdater
tez.runtime.task.TestTaskReporter
tez.runtime.task.TestTaskExecution2
tez.runtime.TestLogicalIOProcessorRuntimeTask
tez.history.TestHistoryParser
tez.dag.history.ats.acls.TestATSHistoryWithACLs
tez.dag.history.logging.ats.TestATSV15HistoryLoggingService
tez.dag.history.ats.acls.TestATSHistoryV15
tez.auxservices.TestIndexCache
tez.auxservices.TestShuffleHandler
tez.auxservices.TestShuffleHandlerJobs
tez.dag.history.logging.proto.TestDagManifestFileScanner
tez.dag.history.logging.proto.TestProtoHistoryLoggingService
tez.dag.history.logging.ats.TestATSHistoryWithMiniCluster
tez.tests.TestExternalTezServicesErrors
tez.tests.TestExtServicesWithLocalMode
tez.tests.TestExternalTezServices
tez.dag.app.TestMockDAGAppMaster
tez.dag.app.dag.impl.TestVertexManager
tez.dag.app.dag.TestRootInputInitializerManager
tez.dag.api.client.TestDAGClientHandler
tez.dag.app.launcher.TestTezLocalCacheManager
tez.dag.history.recovery.TestRecoveryService
tez.client.registry.zookeeper.TestZkAMRegistryClient
tez.dag.app.TestSpeculation
tez.dag.app.TestRecoveryParser
tez.dag.api.client.rpc.TestDAGClientAMProtocolBlockingPBServerImpl
tez.dag.app.TestDAGAppMaster
tez.dag.app.dag.impl.TestVertexImpl
tez.dag.app.TestPreemption
tez.mapreduce.input.TestMRInput
tez.mapreduce.output.TestMROutputLegacy
tez.mapreduce.processor.map.TestMapProcessor
tez.mapreduce.input.TestMultiMRInput
tez.mapreduce.combine.TestMRCombiner
hadoop.mapred.split.TestGroupedSplits
tez.mapreduce.output.TestMultiMROutput
tez.mapreduce.processor.reduce.TestReduceProcessor
tez.mapreduce.output.TestMROutputConfigBuilder
tez.mapreduce.output.TestMROutput
tez.mapreduce.hadoop.TestMRInputHelpers
tez.mapreduce.common.TestMRInputAMSplitGenerator
tez.mapreduce.lib.TestKVReadersWithMR
tez.mapreduce.TestMRRJobsDAGApi
tez.test.TestLocalMode
tez.test.TestAM
tez.mapreduce.TestMRRJobs
tez.test.TestAMRecoveryAggregationBroadcast
tez.test.TestDAGRecovery
tez.test.TestTezJobs
tez.test.TestPipelinedShuffle
tez.test.TestAMRecovery
tez.test.TestTaskErrorsUsingLocalMode
tez.test.TestFaultTolerance
tez.test.TestMiniTezCluster
tez.test.TestExceptionPropagation
tez.test.TestRecovery
tez.test.TestDAGRecovery2
tez.test.TestSecureShuffle
tez.common.TestTezCommonUtils
tez.dag.api.client.rpc.TestDAGClient
tez.dag.api.client.TestTimelineReaderFactory
tez.common.TestReflectionUtils
tez.common.security.TestTokenCache
tez.client.TestTezClient
tez.client.TestTezClientUtils
tez.analyzer.TestAnalyzer
tez.runtime.library.common.sort.impl.TestTezMerger
tez.runtime.library.common.TestValuesIterator
tez.runtime.library.common.sort.impl.TestPipelinedSorter
tez.runtime.library.output.TestOrderedPartitionedKVOutput2
tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
tez.runtime.library.output.TestOnFileUnorderedKVOutput
tez.runtime.library.common.shuffle.orderedgrouped.TestShuffleInputEventHandlerOrderedGrouped
tez.runtime.library.output.TestUnorderedKVOutput2
tez.runtime.library.output.TestOnFileSortedOutput
tez.runtime.library.common.shuffle.orderedgrouped.TestFetcher
tez.runtime.library.common.shuffle.orderedgrouped.TestShuffleScheduler
tez.runtime.library.common.sort.impl.dflt.TestDefaultSorter
tez.runtime.library.common.shuffle.impl.TestShuffleInputEventHandlerImpl
tez.runtime.library.common.shuffle.TestShuffleUtils
tez.runtime.library.common.sort.impl.TestIFile
tez.runtime.library.common.shuffle.orderedgrouped.TestMergeManager
tez.runtime.library.common.readers.TestUnorderedKVReader
tez.runtime.library.common.shuffle.TestFetcher
tez.runtime.library.common.shuffle.impl.TestShuffleManager
tez.runtime.library.common.shuffle.impl.TestSimpleFetchedInputAllocator
tez.runtime.library.common.shuffle.orderedgrouped.TestShuffle
Subsystem Report/Notes
Docker ClientAPI=1.54 ServerAPI=1.54 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/1/artifact/out/Dockerfile
GITHUB PR #482
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs javac javadoc unit xmllint compile
uname Linux d16aa9eb91c2 5.15.0-141-generic #151-Ubuntu SMP Sun May 18 21:35:19 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-home/workspace/tez-multibranch_PR-482/src/.yetus/personality.sh
git revision master / 6773889
Default Java Ubuntu-25.0.2+10-Ubuntu-124.04
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/1/testReport/
Max. process+thread count 1123 (vs. ulimit of 5500)
modules C: tez-plugins/tez-aux-services . U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/1/console
versions git=2.43.0 maven=3.9.14 hadolint=1.18.0-0-g76eee5c codespell=2.4.1 shellcheck=0.7.1
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

@maheshrajus

Copy link
Copy Markdown
Contributor Author

It looks like the Jenkinsfile and Dockerfile have changed and been optimized in the latest commits. I need to rebase with the latest changes and check on this task.

@Aggarwal-Raghav, if we want to test Tez with JDK 25, which fields in the latest Jenkinsfile and Dockerfile do we need to modify? As far as I can see in the latest changes, there are no fields related to the JDK version.

@Aggarwal-Raghav

Aggarwal-Raghav commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

@maheshrajus , please check this.

  • build-tools/docker/Dockerfile basically the eclipse-temurin:21-jdk-noble
  • parent pom
<javaVersion>21</javaVersion>
<maven.compiler.release>21</maven.compiler.release>
  • in tez-dist/src/docker/Dockerfile docker image eclipse-temurin:21-jdk-ubi9-minimaloptional
  • build.yml is already handled in recent commits for github actions

Comment thread tez-plugins/tez-aux-services/pom.xml Outdated
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.6.2</version>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change is required? As we have moved to apache pom 37 using 3.6.1 maven shade plugin, please check if its required now after rebase

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me check after rebase with latest commits whether this changes is required or not. Thank you !

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i changed maven-shade-plugin version to 3.6.2 in parent pom.xml file to support JDK 25(I will push the changes after some time).
Otherwise we are getting compilations issue in tez-aux-services module.
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.6.1:shade (default) on project tez-aux-services: Error creating shaded jar: Unsupported class file major version 69

@maheshrajus maheshrajus changed the title [WIP] TEZ-4677: JDK25 testing in yetus TEZ-4677: Upgrades Tez to compile and run with JDK 25 Jun 26, 2026
@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/2/console in case of problems.

@tez-yetus

Copy link
Copy Markdown

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/3/console in case of problems.

Comment thread pom.xml
<maven.compiler.release>21</maven.compiler.release>
<javaVersion>25</javaVersion>
<maven.compiler.release>25</maven.compiler.release>
<version.maven-shade-plugin>3.6.2</version.maven-shade-plugin>

@Aggarwal-Raghav Aggarwal-Raghav Jun 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to move apache parent pom to 38 as it has been released and has <version.maven-shade-plugin>3.6.2</version.maven-shade-plugin> by default

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me check and verify it. thanks !

@tez-yetus

Copy link
Copy Markdown

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 8s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ master Compile Tests _
+0 🆗 mvndep 0m 39s Maven dependency ordering for branch
+1 💚 mvninstall 2m 53s master passed
+1 💚 compile 2m 28s master passed
+1 💚 javadoc 1m 11s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 9s Maven dependency ordering for patch
+1 💚 mvninstall 2m 11s the patch passed
+1 💚 codespell 0m 48s No new issues.
+1 💚 compile 2m 33s the patch passed
-1 ❌ javac 2m 33s /results-compile-javac-root.txt root generated 21 new + 753 unchanged - 17 fixed = 774 total (was 770)
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 hadolint 0m 1s No new issues.
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 javadoc 1m 10s the patch passed
_ Other Tests _
+1 💚 unit 61m 35s root in the patch passed.
+1 💚 asflicense 0m 51s The patch does not generate ASF License warnings.
77m 31s
Subsystem Report/Notes
Docker ClientAPI=1.55 ServerAPI=1.55 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/3/artifact/out/Dockerfile
Optional Tests dupname compile unit asflicense codespell detsecrets hadolint shellcheck shelldocs javac javadoc xmllint
uname Linux cb8e3b3ad839 5.15.0-179-generic #189-Ubuntu SMP Tue May 5 18:20:56 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality tez-personality.sh
git revision master / 17d1549
Default Java Eclipse Adoptium-25.0.3+9-LTS
Test Results https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/3/testReport/
Max. process+thread count 1389 (vs. ulimit of 5500)
modules C: tez-dist . U: .
Console output https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-482/3/console
versions git=2.43.0 maven=3.9.15 hadolint=2.14.0 codespell=2.4.1 shellcheck=0.11.0
Powered by Apache Yetus 0.15.1 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants