Skip to content

Support OpenSSL 3.0 APIs for Diffie-Hellman#13349

Open
JosiahWI wants to merge 3 commits into
apache:masterfrom
JosiahWI:feat/openssl3-pkey-compat
Open

Support OpenSSL 3.0 APIs for Diffie-Hellman#13349
JosiahWI wants to merge 3 commits into
apache:masterfrom
JosiahWI:feat/openssl3-pkey-compat

Conversation

@JosiahWI

@JosiahWI JosiahWI commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

This is work for #13347. It does not remove the OpenSSL 1.1.1 compatibility, but it switches the implementation to use OpenSSL 3.x APIs when available.

The change is split into three commits. The first one is a suite of unit tests generated by Claude Opus. They are written to test through a high-level API for now, but target the DH key loading. I reviewed them and fixed some issues in the generated code. The tests use OpenSSL 3.x APIs and are only added to the build if those APIs are available.

The second commit is the OpenSSL 3.x implementation. This is the easiest place to view the meat of the change without distraction.

Finally, I split most of the work here into a new file called SSLKeyUtils.cc in the third commit. SSLUtils.cc needs refactoring according to CodeScene (the file is massive and not cohesive), and I saw an opportunity to make a small chip in it here in that direction, with the special benefit of isolating all the messy precompiler directives in the new file so that the SSLUtils.cc function remains sleek and build-configuration-independent.

When reviewing, please take a look at EVP_PKEY_CTX_set_dh_rfc5114. In the OpenSSL 3.x implementation of gen_dh_2048_258 I would rather use that helper function than manually set params. When I tried it, OpenSSL happily failed the corresponding unit test. Closer docs inspection reveals that the key type has to be EVP_PK_DHX, even for the dh version of the function. I don't know the difference between DH and DHX and whether it would be behavior-preserving to switch, so I'm asking for help here.

I have been thorough in error-checking and memory management, referring to OpenSSL's API documentation, but I am not familiar with OpenSSL APIs yet. I would appreciate a review from someone with a lot of OpenSSL experience.

@JosiahWI JosiahWI added this to the 11.0.0 milestone Jun 29, 2026
@JosiahWI JosiahWI self-assigned this Jun 29, 2026
@JosiahWI JosiahWI added Build work related to build configuration or environment SSL labels Jun 29, 2026
Comment on lines +60 to +63
char prime_group[]{"dh_2048_256"};
OSSL_PARAM const params[]{OSSL_PARAM_utf8_string("group", prime_group, sizeof(prime_group) - 1), OSSL_PARAM_END};

if (!EVP_PKEY_CTX_set_params(pctx.get(), params)) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where I want to use the EVP_PKEY_CTX_set_dh_rfc5114 function instead of manually setting parameters.

Comment on lines +95 to +102
set_ctx_dh(SSL_CTX *ctx, dh_key_t *pkey)
{
bool result{SSL_CTX_set_options(ctx, SSL_OP_SINGLE_DH_USE) && SSL_CTX_set0_tmp_dh_pkey(ctx, pkey)};
if (!result) {
EVP_PKEY_free(pkey);
}
return result;
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of special note: the OpenSSL 3.0 API takes ownership of the key, whereas the older API does not. Therefore, the control flow with respect to memory management is different between the two set_ctx_dh implementations, which could be surprising.

Comment thread src/iocore/net/unit_tests/test_SSLDHParams.cc Outdated
@bryancall bryancall requested a review from maskit June 29, 2026 22:05
@JosiahWI JosiahWI force-pushed the feat/openssl3-pkey-compat branch from 948fe92 to 53c6090 Compare June 29, 2026 22:21
@JosiahWI JosiahWI marked this pull request as ready for review June 30, 2026 02:08
CHECK(init_with_dhparams(nullptr));
}

TEST_CASE("ssl_context_enable_dhe: valid ffdhe2048 DH PEM file is accepted")

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name needs to be corrected. I changed the format to dh_2048_256.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Build work related to build configuration or environment SSL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant