8244336: Restrict algorithms at JCE layer

[TheMangovnik]

Backport of JDK-8244336 - Restrict algorithms at JCE layer.

This is in preparation for a backport of JDK-8375549.

This backport will be followed up by JDK-8367583 and JDK-8373690 to fix introduced bugs by this backport.

Extra changes

src/java.base/share/classes/sun/security/util/CryptoAlgorithmConstraints.java

On line 61:

- debug.println("CryptoAlgoConstraints: ", msg);
+ debug.println("CryptoAlgoConstraints: " + msg);

I made this change, because otherwise the compilation would not pass complaining about incorrect static method calling.

Tests

Tests were run on Fedora 44.

New tests - MIXED

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
   jtreg:test/jdk/javax/crypto/Cipher/TestDisabledAlgorithms.java
                                                         1     1     0     0
	jtreg:test/jdk/java/security/KeyStore/TestDisabledAlgorithms.java
>>                                                       1     0     1     0 <<
	jtreg:test/jdk/java/security/MessageDigest/TestDisabledAlgorithms.java
                                                         1     1     0     0
    jtreg:test/jdk/java/security/Signature/TestDisabledAlgorithms.java
                                                         1     1     0     0
    jtreg:test/jdk/sun/security/pkcs11/Cipher/TestDisabledAlgorithms.java
                                                         1     1     0     0
    jtreg:test/jdk/sun/security/pkcs11/Signature/TestNONEwithRSA.java
                                                         1     1     0     0
    jtreg:test/jdk/sun/security/util/AlgorithmConstraints/InvalidCryptoDisabledAlgos.java
                                                         1     1     0     0
==============================

The failing test test/jdk/java/security/KeyStore/TestDisabledAlgorithms.java will be fixed in follow up backport of JDK-8373690.

sun/security - MIXXED

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
>> jtreg:test/jdk/sun/security                         677   676     1     0 <<
==============================
TEST FAILURE

The failing test is sun/security/ssl/X509KeyManager/PreferredKey.java, afaik this unrelated to this backport and is being addressed with another PR.

Tier 1 - PASSED

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
   jtreg:test/hotspot/jtreg:tier1                     1497  1497     0     0
   jtreg:test/jdk:tier1                               1900  1900     0     0
   jtreg:test/langtools:tier1                         3941  3941     0     0
   jtreg:test/nashorn:tier1                              0     0     0     0
   jtreg:test/jaxp:tier1                                 0     0     0     0
==============================
TEST SUCCESS

gtest - PASSED

==============================
Test summary
==============================
   TEST                                              TOTAL  PASS  FAIL ERROR
   gtest:all/server                                    507   507     0     0
==============================
TEST SUCCESS

GHA - RUNNING

https://github.com/TheMangovnik/jdk11u-dev/actions/runs/26746034005

---

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• JDK-8244336 needs maintainer approval
• Commit message must refer to an issue
• Change requires CSR request JDK-8372102 to be approved
• Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issues

• JDK-8244336: Restrict algorithms at JCE layer (Enhancement - P3)
• JDK-8372102: Restrict algorithms at JCE layer (CSR)

Reviewers

• Severin Gehwolf (@jerboaa - Reviewer) 🔄 Re-review required (review applies to b56b2398)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk11u-dev.git pull/3213/head:pull/3213
$ git checkout pull/3213

Update a local copy of the PR:
$ git checkout pull/3213
$ git pull https://git.openjdk.org/jdk11u-dev.git pull/3213/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 3213

View PR using the GUI difftool:
$ git pr show -t 3213

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk11u-dev/pull/3213.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back TheMangovnik! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@TheMangovnik Please do not rebase or force-push to an active PR as it invalidates existing review comments. Note for future reference, the bots always squash all changes into a single commit automatically as part of the integration. See OpenJDK Developers’ Guide for more information.

[openjdk]

This backport pull request has now been updated with issue from the original commit.

[mlbridge]

Webrevs

• 02: Full - Incremental (1fb4eb5f)
• 01: Full - Incremental (b56b2398)
• 00: Full (7cff2ccf)

[jerboaa]

/reviewers 2

[openjdk]

@jerboaa
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

[jerboaa]

This looks OK to me. Two nits.

[jerboaa]

Nit: The JDK 17u patch has this on two lines:

Suggested change

	throw new NoSuchAlgorithmException(transformation + " is disabled");
	throw new NoSuchAlgorithmException(transformation +
	" is disabled");

[jerboaa]

Nit: Odd formatting that is different to the JDK 17u patch:

Suggested change

	Set<String> algorithmsInPropertySet = new
	TreeSet<>(String.CASE_INSENSITIVE_ORDER);
	Set<String> algorithmsInPropertySet =
	new TreeSet<>(String.CASE_INSENSITIVE_ORDER);

[jerboaa]

OK

[gnu-andrew]

• Differences in Keystore.java are due to the absence of JDK-8245679 and are adapted appropriately. 8245679 itself looks a risky backport as it changes Provider lookup behaviour.
• I don't really see the need to move the declaration of MessageDigest md in either patch, but at least in 17, it is moved after the new block which could cause the method to exit early. I don't see any reason to move it in 11u as the context is quite different.
• MessageDigest.java has a missing newline in 11u compared with 17u before if (provider == null)
• There are differences in Signature.java but this is in erased code so no issue
• Cipher.java still contains differences in the indentation of NoSuchAlgorithmException construction as pointed out by Severin. The first where correction was attempted is not indented correctly. The second instance has not been corrected at all.
• AbstractAlgorithmConstraints.java also has bad indentation of the TreeSet line raised by Severin. The first part of the patch also looks different, but this is because 11u never had the SuppressWarnings annotation added; changes are otherwise the same.
• In CryptoAlgorithmConstraints.java, the correction to debug.println should be a change to Debug.println rather than changing the arguments. The method exists in 11u but is static, rather than an instance method. Changing it to call the static version will mean that, if the dual argument method is changing to an instance method at a later date, this will be caught by a compile error. If you change the arguments as in the current patch, this difference will remain unnoticed. The change is made by JDK-8051959 which adds thread info and timestamps to the output. This has already been proposed for 11u, but was blocked by the need for an earlier change, so I suspect this is likely to be integrated at some point.

You can see the differences I'm referring to by running diff -u on your patch vs. the 17u patch. I would strongly suggest doing this for backports so you are aware of differences and can catch ones that shouldn't be there before submitting the patch for review.

[TheMangovnik]

• Differences in Keystore.java are due to the absence of JDK-8245679 and are adapted appropriately. 8245679 itself looks a risky backport as it changes Provider lookup behaviour.

👍

---

• I don't really see the need to move the declaration of MessageDigest md in either patch, but at least in 17, it is moved after the new block which could cause the method to exit early. I don't see any reason to move it in 11u as the context is quite different.

tl;dr: I see the move of the MessageDigest md, but I do not see how it is different between the jdk11 and jdk17 patch.

I have been peering into the patches, both in jdk17 and in this PR, and I fail to fully understand what you are referencing.

• I see that in both patches declaration of MessageDigest md moved in file MessageDigest.java.
• I see that in both patches this occurred in the method public static MessageDigest getInstance(String algorithm)
• I see that in both patches declaration of MessageDigest md swapped places with the already present command below it. Functionally doing what is in the following code block. I remember looking at this when doing this backport and wondering about this change too, but I have decided to keep this in since it should not change anything anyway.

- MessageDigest md;
Object[] objs = Security.getImpl(algorithm, "MessageDigest",
                                             (String)null);
+ MessageDigest md;

• I see that in both patches the declaration of MessageDigest md is after the newly introduced block, resulting in the code snipped below in both jdk17 and jdk11:

if (!CryptoAlgorithmConstraints.permits("MessageDigest", algorithm)) {
     throw new NoSuchAlgorithmException(algorithm + " is disabled");
}

// In this place also starts a try block in jdk11. But that should be inconsequential for this situation.

Object[] objs = Security.getImpl(algorithm, "MessageDigest",
                                             (String)null);
MessageDigest md;

• So as far as I understand -> the declaration is always after the new block, the move of the declaration is relative to command below it. If I understand your comment correctly, you are saying that the previous statement is not true?

I see similar castling in the method overload just below it public static MessageDigest getInstance(String algorithm, String provider). In jdk11 patch there is no change in the position of the declaration of MessageDigest md, since the declaration itself lives in the if block in this method. However there is same position change of the declaration in jdk17 patch, because the declaration is factored out of the if block. But even here the declaration is after the newly introduced block in all cases.

---

• MessageDigest.java has a missing newline in 11u compared with 17u before if (provider == null)

Added.

---

• There are differences in Signature.java but this is in erased code so no issue

👍

---

• Cipher.java still contains differences in the indentation of NoSuchAlgorithmException construction as pointed out by Severin. The first where correction was attempted is not indented correctly. The second instance has not been corrected at all.

This is stupid oversight from my side. Corrected.

---

• AbstractAlgorithmConstraints.java also has bad indentation of the TreeSet line raised by Severin. The first part of the patch also looks different, but this is because 11u never had the SuppressWarnings annotation added; changes are otherwise the same.

Indentation corrected.

---

• In CryptoAlgorithmConstraints.java, the correction to debug.println should be a change to Debug.println rather than changing the arguments. The method exists in 11u but is static, rather than an instance method. Changing it to call the static version will mean that, if the dual argument method is changing to an instance method at a later date, this will be caught by a compile error. If you change the arguments as in the current patch, this difference will remain unnoticed. The change is made by JDK-8051959 which adds thread info and timestamps to the output. This has already been proposed for 11u, but was blocked by the need for an earlier change, so I suspect this is likely to be integrated at some point.

I see, the fact that the Debug.println() is within a if block that checks for an instance of the Debug class confused me here. Nevertheless, it is corrected to the static variant Debug.println() now.

---

You can see the differences I'm referring to by running diff -u on your patch vs. the 17u patch. I would strongly suggest doing this for backports so you are aware of differences and can catch ones that shouldn't be there before submitting the patch for review.

I will keep this in mind and do more throughout job next time.