test_pkey.rb: Add PQC algorithm SLH-DSA (FIPS 205) test

[junaruga]

I want to add SLH-DSA (FIPS 205) test to ruby/openssl to make sure SLH-DSA works in Ruby OpenSSL.

Commit message

Add PQC algorithm SLH-DSA (FIPS 205) test as well as existing ML-KEM (FIPS 203) and ML-DSA (FIPS 204) in test/openssl/test_pkey.rb.

https://csrc.nist.gov/projects/post-quantum-cryptography

OpenSSL 3.5.0 supports PQC algorithm SLH-DSA. It seems LibreSSL and AWS-LC don't support SLH-DSA yet.

https://openssl-library.org/post/2025-04-08-openssl-35-final-release/

Note we don't add SLH-DSA tests in test/openssl/test_ssl.rb. SLH-DSA is used for the signature case like ML-DSA. But SLH-DSA's signature size is larger than ML-DSA's. So, I am not sure that SLH-DSA is used in SSL/TLS signature case.

The used SLH-DSA parameter set in test_slh_dsa is SLH-DSA-SHA2-128s which is one of the total 12 parameter sets. This is the lowest security strength (security category 1) with small key and signature sizes.

This aligns with ML-DSA-44 used in test_ml_dsa. ML-DSA-44 also has the lowest security strength (security category 2) in the 3 ML-DSA parameter sets, ML-DSA-44, ML-DSA-65, ML-DSA-87.

References:

• SLH-DSA parameter sets: FIPS 205 Section 11 Parameter Sets - https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.205.pdf
• ML-DSA parameter sets: FIPS 204 Section 4 Parameter Sets - https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.204.pdf

Assisted-by: Claude:Opus 4.6

[rhenium]

Does SLH-DSA require any different handling from the other signature algorithms in the OpenSSL C API? If SLH-DSA works through the same generic EVP_PKEY paths, I feel these tests would be mostly duplicate.

It also seems rather slow: https://github.com/ruby/openssl/actions/runs/35600244714/job/106334376765?pr=1113#step:10:61

ML-DSA was interesting because it was one of the first algorithms to not have an NID and required special treatment (we changed #inspect and .new_raw_*_key).

[junaruga]

Does SLH-DSA require any different handling from the other signature algorithms in the OpenSSL C API? If SLH-DSA works through the same generic EVP_PKEY paths, I feel these tests would be mostly duplicate.

I see some small differences for example, between ML-DSA and SLH-DSA, while both ML-DSA and SLH-DSA work with the EVP_PKEY_* functions. I will show you the differences later soon, and I want you to judge if we need SLH-DSA test or not.

I would share the manual documents for SLH-DSA.

SLH-DSA
https://docs.openssl.org/master/man7/EVP_PKEY-SLH-DSA/
https://docs.openssl.org/master/man7/EVP_SIGNATURE-SLH-DSA/

It also seems rather slow: https://github.com/ruby/openssl/actions/runs/35600244714/job/106334376765?pr=1113#step:10:61

This is expected. The following document says as follows.

https://docs.openssl.org/master/man7/EVP_PKEY-SLH-DSA/

SLH-DSA (Stateless Hash-based Digital Signature Standard) uses small keys, but has relatively large signatures and is relatively slow performing all operations compared to ML-DSA. It does however have proven security proofs, since it relies only on hash functions.

However, we can improve the testing speed if we want. I used SLH-DSA-SHA2-128s in test_slh_dsa. The "s" in SLH-DSA-SHA2-128s means small sized signatures, but it is slow signature generation. The another parameter set SLH-DSA-SHA2-128f, whose "f" means fast signature generation. I chose SLH-DSA-SHA2-128s rather than SLH-DSA-SHA2-128f. Because I wanted to show the parameter set which can be more used practically due to small sized signature through the testing code, to help users.

FIPS 205 Section 11 Parameter Set says as follows as a reference.

whether the parameter set was designed to create relatively small signatures (‘s’) or to have relatively fast signature generation (‘f’)

I compared the testing speed between SLH-DSA-SHA2-128s and SLH-DSA-SHA2-128f with the following change:

diff --git a/test/openssl/test_pkey.rb b/test/openssl/test_pkey.rb
index 3cf3a32..51ee671 100644
--- a/test/openssl/test_pkey.rb
+++ b/test/openssl/test_pkey.rb
@@ -290,7 +290,7 @@ class OpenSSL::TestPKey < OpenSSL::PKeyTestCase
     assert_equal(true, pub3.verify(nil, sig, "data"))
   end

-  def test_slh_dsa
+  def test_slh_dsa_sha2_128s
     # SLH-DSA (FIPS 205) is supported on OpenSSL 3.5 or later.
     return unless openssl?(3, 5, 0)

@@ -313,6 +313,29 @@ class OpenSSL::TestPKey < OpenSSL::PKeyTestCase
     assert_equal(true, pub3.verify(nil, sig, "data"))
   end

+  def test_slh_dsa_sha2_128f
+    # SLH-DSA (FIPS 205) is supported on OpenSSL 3.5 or later.
+    return unless openssl?(3, 5, 0)
+
+    pkey = OpenSSL::PKey.generate_key("SLH-DSA-SHA2-128f")
+    assert_match(/type_name=SLH-DSA-SHA2-128f/, pkey.inspect)
+    sig = pkey.sign(nil, "data")
+    # See FIPS 205 Section 11 Parameter Sets - Table 2. SLH-DSA parameter sets -
+    # row: SLH-DSA-SHA2-128s, column: sig bytes
+    assert_equal(17088, sig.bytesize)
+    assert_equal(true, pkey.verify(nil, sig, "data"))
+
+    pub2 = OpenSSL::PKey.read(pkey.public_to_der)
+    assert_equal(true, pub2.verify(nil, sig, "data"))
+
+    raw_public_key = pkey.raw_public_key
+    # See FIPS 205 Section 11 Parameter Sets - Table 2. SLH-DSA parameter sets -
+    # row: SLH-DSA-SHA2-128s, column: pk bytes
+    assert_equal(32, raw_public_key.bytesize)
+    pub3 = OpenSSL::PKey.new_raw_public_key("SLH-DSA-SHA2-128f", raw_public_key)
+    assert_equal(true, pub3.verify(nil, sig, "data"))
+  end
+
   def test_ml_kem
     # EVP_PKEY KEM APIs were added in OpenSSL 3.0.
     omit "ML-KEM is not supported" unless openssl?(3, 5, 0)

And the result is below. test_slh_dsa_sha2_128f is 2x slower than test_ml_dsa. But test_slh_dsa_sha2_128f is x18 faster than test_slh_dsa_sha2_128s.

$ ruby -I ./lib/ test/openssl/test_pkey.rb -n /test_.*_dsa.*/ -v
Loaded suite test/openssl/test_pkey
Started
OpenSSL::TestCase:
  OpenSSL::PKeyTestCase:
    OpenSSL::TestPKey:
      test_ml_dsa:																							.: (0.010275)
      test_slh_dsa_sha2_128f:																						.: (0.022549)
      test_slh_dsa_sha2_128s:																						.: (0.405287)

Finished in 0.441391845 seconds.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
3 tests, 22 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
6.80 tests/s, 49.84 assertions/s

ML-DSA was interesting because it was one of the first algorithms to not have an NID and required special treatment (we changed #inspect and .new_raw_*_key).

Ah I see. Now I understood that's why test_slh_dsa has OpenSSL::PKey::PKey#inspect and OpenSSL::PKey.new_raw_public_key. I created test_slh_dsa based on the content of test_ml_dsa. But the content can be different if we still need test_slh_dsa.

[junaruga]

I will show you the differences later soon, and I want you to judge if we need SLH-DSA test or not.

First, I haven't seen any unique features in SLH-DSA in my investigation. There are some differences of features between for example SLH-DSA and ML-DSA. However, these features are what ML-DSA has, and what SLH-DSA don't have. ML-DSA has features that SLH-DSA has.

Below are ML-DSA features that SLH-DSA don't have:

• ML-DSA supports seed parameter (EVP_PKEY-ML-DSA(7))
• ML_DSA supports mu parameter (EVP_SIGNATURE-ML-DSA(7)).
• ML-DSA supports streaming signing with EVP_PKEY_sign_message_update(3) in current OpenSSL master branch. As Ruby OpenSSL doesn't implement the Ruby API for EVP_PKEY_sign_message_update, this is not related for now.

I prepared my scripts to show these feature below as a reference. I generated the C program by Claude Code, and code comments are messy with source file and lines for my local OpenSSL source tree.

https://github.com/junaruga/report-ml-dsa-vs-slh-dsa

So, I concludes that we don't need test_slh_dsa.

What do you think?