Bouncy castle gives unknown HashAlgorithm

Question

I am trying to use bouncy castle for DTLS Handshake.
I have generated key by following this link. I am working by extending DefaultTlsClient. It can generate client_hello packet. But when the server_hello packet arrives it gives org.bouncycastle.crypto.tls.TlsFatalAlert: internal_error(80) Caused by: java.lang.IllegalArgumentException: unknown HashAlgorithm. Can anyone give any hint?

Update:
From Wireshark: In the Certificate Request, there are 9 Signature Hash Algorithms. One of them is rsa_pss_sha256(0x0804). In the public static Digest createHash(short hashAlgorithm) function in TlsUtils.java there is no matching for it. That's why it is giving Unknown hash Algorithm. What does that mean? Using Bouncy Castle, is it possible to establish DTLS with that server?

Here is the code for loading the keystore:

    public static void initKeyStore() {
    char password[] = "bbtone".toCharArray();
    if( !isKeystoreLoaded) {
     try {
        FileInputStream fis = new FileInputStream("bbtone");
        KeyMgmt key = new KeyMgmt();
        key.open(fis, password);
        fis.close();
        crt = key.getCRT("bbtone").getEncoded();
        fingerprintSHA256 = KeyMgmt.fingerprintSHA256(crt);
        ArrayList<byte[]> chain = new ArrayList<byte[]>();
        chain.add(crt);
        java.security.cert.Certificate root = key.getCRT("root");
        if (root != null) {
          chain.add(root.getEncoded());
        }
        privateKey = key.getKEY("bbtone", password).getEncoded();
        initDTLS(chain, privateKey, false);

        isKeystoreLoaded = true;
      } catch(FileNotFoundException e) {
          e.printStackTrace();
      } catch(Exception e) {
          e.printStackTrace();
      }
    }
}

Generating private key and certificate:

      public static boolean initDTLS(java.util.List<byte []> certChain, byte privateKey[], boolean pkRSA) {
    try {
      org.bouncycastle.asn1.x509.Certificate x509certs[] = new org.bouncycastle.asn1.x509.Certificate[certChain.size()];
      for (int i = 0; i < certChain.size(); ++i) {
        x509certs[i] = org.bouncycastle.asn1.x509.Certificate.getInstance(certChain.get(i));
      }
      dtlsCertChain = new org.bouncycastle.crypto.tls.Certificate(x509certs);
      if (pkRSA) {
        RSAPrivateKey rsa = RSAPrivateKey.getInstance(privateKey);
        dtlsPrivateKey = new RSAPrivateCrtKeyParameters(rsa.getModulus(), rsa.getPublicExponent(),
                rsa.getPrivateExponent(), rsa.getPrime1(), rsa.getPrime2(), rsa.getExponent1(),
                rsa.getExponent2(), rsa.getCoefficient());
            } else {
              dtlsPrivateKey = PrivateKeyFactory.createKey(privateKey);
            }
      return true;
    } catch (Exception e) {
      return false;
    }
  }

Starting DTLS Handshake:

  public void startDTLS() {

        socket.connect(recvPacket.getAddress(), recvPacket.getPort());
        try {
            dtlsClient = new DTLSClientProtocol(new SecureRandom());
          } catch (Exception e) {
            e.printStackTrace();
            dtlsClient = null;
            return;
          }
        tlsClient = new DefaultTlsClient2() {
            protected TlsSession session;
            public TlsSession getSessionToResume()
            {
              return this.session;
            }

            public ProtocolVersion getClientVersion() {
              return ProtocolVersion.DTLSv12;
            }

            public ProtocolVersion getMinimumVersion() {
              return ProtocolVersion.DTLSv10;
            }

            public Hashtable getClientExtensions() throws IOException {
              //see : http://bouncy-castle.1462172.n4.nabble.com/DTLS-SRTP-with-bouncycastle-1-49-td4656286.html
              logger.debug("Extending getClientExtensions\n");
              Hashtable table = super.getClientExtensions();
              if (table == null) table = new Hashtable();
              //adding the protection profiles

              int[] protectionProfiles = {
                SRTPProtectionProfile.SRTP_AES128_CM_HMAC_SHA1_80  //this is the only one supported for now
  //             SRTPProtectionProfile.SRTP_AES128_CM_HMAC_SHA1_32
  //              SRTPProtectionProfile.SRTP_NULL_HMAC_SHA1_32
  //              SRTPProtectionProfile.SRTP_NULL_HMAC_SHA1_80
              };
              byte mki[] = new byte[0];  //do not use mki
              UseSRTPData srtpData = new UseSRTPData(protectionProfiles, mki);
              TlsSRTPUtils.addUseSRTPExtension(table, srtpData);
              return table;
            }

            public TlsAuthentication getAuthentication() throws IOException {
              return new TlsAuthentication() {
                public void notifyServerCertificate(org.bouncycastle.crypto.tls.Certificate serverCertificate)
                    throws IOException
                {
                  //info only
                }

                public TlsCredentials getClientCredentials(CertificateRequest certificateRequest)
                    throws IOException
                {
                  short[] certificateTypes = certificateRequest.getCertificateTypes();
                  if (certificateTypes == null) return null;
                  boolean ok = false;
                  for(int a=0;a<certificateTypes.length;a++) {
                    if (certificateTypes[a] == ClientCertificateType.rsa_sign) {
                      ok = true;
                      break;
                    }
                  }
                  if (!ok) return null;

                  SignatureAndHashAlgorithm signatureAndHashAlgorithm = null;
                  Vector sigAlgs = certificateRequest.getSupportedSignatureAlgorithms();
                  if (sigAlgs != null)
                  {
                    for (int i = 0; i < sigAlgs.size(); ++i)
                    {
                      SignatureAndHashAlgorithm sigAlg = (SignatureAndHashAlgorithm) sigAlgs.elementAt(i);
                      if (sigAlg.getSignature() == SignatureAlgorithm.rsa)
                      {
                        signatureAndHashAlgorithm = sigAlg;
                        break;
                      }
                    }

                    if (signatureAndHashAlgorithm == null)
                    {
                      return null;
                    }
                  }

                  return new DefaultTlsSignerCredentials(context, dtlsCertChain, dtlsPrivateKey, signatureAndHashAlgorithm);
                }
              };
            }
            public void notifyHandshakeComplete() throws IOException
            {
              logger.debug("SRTPChannel:DTLS:Client:Handshake complete");
              super.notifyHandshakeComplete();

              TlsSession newSession = context.getResumableSession();
              if (newSession != null)
              {
                this.session = newSession;
              }
              getKeys();
            }
          };  
          try {
              logger.debug("SRTPChannel:connecting to DTLS server");
              dtlsTransport = dtlsClient.connect(tlsClient, new UDPTransport(socket, 1500 - 20 - 8));
           } catch (Exception e) {
              e.printStackTrace();
           }
    }

Error:

    org.bouncycastle.crypto.tls.TlsFatalAlert: internal_error(80)
    at org.bouncycastle.crypto.tls.DTLSClientProtocol.connect(DTLSClientProtocol.java:75)
    at processor.ClientMediaHandler.startDTLS(ClientMediaHandler.java:459)
    at processor.ClientMediaHandler.run(ClientMediaHandler.java:538)
Caused by: java.lang.IllegalArgumentException: unknown HashAlgorithm
    at org.bouncycastle.crypto.tls.TlsUtils.createHash(TlsUtils.java:1184)
    at org.bouncycastle.crypto.tls.DeferredHash.checkTrackingHash(DeferredHash.java:203)
    at org.bouncycastle.crypto.tls.DeferredHash.trackHashAlgorithm(DeferredHash.java:68)
    at org.bouncycastle.crypto.tls.TlsUtils.trackHashAlgorithms(TlsUtils.java:1358)
    at org.bouncycastle.crypto.tls.DTLSClientProtocol.clientHandshake(DTLSClientProtocol.java:241)
    at org.bouncycastle.crypto.tls.DTLSClientProtocol.connect(DTLSClientProtocol.java:60)
    ... 2 more

Answer 1

I have solved the problem using DTLS1.0. It can now finish the handshake.

I replaced

public ProtocolVersion getClientVersion() {
  return ProtocolVersion.DTLSv12;
}

with the following code:

public ProtocolVersion getClientVersion() {
  return ProtocolVersion.DTLSv10;
}

Answer 2

@Rashed, downgrading TLS version is not a good choice. There is already comment from @bbaldino but I think it should be added as answer - version 1.60 of bouncy castle has a bug. Instead of downgrading TLS, you should upgrade bouncy castle to at least 1.61

https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk15on