Generating, Signing and Verifying Digital Signature

Question

So here's a question from my project.

In this task, we will use OpenSSL to generate digital signatures. Please prepare a file (example.txt) of any size. Also prepare an RSA public/private key pair. Then do the following:

1. Sign the SHA256 hash of example.txt; save the output in example.sha256.
2. Verify the digital signature in example.sha256.
3. Slightly modify example.txt, and verify the digital signature again.

Please describe how you performed the above three operations (e.g., the exact commands that you used, etc.). Describe what you observed and explain your observations. Please also explain why digital signatures are useful in general.

So, I do the following.

1.Create private/public key pair

openssl genrsa -out private.pem 1024

2. Extracting Public key.

openssl rsa -in private.pem -out public.pem -outform PEM -pubout

3. Create hash of the data.

echo 'data to sign' > example.txt

openssl dgst -sha256 < example.txt > hash

4. Sign the hash using Private key to a file called example.sha256

openssl rsautl -sign -inkey private.pem -keyform PEM -in hash  > example.sha256

5. Verify the file (example.txt)and the digital signature (example.sha256)

openssl dgst -sha256 -verify public.pem -signature example.sha256 example.txt

After doing all this, I get an error message saying "Verification Failure"

Please correct me if I went wrong somewhere.

Answer 1

Don’t use rsautl for this.

According to PKCS1.5, when signing the format of the data that goes into the RSA operation looks something like this:

<padding><metadata><hash of input>

(The metadata specifies which hash function has been used.)

This format is what openssl dgst -verify is looking for when you try to verify the signature. However this is not what you create in your steps.

First of all the default output of openssl dgst is the hex encoding of the resulting hash, not the raw bytes.

Secondly, rsautl is fairly “low level”, and when signing doesn’t add the metadata that openssl dgst -verify is expecting, although it does add the padding.

These two things together mean that the data you are using looks like this:

<padding><hex digits of hash of input>

Obviously this doesn’t match what openssl dgst -verify is expecting, so the verification fails.

It would be possible to create a correctly formatted input for rsautl, but it would be awkward and involve dealing with ASN.1 details. You could also use rsautl -verify instead of dgst -verify, but that would also require a few more details and would mean you are using a non-standard signature format.

The simplest solution is to use openssl dgst for both the creation and verification of the signature. Replace your steps 3 and 4 (except for creating the example.txt file) with the single command:

$ openssl dgst -sha256 -sign private.pem -out example.sha256 example.txt

This hashes the data, correctly formats the hash and performs the RSA operation it. The resulting file should correctly verify with the openssl dgst -verify command.

Answer 2

Thanks to Matt for the solution. minor nit : OP seems to want to sign the hash rather than the actual entire data (also something I am looking to do). So use below to generate the signature:

openssl dgst -sha256 -sign private.pem -out hash.sig hash

And below to verify the signature

openssl dgst -sha256 -verify public.pem -signature hash.sig hash

And finally something like below to verify integrity of the payload :

echo "`hash` example.txt" | sha256sum -c --strict