First 16 bytes corrupted while decrypting using AES

Question

I encoded a text file using openssl as follows:

openssl enc -nosalt -aes-128-cbc -k mypass -in "test.txt" -out "test_enc.txt" -p

and it returns the key and iv as follows:

key=A029D0DF84EB5549C641E04A9EF389E5
iv =A10CE9C4682486F8622F2F18E7291367

and here is the code I used to decryp the file:

 int main() {

      streampos size;
      char * indata;

      ifstream file ("test_enc.txt", ios::in|ios::binary|ios::ate);
      ofstream outfile ("test_decoded.txt",std::ofstream::binary);

      if (file.is_open())
      {
        size = file.tellg();

        indata = new char [size];
        file.seekg (0, ios::beg);
        file.read (indata, size);

        file.close();

        unsigned char* outdata=new unsigned char [size];

        unsigned char ckey[] = "\xA0\x29\xD0\xDF\x84\xEB\x55\x49\xC6\x41\xE0\x4A\x9E\xF3\x89\xE5";
        unsigned char ivec[] = "\xA1\x0C\xE9\xC4\x68\x24\x86\xF8\x62\x2F\x2F\x18\xE7\x29\x13\x67";

        AES_KEY key;

        AES_set_decrypt_key(ckey, 256, &key);

        AES_cbc_encrypt((unsigned char*) indata, outdata, size, &key, ivec, AES_DECRYPT);


        outfile.write ((char*) outdata,size);

        delete[] indata;
        delete[] outdata;
      }
      else{
          cout << "Unable to open file";
          cerr << "Error: " << strerror(errno);
      }
      outfile.close();
      file.close();
      return 0;
}

This code works perfectly. However, when I use salt while encoding as in the following command:

openssl enc -aes-128-cbc -k mypass -in "test.txt" -out "test_enc.txt" -p

and appropriately replace the key and ivec in the code, the whole file is correctly decrypted but the first 16 bytes! What I have learnt from other posts with similar issues, I know that the iv value is wrong, but I don't know what should be the correct iv value. I am using only the key and iv value return after encryption and I am also not taking into account the salt value (actually I don't know how to). What should be the correct iv value?

Answer 1

From your symptoms of problem, it appears to me that your following logic might be releasing the memory which later code in your program is reading it. This results into this situation. Normally it happens because after releasing the memory, heap manager typically writes some housekeeping information for their own usage.

The point is we should not use the memory after freeing it as it is dangling.

//Allocate the memory
unsigned char* outdata=new unsigned char [size];

// It appears to me that here the outdata is getting freed. I do not see anywhere
// else this memory is getting freed.
AES_cbc_encrypt((unsigned char*) indata, outdata, size, &key, ivec, AES_DECRYPT);
// Due to this, while accessing it we are seeing the 16 bytes corrupted.
outfile.write ((char*) outdata,size);

You can use any dynamic tool to identify such problem.