double free or corruption (!prev) while calling __do_global_dtors_aux

Question

I'm getting this error message after my app has done everything right

/lib64/libc.so.6[0x3f1ee70d7f]
/lib64/libc.so.6(cfree+0x4b)[0x3f1ee711db]
/home/user/workspace/NewProject/build/bin/TestApp(_ZN9__gnu_cxx13new_allocatorIN5boost10shared_ptrINS1_5uuids4uuidEEEE10deallocateEPS5_m+0x20)[0x49c174]
/home/user/workspace/NewProject/build/bin/TestApp(_ZNSt12_Vector_baseIN5boost10shared_ptrINS0_5uuids4uuidEEESaIS4_EE13_M_deallocateEPS4_m+0x32)[0x495b84]
/home/user/workspace/NewProject/build/bin/TestApp(_ZNSt12_Vector_baseIN5boost10shared_ptrINS0_5uuids4uuidEEESaIS4_EED2Ev+0x47)[0x49598b]
/home/user/workspace/NewProject/build/bin/TestApp(_ZNSt6vectorIN5boost10shared_ptrINS0_5uuids4uuidEEESaIS4_EED1Ev+0x65)[0x48bf27]
/lib64/libc.so.6(__cxa_finalize+0x8e)[0x3f1ee337fe]
/home/user/workspace/NewProject/build/components/lib_path/libhelper-d.so[0x2aaaab052b36]

If I run the program in gdb I can get the following backtrace, but it is all I get:

#0  0x0000003f1ee30285 in raise () from /lib64/libc.so.6
#1  0x0000003f1ee31d30 in abort () from /lib64/libc.so.6
#2  0x0000003f1ee692bb in __libc_message () from /lib64/libc.so.6
#3  0x0000003f1ee70d7f in _int_free () from /lib64/libc.so.6
#4  0x0000003f1ee711db in free () from /lib64/libc.so.6
#5  0x000000000049c174 in __gnu_cxx::new_allocator<boost::shared_ptr<boost::uuids::uuid> >::deallocate (this=0x2aaaab2cea50, __p=0x1cfd8d0)
    at /opt/local/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.4.5/../../../../include/c++/4.4.5/ext/new_allocator.h:95
#6  0x0000000000495b84 in std::_Vector_base<boost::shared_ptr<boost::uuids::uuid>, std::allocator<boost::shared_ptr<boost::uuids::uuid> > >::_M_deallocate (
    this=0x2aaaab2cea50, __p=0x1cfd8d0, __n=8) at /opt/local/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.4.5/../../../../include/c++/4.4.5/bits/stl_vector.h:146
#7  0x000000000049598b in std::_Vector_base<boost::shared_ptr<boost::uuids::uuid>, std::allocator<boost::shared_ptr<boost::uuids::uuid> > >::~_Vector_base (
    this=0x2aaaab2cea50, __in_chrg=<value optimized out>)
    at /opt/local/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.4.5/../../../../include/c++/4.4.5/bits/stl_vector.h:132
#8  0x000000000048bf27 in std::vector<boost::shared_ptr<boost::uuids::uuid>, std::allocator<boost::shared_ptr<boost::uuids::uuid> > >::~vector (this=0x2aaaab2cea50,
    __in_chrg=<value optimized out>) at /opt/local/bin/../lib/gcc/x86_64-unknown-linux-gnu/4.4.5/../../../../include/c++/4.4.5/bits/stl_vector.h:313
#9  0x0000003f1ee337fe in __cxa_finalize () from /lib64/libc.so.6
#10 0x00002aaaab052b36 in __do_global_dtors_aux ()
from /home/user/workspace/NewProject/build/components/lib_path/libhelper-d.so
#11 0x0000000000000000 in ?? ()

I really have no idea of how to proceed from here.

UPDATE I forgot to mention that the only global variable of the type which appears in the error is cleared m_uuids.size() == 0 by the time the error appear.

Answer 1

I had this same problem using glog. In my case, it was this scenario:

1. I had a share library, call it 'common.so' that linked glog.
2. My main executable, call it 'app' also linked glog, and linked in common.so.

The problem I had was that glog was linked statically in both the .so and the exectuable. When I changed both #1 and #2 to link the .so instead of the .a, the problem went away.

Not sure this is your problem, but it could be. Generally speaking, corruption when freeing up memory often means that you corrupted the memory pool (such as deleting the same pointer twice). I believe linking in the .a in both cases, I was getting cleanup behavior on the same global pointer (an std::string in my case) twice.

Update: After much investigation, this is very likely the problem. What happens is that each the executable and the .so have a global variable of std::string type (part of glog). These std::string global variables must be constructed when the object (exe, .so) is loaded by the dynamic linker/loader. Also, a destructor for each is added for cleanup using at_exit. However, when it comes time for at_exit functions to be called, both global reference point to the same std::string. That means the std::string destructor is called twice, but on the same object. Then free is called on the same memory location twice. Global std::string (or any class with a constructor) are a bad idea. If you choose to have a .so based architecture (a good idea), you have to be careful with all 3rd party libraries and how they handle globals. You stay out of most danger by linking to the .so for all 3rd party libraries.