VLD stack overflow at app exit

Apr 25, 2011 at 6:29 PM
Edited Apr 26, 2011 at 10:17 AM

I have a ridiculous issue that I appear to be unable to isolate into a simple repro case. Please forgive me for not providing one; it's not for the want of trying, believe me.

I have a .vcproj as part of a wider solution with a single .cpp file consisting of the following:

#include <vld.h>

int main(int argc, char **argv)
{
	return 0;
}

This is crashing with a stack overflow at exit, in VLD 1.9h and 2.1, unless I either:

  1. Remove the #include <vld.h>, thus removing leak detection
  2. Change the Output File in the vcproj properties from $(OutDir)\Debug-$(ProjectName).exe back to the default $(OutDir)\$(ProjectName).exe.
  3. Rename the generated .exe in Explorer from Debug-MyProject.exe to simply MyProject.exe.

I don't understand why the filename of the executable is making such a big difference. I can take the crashing version and make it not crash by renaming the executable, and vice versa. I've even tried deleting .pdb files (as well as ensuring they match) but this seems to make no difference. I've tried to repro this in a test solution with a fresh vcproj but it works fine, even when copying the vcproj settings from the "broken" project to the test one. No repro.

I'm at my wit's end here. The vcproj doesn't have any dependencies on any external libs or other projects in the solution. All the settings - including the directory structure (bar renames) - are the same between the crashing solution and the test case that fails to repro. I've even tried removing all other projects from the solution that crashes, to no avail.

I could post the callstack but since I don't have symbols for VLD I'm not sure if this would be any use. Would it be worth recompiling VLD from source in order to give better details? Or perhaps the problem is obvious to a more trained eye, in which case I'd be very grateful for any advice.

I'm on VS2005, Win7 x64, compiling for x86.

$(OutDir)\Debug-$(ProjectName).exe
Apr 27, 2011 at 7:54 PM

OK, so I've managed to reproduce this in the test project by generating an executable with the same name as the one in the original project. Any other variations of the name seem to work fine. Since it seems to be tied directly to the name of the executable, is it possible that this is some kind of hash collision issue?

Developer
Apr 27, 2011 at 10:00 PM

So, if naming your binary Debug-xxx.exe is causing problems, then don't do it. 

As a general rule, I never rename binaries based off of their build configuration. I also leave my project output directory settings like this:

$(ProjectDir)$(Platform)\$(Configuration)\

which ensures that my 4 different exe files are never in the same directory.

Apr 27, 2011 at 10:13 PM

So your advice is "if you have a solid repro case for a crash bug, then stop reproing it"?

I don't quite know what to say to that.

Apr 27, 2011 at 10:46 PM
Edited Apr 27, 2011 at 10:54 PM

I noticed that KindDragon checked in a fix for a rare crash in SVN so I grabbed latest source and recompiled, but no change. Still, I now have a callstack for the stack overflow:

ntdll.dll!@RtlpAllocateHeap@24()  + 0xa bytes
ntdll.dll!_RtlAllocateHeap@12()  + 0x502a bytes   
ntdll.dll!_RtlDebugAllocateHeap@12()  + 0xb5 bytes   
ntdll.dll!@RtlpAllocateHeap@24()  + 0x57c17 bytes   
ntdll.dll!_RtlAllocateHeap@12()  + 0x502a bytes   
vld_x86.dll!VisualLeakDetector::_RtlAllocateHeap(void * heap=0x05ab0000, unsigned long flags=16781314, unsigned long size=88)  Line 2586 + 0x18 bytes    C++
verifier.dll!_AVrfpDphNormalHeapAllocate@16()  + 0xb2 bytes   
verifier.dll!_AVrfDebugPageHeapAllocate@12()  + 0x30e bytes
ntdll.dll!_RtlDebugAllocateHeap@12()  + 0x30 bytes   
ntdll.dll!@RtlpAllocateHeap@24()  + 0x57c17 bytes   
ntdll.dll!_RtlAllocateHeap@12()  + 0x502a bytes   
vld_x86.dll!vldnew(unsigned int size=0, const char * file=0x00360d7c, int line=1152)  Line 190 + 0x14 bytes    C++
vld_x86.dll!VisualLeakDetector::gettls()  Line 1153    C++
vld_x86.dll!VisualLeakDetector::_RtlAllocateHeap(void * heap=0x05ab0000, unsigned long flags=16781314, unsigned long size=88)  Line 2588 + 0x1f bytes    C++
verifier.dll!_AVrfpDphNormalHeapAllocate@16()  + 0xb2 bytes   
verifier.dll!_AVrfDebugPageHeapAllocate@12()  + 0x30e bytes   
ntdll.dll!_RtlDebugAllocateHeap@12()  + 0x30 bytes   
ntdll.dll!@RtlpAllocateHeap@24()  + 0x57c17 bytes   
ntdll.dll!_RtlAllocateHeap@12()  + 0x502a bytes   
vld_x86.dll!vldnew(unsigned int size=0, const char * file=0x00360d7c, int line=1152)  Line 190 + 0x14 bytes    C++
vld_x86.dll!VisualLeakDetector::gettls()  Line 1153    C++
vld_x86.dll!VisualLeakDetector::_RtlAllocateHeap(void * heap=0x05ab0000, unsigned long flags=16781314, unsigned long size=88)  Line 2588 + 0x1f bytes    C++

...etc.

Does that help at all? The executable name that's causing me problems is Debug-TerraFirma_Server.exe. All other ones I've tried are fine.

Coordinator
Apr 27, 2011 at 10:48 PM

Can you download symbols for ntdll.dll from Mycrosoft Symbol Server and repeat?

Apr 27, 2011 at 10:54 PM

I've updated my previous post with symbols now. Thanks for your help.

Coordinator
Apr 27, 2011 at 10:55 PM

Problem because AppVerifier(verifier.dll) on your system configurated to work with app named Debug-TerraFirma_Server.exe. So, it's not exactly our bug.

Apr 27, 2011 at 10:59 PM

Aha, that rings a bell. I'll try to remember what I did to get it tracked and see if removing it fixes the issue. Thanks a lot!

Apr 27, 2011 at 11:22 PM

Bingo, fixed. Thanks a lot for the pointer, and apologies for the wild accusations ;)

Developer
May 1, 2011 at 9:17 AM

I was also able to reproduce a stack overflow while trying VLD under Devpartners boundschecker. So it was the same type of issue.

Jul 19, 2012 at 1:41 PM

I'm seeing the same stack overflow crash using PageHeap (AppVerifier). While it may not be "our" crash, it would be nice to have a workaround. I'll see about implementing one.

I patched addLoadedModule to skip verifier.dll. Simplistic, but it works.