www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 9242] New: Add stack smashing code to flush out heisenbugs

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242

           Summary: Add stack smashing code to flush out heisenbugs
           Product: D
           Version: D2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: DMD
        AssignedTo: nobody puremagic.com
        ReportedBy: bugzilla digitalmars.com



16:42:28 PST ---
We've lately had some very hard to track down heisenbugs that ultimately turned
out to be references to stack frames that have gone out of scope. This
particularly is happening when there are bugs in the lambda implementation, but
it is quite possible that such can still happen with user code.

It's not possible to always detect these at runtime, but their incidence can be
reduced, and bugs should be easier to track down because those references will
not randomly appear to work.

The first part is to replace the stack frame cleanup code:

    mov ESP,EBP
    pop EBP
    ret

with:

    call __stack_frame_smash
    mov ESP,EBP
    pop EBP
    ret

What __stack_frame_smash does is:

    1. set all memory [ESP..EBP] to something like 0xDEADBEEF
    2. set to 0xDEADBEEF all registers that are not guaranteed to be preserved
    across function calls.

Unfortunately, this won't smash the parameter stack, and it can't because the
callee cannot know how many parameters are on that stack (according to the
ABI). But, ya can't have everything.

The second part is, when a pointer, reference, dynamic array, or delegate is
returned from a function, add the following code to the epilog before the call
to __stack_frame_smash:

    cmp EAX,EBP
    ja  Ok
    cmp EAX,ESP
    jb  Ok
    halt
Ok:

or EDX in the case of dynamic arrays. This will halt the machine if a pointer
into the deallocated stack frame is returned.

Insertion of this code is done if the -gh switch is thrown.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242


bearophile_hugs eml.cc changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bearophile_hugs eml.cc





All this sounds very nice.


 Insertion of this code is done if the -gh switch is thrown.


Maybe it's better to name/syntax it differently, to make it more future-proof
in case we'll want to add other related runtime safeties.

Example: in FreePascal there is an option for stack checking:

http://www.freepascal.org/docs-html/prog/progsu101.html#x108-1090001.2.25

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242







 Maybe it's better to name/syntax it differently, to make it more future-proof
 in case we'll want to add other related runtime safeties.


A simpler possibility is not add a switch, and just turn on this feature (and
the feature in Issue 9243 ) when the "-debug" switch is used.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242


Jonathan M Davis <jmdavisProg gmx.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jmdavisProg gmx.com



PST ---

 A simpler possibility is not add a switch, and just turn on this feature (and


the feature in Issue 9243 ) when the "-debug" switch is used.

Let's not overload the -debug flag. All it does is enable debug blocks which
are used primarily for debug output. That's fundamentally different from
something like stack smashing, and depending, someone might actually want this
feature with -release, and while you _can_ use -debug with -release, you
generally don't want to, and I wouldn't expect anyone who wanted this feature
with -release to also want debug blocks to be enabled.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Dec 29 2012
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242




This is also useful against certain kinds of attacks. This seems useful for a
language like D that tries to be "safer" than C/C++. Some info:

http://en.wikipedia.org/wiki/Buffer_overflow_protection#StackGuard

http://www.research.ibm.com/trl/projects/security/ssp/

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 01 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242




21:33:17 PST ---
https://github.com/D-Programming-Language/dmd/pull/1542

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242




21:42:13 PST ---


 Example: in FreePascal there is an option for stack checking:


Stack overflow checking is common on CPUs with no virtual memory, such as 16
bit DOS. Virtual memory systems get stack overflow checking "for free", by
marking the memory page beyond the end of the stack as neither readable nor
writeable. Then the hardware does the check for you.

The stack smashing thing is completely different.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242







 Stack overflow checking is common on CPUs with no virtual memory, such as 16
 bit DOS. Virtual memory systems get stack overflow checking "for free", by
 marking the memory page beyond the end of the stack as neither readable nor
 writeable. Then the hardware does the check for you.


OK. (I'd like an error message plus a stack trace when D programs overflow the
stack.)



 The stack smashing thing is completely different.


I am aware of this.


See also those pages on StackGuard, that is a third different thing.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 24 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242




02:10:33 PST ---


 OK. (I'd like an error message plus a stack trace when D programs overflow the
 stack.)


Why not try it and see what happens? In any case, discussing stack overflow
here is not the right place, as this issue has nothing in common with it.



 See also those pages on StackGuard, that is a third different thing.


I know what stackguard is. This is not stackguard, and has nothing to do with
it.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 24 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242







 Why not try it and see what happens?


I have just tried this code with and without -gx, and on Windows32 the program
segfaults with no error message and no stack trace:


import std.c.stdio;
void recurse(in uint i=0) {
    printf("%u ", i);
    recurse(i + 1);
}
void main() {
    recurse();
}



 In any case, discussing stack overflow
 here is not the right place, as this issue has nothing in common with it.


OK.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 24 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=9242




21:59:02 PST ---



 

 Why not try it and see what happens?


 
 I have just tried this code with and without -gx,


What do I need to do to convince you that stack overflow has nothing at all to
do with what -gx does?


 and on Windows32 the program
 segfaults with no error message and no stack trace:


And it seg faults because the stack runs into the guard page.

A stack trace would be problematic, because there'd be thousands of entries,
all the same. Generally, once you run out of stack, there's not a whole lot of
function calling you can do to do more processing.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jan 24 2013