www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 10701] New: segfault after GC.realloc

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701

           Summary: segfault after GC.realloc
           Product: D
           Version: D2
          Platform: x86_64
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: druntime
        AssignedTo: nobody puremagic.com
        ReportedBy: monarchdodra gmail.com



Created an attachment (id=1238)
segfault after realloc

Only reproduced on 64 bit posix systems.

This is kind of complicated. The reduced program needs to iterate a couple of
times (probably to corrupt the GC?) a few times before the problem will
trigger. The good news, is that the segfault deterministically repeats itselft,
so the debug "should" be easy to do.

It would *appear* that the core culprit is creating a dynamic array (or size
larger than 4080), and then calling realloc on "array.ptr". Now, I'm not 100%
sure this is legal to begin with, since "array.ptr" is actually offset by 16
bytes from the start of the memory block. Is that actually undefined behavior,
or does it just reduce the chances of the program working?

I'd simply leave it at that and move on, but there is something that bothers me
deeply:

//----
ubyte[] arr = new ubyte[](5000);
GC.realloc(arr.ptr, 0, GC.BlkAttr.NO_SCAN);
//----
This works 100% fine (AFAIK, never segfaulted), but this:

//----
ubyte[] arr;
arr.length = 5000;
GC.realloc(arr.ptr, 0, GC.BlkAttr.NO_SCAN);
//----

Doing this ends up segfaulting later down the line.
*** Why is the behavior different? ***
I believe it is worth trying to investigate this at least a little, we might be
able to unravel a bug somewhere inside the code...

The code is in the attachment. I've reduced it as much as I could.

If somebody with more skills (and is more used to debugging in a *nix
environment) could take a peak?

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 23 2013
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701


Maxim Fomin <maxim maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |maxim maxim-fomin.ru



---
Reduced:

import core.memory, std.array;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr);
    result = null;
}

string toStr(long src)
{
    auto w = appender!string();
    return "";
}

void main()
{
    foreach(int i; 0 .. 256)
    {
        printf("Step: %d\n", i);
        string corruptme = "./temp";
        foreach(e; 0 .. 256)
        {
            corruptme ~= toStr(1);
        }
        readt();
    }
}


Removing appender makes bug go away.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701






 Reduced:
 [...]
 Removing appender makes bug go away.


Nice.

Appender in itself isn't doing anything much. As long as an allocation occurs,
any function will do:

//----
string toStr(long src)
{   
    new int;
    return "";
}
//----

EG:
//----
import core.memory;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr);
    result = null;
}

string toStr(long src)
{   
    new int;
    return "";
}

void main()
{
    foreach(int i; 0 .. 1024)
    {
        printf("Step: %d\n", i);
        string corruptme = "./temp";
        foreach(e; 0 .. 256)
        {
            corruptme ~= toStr(1);
        }
        readt();
    }
}
//----

This still preserves the "This works/But this fails" issue. Cores on iteration
255 (adding more "new int" will divide that number by the amount of "new")

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701


Maxim Fomin <maxim maxim-fomin.ru> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|segfault after GC.realloc   |segfault in GC
           Severity|normal                      |critical



---




 Reduced:
 [...]
 Removing appender makes bug go away.


 
 Nice.
 
 Appender in itself isn't doing anything much. As long as an allocation occurs,
 any function will do:
 
 //----
 string toStr(long src)
 {   
     new int;
     return "";
 }
 //----


Then futher reduced:

import core.memory;

extern(C) int printf(const char*, ...);

void readt()
{
    //ubyte[] result = new ubyte[](5000); //This works
    ubyte[] result; result.length = 5000; //But this fails
    GC.free(result.ptr); //works if commented out 
    result = null;
}

void main()
{
    foreach(i; 0 .. 1024)
    {
        foreach(e; 0 .. 1024)
        {
            new int;
        }

        readt();
    }
}

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Jul 23 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701


safety0ff.bugz <safety0ff.bugz gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |pull
                 CC|                            |safety0ff.bugz gmail.com



12:16:48 PDT ---
https://github.com/D-Programming-Language/druntime/pull/642

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 25 2013
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701


Martin Nowak <code dawg.eu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |code dawg.eu



cat > bug.d << CODE
import core.memory;
unittest
{
    ubyte[] result; result.length = 4096;
    GC.free(result.ptr);
    GC.collect();
}
CODE

dmd -main -unittest -run bug

----

Happens in Gcx.isMarked which incorrectly handles B_FREE pages.

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013
prev sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=10701




Commits pushed to master at https://github.com/D-Programming-Language/druntime

https://github.com/D-Programming-Language/druntime/commit/b28fc2b8cc1099d6da7ba935c222b435f46de77a
regression test for Issue 10701

- The test isn't 100% reliable because it depends
  on the GC and array append cache state.

https://github.com/D-Programming-Language/druntime/commit/525a9b5e0ad0a831d3d392184a8ed9f04c4293f2


regression test for Issue 10701

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
Oct 30 2013