www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.bugs - [Issue 1786] New: Phobos cannot accommodate multiple stacks

reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1786

           Summary: Phobos cannot accommodate multiple stacks
           Product: D
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P2
         Component: Phobos
        AssignedTo: bugzilla digitalmars.com
        ReportedBy: default_357-line yahoo.de


Since the current version of Phobos always scans one whole stack (defined as
"stored stack bottom to ESP"), any advanced constructs which require multiple
stacks (like coroutines) are completely impossible to implement without
disabling the entire GC whenever a stackthread is somewhere active, which in
multi-threaded programs might disable the GC completely.

This issue can be fixed by introducing a way to override the phobos-perceived
stack top, thus limiting the GC to the actual stack. The attached patch against
GDC 0.23's Phobos does this.

Another problem is that stacks should be allocated as they're being used;
however, dynamic arrays cannot do that because resizing might require us to
move the stack elsewhere, which causes problems if we're currently using it :)

A solution to this is using memory-mapped files to contain the stack (as they
don't allocate memory unless it's being used, but only reserve some address
space). This leads to another problem: if the entire MmFile stack is registered
with the GC as a Range, the subsequent GC scan causes the operating system to
allocate memory for the whole range of the stack, causing an explosion of
memory usage.

The way to prevent this is to only register the specific range that is
currently in use, After the threads have been stopped and ESPs updated, but
before the garbage collection is run.

For this, a way is required for the program to register a callback with the GC
that is evoked when the garbage collection is being run (this callback must of
course not allocate any memory).

The attached patch adds this also.

The patch has been tested using the stackthreads implementation in
scrapple.tools, which now depends on it. For all I know, it works.

Since it is impossible to solve the problems described above without this
patch, I request it, or some functionally equivalent version, be included in
the official Phobos trunk.

Looking forward to responses and criticism,
 --feep (downs).


--
Jan 14 2008
next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1786






Created an attachment (id=218)
 --> (http://d.puremagic.com/issues/attachment.cgi?id=218&action=view)
Filename is descriptive enough.

Agh, forgot to attach the patch. Here is it.


--
Jan 14 2008
prev sibling next sibling parent d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1786


default_357-line yahoo.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------

        description|enough.                     |_and_callback.patch




--
Jan 14 2008
prev sibling parent reply d-bugmail puremagic.com writes:
http://d.puremagic.com/issues/show_bug.cgi?id=1786


default_357-line yahoo.de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID





Mikola Lysenko, the author of the Original StackThreads
(http://assertfalse.com/about.shtml), has pointed out a fascinating way of
making the GC work despite its shortcomings.
With this, I have to revise my original opinion - looks like making the GC work
with StackThreads is possible after all :)

Goes to show that using absolutes like "can't possibly work" is always a bad
idea. Stupid me.

I'll be trying his technique out later today, but for now I don't see a reason
why it wouldn't work. Thus, tentatively marked INVALID.

Sorry for the confusion.

 --downs


--
Jan 18 2008
parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
d-bugmail puremagic.com wrote:

 http://d.puremagic.com/issues/show_bug.cgi?id=1786
 
 
 default_357-line yahoo.de changed:
 
            What    |Removed                     |Added
 ----------------------------------------------------------------------------
              Status|NEW                         |RESOLVED
          Resolution|                            |INVALID
 
 
 
 

 Mikola Lysenko, the author of the Original StackThreads
 (http://assertfalse.com/about.shtml), has pointed out a fascinating way of
 making the GC work despite its shortcomings.
 With this, I have to revise my original opinion - looks like making the GC work
 with StackThreads is possible after all :)
 
 Goes to show that using absolutes like "can't possibly work" is always a bad
 idea. Stupid me.


Looks like it was a good idea in this case.  It goaded someone into 
finding a way to do what you wanted to do.  :-)

--bb
Jan 18 2008
parent reply downs <default_357-line yahoo.de> writes:
Bill Baxter wrote:

 
 Looks like it was a good idea in this case.  It goaded someone into
 finding a way to do what you wanted to do.  :-)
 
 --bb


Hehe :)

Also, I still think the GC should be changed to natively accomodate multiple
stacks.
The current solution works, but is slower by a factor of five or so.

Tango has stack contexts natively integrated with the GC, and it irks me daily
that Phobos is behind in this regard.

 --downs
Jan 21 2008
parent "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"downs" <default_357-line yahoo.de> wrote in message 
news:fn2l4m$1f9$2 digitalmars.com...

 Bill Baxter wrote:

 Looks like it was a good idea in this case.  It goaded someone into
 finding a way to do what you wanted to do.  :-)

 --bb


 Hehe :)

 Also, I still think the GC should be changed to natively accomodate 
 multiple stacks.
 The current solution works, but is slower by a factor of five or so.

 Tango has stack contexts natively integrated with the GC, and it irks me 
 daily that Phobos is behind in this regard.

 --downs


Then maybe you should use Tango HURR HURR
Jan 21 2008