• ninjadroid gazuga.net (24/24) Sep 09 2004 This is NOT flamebait!
• Vathix (21/45) Sep 09 2004 post
• WOB (14/38) Sep 09 2004 I think i found in the manual that the GC cycle cannot start if no
• Norbert Nemec (6/12) Sep 09 2004 Depends on whether "your OS" really refers to Windows as you silently
• WOB (7/10) Sep 10 2004 Well I didn't silently assume Windows but yes I did silently assume
• Ben Hinkle (13/49) Sep 09 2004 sure - that wouldn't be hard at all. I would modify two files: std/threa...
• Walter (18/29) Sep 09 2004 of
• ninjadroid gazuga.net (13/21) Sep 10 2004 OK, so to rephrase for those of us (i.e. me) that don't follow this kind...
• Sean Kelly (6/10) Sep 10 2004 No. Though it's worth noting that most memory allocators have critical
• Sha Chancellor (5/33) Sep 12 2004 "it would be incumbent upon the programmer to make sure* that the

ninjadroid gazuga.net writes:

This is NOT flamebait!

I like D.  I want the answer to the question posed in the subject of this post
to be a hearty and resounding "yes!"  However, I have some concerns that such
won't be the case.

D is a GC'd language through and through, and I'm not interested in trying to
extirpate that quality.  I imagine that if I wrote a program that didn't tickle
the GC in any way, shape, or form (i.e. 100% static memory), then it would be
Hard-RT compliant.  However, it probably wouldn't be terribly useful.

The specific scenario I'm interested in is multi-threaded programming with one
Hard-RT thread, communicating with the non-RT threads via a lock-free
ring-buffer.  The Hard-RT thread would be very well behaved and not do one iota
of memory allocation.  Conversely, the other threads, especially the GUI thread,
would.

The D spec says that all threads must pause when the GC runs, regardless of
which thread it runs in.  This seems to invalidate my design's RT-ness.  The
obvious solution of making the program multiprocess (barf) isn't too useful,
because that would make adjusting the RT thread's memory allocation a rather
convoluted process.  With the current design, I can adjust the RT thread's
memory structures in one of the non-RT threads, and protect the data with a
mutex that is try_locked().  A multi-process solution would be markedly uglier,
and I can't think of any method that would be 100% correct too boot; tickling
the GC seems inevitable.

Is there any way around this?  Is it even possible to write a GC that doesn't
stop up all threads?

"Vathix" <vathixSpamFix dprogramming.com> writes:

<ninjadroid gazuga.net> wrote in message
news:chqq71$k0o$1 digitaldaemon.com...
 This is NOT flamebait!

 I like D.  I want the answer to the question posed in the subject of this

post
 to be a hearty and resounding "yes!"  However, I have some concerns that

such
 won't be the case.

 D is a GC'd language through and through, and I'm not interested in trying

to
 extirpate that quality.  I imagine that if I wrote a program that didn't

tickle
 the GC in any way, shape, or form (i.e. 100% static memory), then it would

be
 Hard-RT compliant.  However, it probably wouldn't be terribly useful.

 The specific scenario I'm interested in is multi-threaded programming with

one
 Hard-RT thread, communicating with the non-RT threads via a lock-free
 ring-buffer.  The Hard-RT thread would be very well behaved and not do one

iota
 of memory allocation.  Conversely, the other threads, especially the GUI

thread,
 would.

 The D spec says that all threads must pause when the GC runs, regardless

of
 which thread it runs in.  This seems to invalidate my design's RT-ness.

The
 obvious solution of making the program multiprocess (barf) isn't too

useful,
 because that would make adjusting the RT thread's memory allocation a

rather
 convoluted process.  With the current design, I can adjust the RT thread's
 memory structures in one of the non-RT threads, and protect the data with

a
 mutex that is try_locked().  A multi-process solution would be markedly

uglier,
 and I can't think of any method that would be 100% correct too boot;

tickling
 the GC seems inevitable.

 Is there any way around this?  Is it even possible to write a GC that

doesn't
 stop up all threads?

You can still use C's malloc() and free(), and create threads that are
separate from the GC using C functions, but you can't easily use the GC with
them.

WOB <wb sapo.pt> writes:

On Thu, 9 Sep 2004 23:49:53 +0000 (UTC), ninjadroid gazuga.net wrote:

This is NOT flamebait!

I like D.  I want the answer to the question posed in the subject of this post
to be a hearty and resounding "yes!"  However, I have some concerns that such
won't be the case.

D is a GC'd language through and through, and I'm not interested in trying to
extirpate that quality.  I imagine that if I wrote a program that didn't tickle
the GC in any way, shape, or form (i.e. 100% static memory), then it would be
Hard-RT compliant.  However, it probably wouldn't be terribly useful.

The specific scenario I'm interested in is multi-threaded programming with one
Hard-RT thread, communicating with the non-RT threads via a lock-free
ring-buffer.  The Hard-RT thread would be very well behaved and not do one iota
of memory allocation.  Conversely, the other threads, especially the GUI thread,
would.

The D spec says that all threads must pause when the GC runs, regardless of
which thread it runs in.  This seems to invalidate my design's RT-ness.  The
obvious solution of making the program multiprocess (barf) isn't too useful,
because that would make adjusting the RT thread's memory allocation a rather
convoluted process.  With the current design, I can adjust the RT thread's
memory structures in one of the non-RT threads, and protect the data with a
mutex that is try_locked().  A multi-process solution would be markedly uglier,
and I can't think of any method that would be 100% correct too boot; tickling
the GC seems inevitable.

Is there any way around this?  Is it even possible to write a GC that doesn't
stop up all threads?

I think i found in the manual that the GC cycle cannot start if no
allocation is done anywhere in the code. So maybe you can allocate all
memory you need before the RT condition starts. So with a careful
design you can make sure the gc doesn't get called during RT critical
time slices. if you need to allocate, you can disable gc before and
enable after doing so. Or you can allocate with malloc, the gc will
ignore that but then you have to clean up yourself.

At a first glance it should not be very difficult to build hard RT
conditions into your D-code, but how do you want to obtain hard RT
conditions from your OS? Check this out:
http://www.flounder.com/time.htm - if that is true you will have to
boot your machine with your program, in order to obtain hard RT
conditions.

Norbert Nemec <Norbert Nemec-online.de> writes:

WOB wrote:

 At a first glance it should not be very difficult to build hard RT
 conditions into your D-code, but how do you want to obtain hard RT
 conditions from your OS? Check this out:
 http://www.flounder.com/time.htm - if that is true you will have to
 boot your machine with your program, in order to obtain hard RT
 conditions.

Depends on whether "your OS" really refers to Windows as you silently
assume. There is a number of operating systems out there that support hard
RT. One example is RTLinux, a modification of the original Linux kernel.
There even is a free version of that to check out:
 http://www.fsmlabs.com/products/openrtlinux/

WOB <wb sapo.pt> writes:

On Fri, 10 Sep 2004 08:46:15 +0200, Norbert Nemec
<Norbert Nemec-online.de> wrote:

Depends on whether "your OS" really refers to Windows as you silently
assume. There is a number of operating systems out there that support hard
RT. 

Well I didn't silently assume Windows but yes I did silently assume
any kind of general-purpose OS since there was also a reference to GUI
threads in the original post, but no reference to RT constraints of
the OS. But you are right, I should have written "... with your own
program, or install a real time OS..."

Ben Hinkle <bhinkle4 juno.com> writes:

ninjadroid gazuga.net wrote:

 This is NOT flamebait!
 
 I like D.  I want the answer to the question posed in the subject of this
 post
 to be a hearty and resounding "yes!"  However, I have some concerns that
 such won't be the case.
 
 D is a GC'd language through and through, and I'm not interested in trying
 to
 extirpate that quality.  I imagine that if I wrote a program that didn't
 tickle the GC in any way, shape, or form (i.e. 100% static memory), then
 it would be
 Hard-RT compliant.  However, it probably wouldn't be terribly useful.
 
 The specific scenario I'm interested in is multi-threaded programming with
 one Hard-RT thread, communicating with the non-RT threads via a lock-free
 ring-buffer.  The Hard-RT thread would be very well behaved and not do one
 iota
 of memory allocation.  Conversely, the other threads, especially the GUI
 thread, would.
 
 The D spec says that all threads must pause when the GC runs, regardless
 of
 which thread it runs in.  This seems to invalidate my design's RT-ness. 
 The obvious solution of making the program multiprocess (barf) isn't too
 useful, because that would make adjusting the RT thread's memory
 allocation a rather
 convoluted process.  With the current design, I can adjust the RT thread's
 memory structures in one of the non-RT threads, and protect the data with
 a
 mutex that is try_locked().  A multi-process solution would be markedly
 uglier, and I can't think of any method that would be 100% correct too
 boot; tickling the GC seems inevitable.
 
 Is there any way around this?  Is it even possible to write a GC that
 doesn't stop up all threads?

sure - that wouldn't be hard at all. I would modify two files: std/thread.d
and internal/gc/gcx.d
To thread.d I'd add a flag indicating if the thread is not to be stopped
during GC (and ignore such threads during pauseAll) and to gcx.d in the
function fullCollect I'd weed out those threads that are flagged. It would
be up to the programmer to make sure the non-GC thread doesn't contain any
sole references to GC-managed data. It would probably be a while before
anything like this made it into D proper - if at all because it is trusting
the programmer quite a lot to mark a thread as invisible to the GC.

Am I understanding the situation - or are there GC-managed objects that have
the only live reference from the RT thread?

-Ben

"Walter" <newshound digitalmars.com> writes:

<ninjadroid gazuga.net> wrote in message
news:chqq71$k0o$1 digitaldaemon.com...
 The D spec says that all threads must pause when the GC runs, regardless

of
 which thread it runs in.  This seems to invalidate my design's RT-ness.

The
 obvious solution of making the program multiprocess (barf) isn't too

useful,
 because that would make adjusting the RT thread's memory allocation a

rather
 convoluted process.  With the current design, I can adjust the RT thread's
 memory structures in one of the non-RT threads, and protect the data with

a
 mutex that is try_locked().  A multi-process solution would be markedly

uglier,
 and I can't think of any method that would be 100% correct too boot;

tickling
 the GC seems inevitable.

 Is there any way around this?  Is it even possible to write a GC that

doesn't
 stop up all threads?

The gc only pauses all threads that it knows about. There's nothing stopping
a D program from bypassing std.thread and creating a thread directly from
the operating system APIs. That thread wouldn't be paused during a gc
collection cycle, however, it would be incumbent upon the programmer to make
*sure* that the unpaused thread does not hold any references to objects that
*are not also held by any paused threads*.

D, being a systems programming language, lets you cheat and break the rules
when necessary.

ninjadroid gazuga.net writes:

In article <chr729$uvr$1 digitaldaemon.com>, Walter says...
The gc only pauses all threads that it knows about. There's nothing stopping
a D program from bypassing std.thread and creating a thread directly from
the operating system APIs. That thread wouldn't be paused during a gc
collection cycle, however, it would be incumbent upon the programmer to make
*sure* that the unpaused thread does not hold any references to objects that
*are not also held by any paused threads*.

OK, so to rephrase for those of us (i.e. me) that don't follow this kind of
stuff too well:

If I create a thread via a direct call to the OS, the GC won't know about it.
If the GC doesn't know about the thread, it won't stop it during a collection
run.  However, I need to then make sure that all the allocated memory the
"renegade" thread refers to is also referenced by a thread that the GC *is*
aware of; otherwise, the GC will think that memory is actually unused, and
possibly make my life miserable as a result.

Somewhat OT question: in a GC thread, will calls to malloc() register with the
GC?  That is, if you allocate memory with malloc(), you don't have to worry
about the GC gobbling it up?

D, being a systems programming language, lets you cheat and break the rules
when necessary.

That is *music* to my ears!

Sean Kelly <sean f4.ca> writes:

In article <chsal1$1guv$1 digitaldaemon.com>, ninjadroid gazuga.net says...
Somewhat OT question: in a GC thread, will calls to malloc() register with the
GC?  That is, if you allocate memory with malloc(), you don't have to worry
about the GC gobbling it up?

No.  Though it's worth noting that most memory allocators have critical
sections.  It may be worth writing your own allocator or seeing if you can
integrate an aftermarket version if you are concerned about this bottleneck.

That is *music* to my ears!

Where is BeOS when you need it? ;)


Sean

Sha Chancellor <schancel pacific.net> writes:

In article <chsal1$1guv$1 digitaldaemon.com>, ninjadroid gazuga.net 
wrote:

 In article <chr729$uvr$1 digitaldaemon.com>, Walter says...
The gc only pauses all threads that it knows about. There's nothing stopping
a D program from bypassing std.thread and creating a thread directly from
the operating system APIs. That thread wouldn't be paused during a gc
collection cycle, however, it would be incumbent upon the programmer to make
*sure* that the unpaused thread does not hold any references to objects that
*are not also held by any paused threads*.

 
 OK, so to rephrase for those of us (i.e. me) that don't follow this kind of
 stuff too well:
 
 If I create a thread via a direct call to the OS, the GC won't know about it.
 If the GC doesn't know about the thread, it won't stop it during a collection
 run.  However, I need to then make sure that all the allocated memory the
 "renegade" thread refers to is also referenced by a thread that the GC *is*
 aware of; otherwise, the GC will think that memory is actually unused, and
 possibly make my life miserable as a result.


"it would be incumbent upon the programmer to make sure* that the 
unpaused thread does _NOT_ hold any references to objects that are not 
also held by any paused threads*."


 
 Somewhat OT question: in a GC thread, will calls to malloc() register with the
 GC?  That is, if you allocate memory with malloc(), you don't have to worry
 about the GC gobbling it up?
 
D, being a systems programming language, lets you cheat and break the rules
when necessary.

 
 That is *music* to my ears!