www.digitalmars.com         C & C++   DMDScript  

D - coroutine?

reply nobody yahoo.com writes:
Has coroutine (sather style iterator) been discussed before on this list?  Will
D has something similar?




http://gary.burd.info/space/entry-7.html

Sather has it:

http://www.icsi.berkeley.edu/~sather/Publications/toplas.html

Python has it:

http://www.python.org/doc/current/whatsnew/section-generators.html

some discussion on comp.lang.eiffel:

http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&threadm=uy8zjfnmc.fsf%40wanadoo.fr&prev=/groups%3Fdq%3D%26num%3D25%26hl%3Den%26lr%3D%26ie%3DUTF-8%26group%3Dcomp.lang.eiffel%26start%3D25

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&frame=right&th=4cd646634db7588c&seekm=b0f57d2d.0110101547.78d69f50%40posting.google.com#link21
Aug 09 2003
parent reply "Walter" <walter digitalmars.com> writes:
Coroutines are pretty cool. They aren't in D now, but they could be added in
the future.
Aug 10 2003
parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Walter" <walter digitalmars.com> wrote in message
news:bh617r$1vst$1 digitaldaemon.com...

 Coroutines are pretty cool. They aren't in D now, but they could be added


in

 the future.


can't NT (98 or NT3.51+) fibers be used ?

extern(Windows) {
LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
lpStartAddress,LPVOID lpParameter);
VOID DeleteFiber(LPVOID lpFiber);
LPVOID ConvertThreadToFiber(LPVOID lpParameter);
VOID SwitchToFiber(LPVOID lpFiber);
BOOL SwitchToThread();
}

not sure if linux/pthreads have similar facilities.

I know they do work I've tried a few basic tests with fibers, if someone
posts an example of what they would like I'll see if I can produce a version
in current D that uses fibers as an example.
Aug 10 2003
parent reply "Walter" <walter digitalmars.com> writes:
I've been reading about fibers on google, but I am not yet convinced they
are the right solutions for coroutines because:
1) seems like a lot of overhead to manage them
2) not portable

"Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
news:bh63a0$21m6$1 digitaldaemon.com...

 "Walter" <walter digitalmars.com> wrote in message
 news:bh617r$1vst$1 digitaldaemon.com...

 Coroutines are pretty cool. They aren't in D now, but they could be




added

 in

 the future.


 can't NT (98 or NT3.51+) fibers be used ?

 extern(Windows) {
 LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
 lpStartAddress,LPVOID lpParameter);
 VOID DeleteFiber(LPVOID lpFiber);
 LPVOID ConvertThreadToFiber(LPVOID lpParameter);
 VOID SwitchToFiber(LPVOID lpFiber);
 BOOL SwitchToThread();
 }

 not sure if linux/pthreads have similar facilities.

 I know they do work I've tried a few basic tests with fibers, if someone
 posts an example of what they would like I'll see if I can produce a


version

 in current D that uses fibers as an example.
Aug 10 2003
parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
Fibers are as lightweight as you want to make them.  They have almost zero
overhead compared with threads.  All it is is swapping to a different stack;
very similar to cooperative multitasking.

You can implement them yourself using nothing more than setjmp/longjmp (so
long as you're able to figure out what the stack pointer is in the registers
structure, which is platform dependent).  Or you can use Windows fibers, but
you gain nothing by it except avoiding the hairy details of switching stacks
and registers.

The compiler can internally implement fibers however it wants;  it's really
quite simple and I can give you advice on implementation if you'd like.  You
would want a custom implementation for each platform to get the most
performance.

My coworkers used to call this "stack twiddling".  ;)  It can sure clear up
the logic behind a state machine like you wouldn't believe.

Coroutines are just language syntax that enables transparent utilization of
fibers.  "Yield" keyword just calls out to the fiber manager and saves the
state of this fiber and lets the next fiber run, or switches back to the
main fiber.  Calling a coroutine starts a new fiber or switches to one
that's already running, resuming at just past where the previous yield left
off.

You don't need language support to do this, but it would be cleaner, and the
portability hassle of switching stacks would be best done in the compiler or
standard library.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:bh667t$24js$1 digitaldaemon.com...

 I've been reading about fibers on google, but I am not yet convinced they
 are the right solutions for coroutines because:
 1) seems like a lot of overhead to manage them
 2) not portable

 "Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
 news:bh63a0$21m6$1 digitaldaemon.com...

 "Walter" <walter digitalmars.com> wrote in message
 news:bh617r$1vst$1 digitaldaemon.com...

 Coroutines are pretty cool. They aren't in D now, but they could be




 added

 in

 the future.


 can't NT (98 or NT3.51+) fibers be used ?

 extern(Windows) {
 LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
 lpStartAddress,LPVOID lpParameter);
 VOID DeleteFiber(LPVOID lpFiber);
 LPVOID ConvertThreadToFiber(LPVOID lpParameter);
 VOID SwitchToFiber(LPVOID lpFiber);
 BOOL SwitchToThread();
 }

 not sure if linux/pthreads have similar facilities.

 I know they do work I've tried a few basic tests with fibers, if someone
 posts an example of what they would like I'll see if I can produce a


 version

 in current D that uses fibers as an example.
Aug 10 2003
next sibling parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
Win32 fibers have various problems, such as not being able to de-fiberise a
thread, nor to have multiple fibers within a thread, or the prosaic one of
not being implemented in 9x.

I've wanted to do a platform-independent fibers API for ages, but just have
never got round to it. I'd love to help do that for D, but alas would
probably be more hindrance than help, asm not being my speciality ...
<blushes>.

They definitely should be in D, as they're very useful things to have, but
should be D's own, not layered on top of Win32.

"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bh6euu$2ci7$1 digitaldaemon.com...

 Fibers are as lightweight as you want to make them.  They have almost zero
 overhead compared with threads.  All it is is swapping to a different


stack;

 very similar to cooperative multitasking.

 You can implement them yourself using nothing more than setjmp/longjmp (so
 long as you're able to figure out what the stack pointer is in the


registers

 structure, which is platform dependent).  Or you can use Windows fibers,


but

 you gain nothing by it except avoiding the hairy details of switching


stacks

 and registers.

 The compiler can internally implement fibers however it wants;  it's


really

 quite simple and I can give you advice on implementation if you'd like.


You

 would want a custom implementation for each platform to get the most
 performance.

 My coworkers used to call this "stack twiddling".  ;)  It can sure clear


up

 the logic behind a state machine like you wouldn't believe.

 Coroutines are just language syntax that enables transparent utilization


of

 fibers.  "Yield" keyword just calls out to the fiber manager and saves the
 state of this fiber and lets the next fiber run, or switches back to the
 main fiber.  Calling a coroutine starts a new fiber or switches to one
 that's already running, resuming at just past where the previous yield


left

 off.

 You don't need language support to do this, but it would be cleaner, and


the

 portability hassle of switching stacks would be best done in the compiler


or

 standard library.

 Sean

 "Walter" <walter digitalmars.com> wrote in message
 news:bh667t$24js$1 digitaldaemon.com...

 I've been reading about fibers on google, but I am not yet convinced




they

 are the right solutions for coroutines because:
 1) seems like a lot of overhead to manage them
 2) not portable

 "Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
 news:bh63a0$21m6$1 digitaldaemon.com...

 "Walter" <walter digitalmars.com> wrote in message
 news:bh617r$1vst$1 digitaldaemon.com...

 Coroutines are pretty cool. They aren't in D now, but they could be




 added

 in

 the future.


 can't NT (98 or NT3.51+) fibers be used ?

 extern(Windows) {
 LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
 lpStartAddress,LPVOID lpParameter);
 VOID DeleteFiber(LPVOID lpFiber);
 LPVOID ConvertThreadToFiber(LPVOID lpParameter);
 VOID SwitchToFiber(LPVOID lpFiber);
 BOOL SwitchToThread();
 }

 not sure if linux/pthreads have similar facilities.

 I know they do work I've tried a few basic tests with fibers, if






someone

 posts an example of what they would like I'll see if I can produce a


 version

 in current D that uses fibers as an example.
Aug 10 2003
parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Matthew Wilson" <matthew stlsoft.org> wrote in message
news:bh6g92$2dpv$1 digitaldaemon.com...

 Win32 fibers have various problems, such as not being able to de-fiberise


a

 thread, nor to have multiple fibers within a thread, or the prosaic one of
 not being implemented in 9x.

 I've wanted to do a platform-independent fibers API for ages, but just


have

 never got round to it. I'd love to help do that for D, but alas would
 probably be more hindrance than help, asm not being my speciality ...
 <blushes>.

 They definitely should be in D, as they're very useful things to have, but
 should be D's own, not layered on top of Win32.


why? the OS gives you a feature and you want to manually stack switch!

you don't need to de-fiber, they are cooperative so once you finish with a
fiber (other than main) you delete it
the only reason to "fiberise" main is to allow you to get back to it. the OS
could make all threads single fibers initially, but why when fiber use is
rare. (same might be said for a process why have a process with a thread,
why not have a process, that then has to do ConvertProcessToThread is you
want threading)

fibers are a lot better than trying to do coroutines with setjump/longjump
no manual manipulation of the stack
from my MS docs fibers are supported on win98+ and NT3.51 sp 3
you can have multipul fibers in a thread (my D test code had 3)
there are 2 ways to do coroutines (that I know of) (threads and semaphores
or monitors (rendezvous are better)) should work cross platform or abuse the
stack and use setjump/longjump to get between then but you can either rely
on the stack not getting shrunk by the OS or if is checks ESP as task switch
and free unused stack pages you'll have to use alloca or similar to get your
stack back and run into issues there.
and I believe there are OS's that have non contiguious stacks (you can think
of the stack as a singly linked list of frames) usual function epilog is
`get my old sp back and jump to the instruction after the one that called
me`
so on most arch's you can alloc a block of memory, store the sp params and
anything else in the block as required change the SP so the pop in the
epilog will get it and the back, and jump. (on risc arch's its very easy to
do (they have link registers so the return address is stored by the callee
(and usually no in leaf funcs) not the caller).
getting OT, but as I see it use fibers on win32, on other platform you'll
have to do something else.
to get fiber like behavoiur you need :
    a way to stop the current and switch to an existing thread/process (I
belive that on freebsd threads are just processes that share data space,
from the OS each threads is a process rather than a process containing
several threads)




 "Sean L. Palmer" <palmer.sean verizon.net> wrote in message
 news:bh6euu$2ci7$1 digitaldaemon.com...

 Fibers are as lightweight as you want to make them.  They have almost




zero

 overhead compared with threads.  All it is is swapping to a different


 stack;

 very similar to cooperative multitasking.




from what I see they are cooperative multithreading (within your thread
which is prementively threaded just (don't you just love the random priority
boosting)).
Aug 10 2003
parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
 you don't need to de-fiber, they are cooperative so once you finish with a
 fiber (other than main) you delete it
 the only reason to "fiberise" main is to allow you to get back to it. the


OS

 could make all threads single fibers initially, but why when fiber use is
 rare. (same might be said for a process why have a process with a thread,
 why not have a process, that then has to do ConvertProcessToThread is you
 want threading)


This is not true. (Don't ask me to explain, as this is all in an article
that's yet-to-be-published, and I want to keep my powder dry. I'm know
that's an *incredibly* lame thing to say, so I'm not going to attempt to
press any points on this thread further.)


 fibers are a lot better than trying to do coroutines with setjump/longjump
 no manual manipulation of the stack


Indeed. Fibers are great, and have lots of unforeseen uses. If only they
were implemented a bit better in Win32, or, better, were a standard part of
C.

I started a SourceForge project last year -
http://fiberspp.sourceforge.net/ - but have yet to do anything. I'm looking
for talented people who know more about asm stuff than me to help me out ...


 from my MS docs fibers are supported on win98+ and NT3.51 sp 3


True, I forgot that.


 you can have multipul fibers in a thread (my D test code had 3)


Hmm. This does not correspond with my understanding and experience of them.
Can you post your code?


 there are 2 ways to do coroutines (that I know of) (threads and semaphores
 or monitors (rendezvous are better)) should work cross platform or abuse


the

 stack and use setjump/longjump to get between then but you can either rely
 on the stack not getting shrunk by the OS or if is checks ESP as task


switch

 and free unused stack pages you'll have to use alloca or similar to get


your

 stack back and run into issues there.
 and I believe there are OS's that have non contiguious stacks (you can


think

 of the stack as a singly linked list of frames) usual function epilog is
 `get my old sp back and jump to the instruction after the one that called
 me`
 so on most arch's you can alloc a block of memory, store the sp params and
 anything else in the block as required change the SP so the pop in the
 epilog will get it and the back, and jump. (on risc arch's its very easy


to

 do (they have link registers so the return address is stored by the callee
 (and usually no in leaf funcs) not the caller).
 getting OT, but as I see it use fibers on win32, on other platform you'll
 have to do something else.
 to get fiber like behavoiur you need :
     a way to stop the current and switch to an existing thread/process (I
 belive that on freebsd threads are just processes that share data space,
 from the OS each threads is a process rather than a process containing
 several threads)
Aug 10 2003
parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Matthew Wilson" <matthew stlsoft.org> wrote in message
news:bh6rqi$2o4e$1 digitaldaemon.com...

 you don't need to de-fiber, they are cooperative so once you finish with




a

 fiber (other than main) you delete it
 the only reason to "fiberise" main is to allow you to get back to it.




the

 OS

 could make all threads single fibers initially, but why when fiber use




is

 rare. (same might be said for a process why have a process with a




thread,

 why not have a process, that then has to do ConvertProcessToThread is




you

 want threading)


 This is not true. (Don't ask me to explain, as this is all in an article
 that's yet-to-be-published, and I want to keep my powder dry. I'm know
 that's an *incredibly* lame thing to say, so I'm not going to attempt to
 press any points on this thread further.)


fine if your not going to explain I'll contine in ignorance of which bit of
my conceptual overview of fiber, threads, process and sessions is wrong!!


 fibers are a lot better than trying to do coroutines with




setjump/longjump

 no manual manipulation of the stack


 Indeed. Fibers are great, and have lots of unforeseen uses. If only they
 were implemented a bit better in Win32, or, better, were a standard part


of

 C.


what more need to be done ?
with win32 fibers you can have several and jump between them :) see the
example pulled from MS's own docs on the subject


 I started a SourceForge project last year -
 http://fiberspp.sourceforge.net/ - but have yet to do anything. I'm


looking

 for talented people who know more about asm stuff than me to help me out


...

 from my MS docs fibers are supported on win98+ and NT3.51 sp 3


 True, I forgot that.


 you can have multipul fibers in a thread (my D test code had 3)


 Hmm. This does not correspond with my understanding and experience of


them.

 Can you post your code?


-----------------------------------------

import c.stdio;
import windows;

extern(Windows) {

alias void function( LPVOID ) LPFIBER_START_ROUTINE;
LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
lpStartAddress,LPVOID lpParameter);
VOID DeleteFiber(LPVOID lpFiber);
LPVOID ConvertThreadToFiber(LPVOID lpParameter);
VOID SwitchToFiber(LPVOID lpFiber);
BOOL SwitchToThread();
}

LPVOID lpFiber1;        // Addresses of each fiber that's created
LPVOID lpFiber2;
LPVOID lpFiber3;
LPVOID lpFiberMain;

extern (Windows) void FiberRoutine( LPVOID param )
{
 printf( " In fiber %u\n", param ); // Tell the world who we are
 if ( 1 == cast(DWORD)param ) {
  printf( " changing fiber 1->2\n" );
  SwitchToFiber( lpFiber2 );      // fiber1 -> fiber2
   } else if ( 2 == cast(DWORD)param ) {
  printf( " changing fiber 2->3\n" );
  SwitchToFiber( lpFiber3 );      // fiber2 -> fiber3
 } else if ( 3 == cast(DWORD)param ) {
  printf( " changing fber 3->main\n" );
  SwitchToFiber( lpFiberMain );   // fiber3->main fiber (original thread)
 }
 printf( " leaving fiber %u\n", param ); // Tell the world who we are
 printf( " returning to main fiber\n" ); // Tell the world who we are
 SwitchToFiber( lpFiberMain );   // fiber->main fiber (original)
 // if you don't end with this then you code will exit.
}

int main( char[][] argv )
{
    // Create 3 fibers, with starting addresses of the above routine
    lpFiber1 = CreateFiber( 16384, &FiberRoutine, cast(PVOID)1 );
    lpFiber2 = CreateFiber( 16384, &FiberRoutine, cast(PVOID)2 );
    lpFiber3 = CreateFiber( 16384, &FiberRoutine, cast(PVOID)3 );


    if ( !lpFiber1 || !lpFiber2 || !lpFiber3 )  // Make sure the fibers were
        return 0;                               // created

    // Make this thread a fiber, so that it can force the other fibers to
run
    lpFiberMain = ConvertThreadToFiber( cast(LPVOID)0 );

    // Sleep for 1 second, just to prove that the other
    printf( "Sleeping for 1 second\n" );
    Sleep( 1000 );

    printf( "Switching to first fiber\n" );
    SwitchToFiber( lpFiber1 );                  // Switch to fiber1
    printf( "Returning from SwitchToFiber ( fiber-1 )\n" );
    printf( "Switching to first fiber again\n" );
    SwitchToFiber( lpFiber1 );                  // Switch to fiber1
    printf( "Returning from (again) SwitchToFiber( fiber-1)\n" );

    DeleteFiber( lpFiber1 );    // Clean up the 3 fibers we created earlier
    DeleteFiber( lpFiber2 );
    DeleteFiber( lpFiber3 );

    return 0;
}
---------------------------------------------
straight port (+ a bit) from the MSDN
Aug 10 2003
parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
 This is not true. (Don't ask me to explain, as this is all in an article
 that's yet-to-be-published, and I want to keep my powder dry. I'm know
 that's an *incredibly* lame thing to say, so I'm not going to attempt to
 press any points on this thread further.)


 fine if your not going to explain I'll contine in ignorance of which bit


of

 my conceptual overview of fiber, threads, process and sessions is wrong!!


Oh no, I knew I'd come accross sounding like a tool. <blush>. I'm not trying
to be a princess, but once you've signed over the rights to your article,
you're not at liberty to go splashing the details in public until it comes
out. I wish I'd never said anything now .... <glum>.

I certainly don't wish to imply you are ignorant of the subject. In fact,
apart from the few areas in which I've used them, and found the pitfalls, I
would state that I'm pretty ignorant of them.

(btw, I think I was using the wrong terminology earlier. I was using fiber
for fiber-group. Naturally one can have many fibers; to only have one would
be a bit pointless.)


 Indeed. Fibers are great, and have lots of unforeseen uses. If only they
 were implemented a bit better in Win32, or, better, were a standard part


 of

 C.


 what more need to be done ?
 with win32 fibers you can have several and jump between them :) see the
 example pulled from MS's own docs on the subject


Your example is, with respect, a little trivial. What if I want to create a
fiber group with a certain peice of information, and use the (cooperative)
multi-tasking nature to carry out some operation that must be done that way,
and then later I want to do the same again. There's no SetFiberData(), so
there's no way to start off a new fiber "task" (i.e a job involving a fiber
group), unless I kick off a new thread (hardly the solution).

One option would be to have a fiber-data indirection, and then you can
change the task information at some later stage. But in that case we would
have to ensure that all fibers in our "group" would have finished with their
use of the fiber data, and were back at an initialisable position. Not
impossible, but not necessarily trivial.

Finally, all that is perhaps feasible when you're totally in control of all
the code, but what about if it's in a library (either header, or via some
API). You cannot turn a thread into a fiber twice, so if you have two libs
making use of fibers the second one's going to die. (I can't remember
whether the problem comes not with the second call to ConvertThreadToFiber()
or on the next context switch, but I can tell you it does die.)


 I started a SourceForge project last year -
 http://fiberspp.sourceforge.net/ - but have yet to do anything. I'm


 looking

 for talented people who know more about asm stuff than me to help me out




Anyway, as I said, I'm not an asm-head, but am happy to learn from/cooperate
with someone who is.

If anyone wants to create a cross-platform multi-fiber-group API (if such a
thing is possible, of course, there're all those nasty stack guard page
issues on Win32), then please shout up. It doesn't have to be on
SourceForge, it could be a DMC/DMD thing.

Matthew
Aug 10 2003
next sibling parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
Just found the bits out of my article (it was written quite a long while
back):

"Unfortunately there is no way to close all the fibers without closing the
thread (i.e. there is no CloseThreadFiber() function), in other
words one cannot "de-fiber-ise" the thread. It is possible (at least on
Windows 2000) to ignore a previous initialisation
and call ConvertThreadToFiber() again, but the previously allocated fiber
management block remains allocated.
Although the MSDN knowledge base article Q185231 implies that one can simply
call LocalFree() on this block, this is
sailing far too close to the wind for me to recommend as a serious technique
to others (since it is not part of the
documented API, and therefore could be subject to breaking changes), and is
certainly not suitable for a robust and
flexible library.

Neither is it possible to call GetFiberData() in order to determine whether
fiber support has been initialised, as this
raises an access violation if it has not (and I don't like the hack of
catching that exception as the indicator), rather than
returning NULL; even if it were, the problems of interposing one fiber
sub-system on another would be inhibitive.

The hitch is, of course, what if some other part of the process is already
using fibers, or starts to use them during/after
the initialisation of our use of them? The simple and unappealing answer is
that it will crash. If two fiber sub-systems
run concurrently (within the scope defined by ConvertThreadToFiber() and the
exit from the calling thread) then the
second one to be initialised will destroy the execution contexts of the
first. If they run consecutively, then there will be a
memory leak (although it is only experience, as opposed to explicit
documentation on the subject, that lends this
understanding)"

Make sense? (I hope so, because that's what actually happens on all the
Win32 OSs that I've tested it on.)

Matthew

"Matthew Wilson" <matthew stlsoft.org> wrote in message
news:bh7cqu$7v3$1 digitaldaemon.com...

 This is not true. (Don't ask me to explain, as this is all in an






article

 that's yet-to-be-published, and I want to keep my powder dry. I'm know
 that's an *incredibly* lame thing to say, so I'm not going to attempt






to

 press any points on this thread further.)


 fine if your not going to explain I'll contine in ignorance of which bit


 of

 my conceptual overview of fiber, threads, process and sessions is




wrong!!

 Oh no, I knew I'd come accross sounding like a tool. <blush>. I'm not


trying

 to be a princess, but once you've signed over the rights to your article,
 you're not at liberty to go splashing the details in public until it comes
 out. I wish I'd never said anything now .... <glum>.

 I certainly don't wish to imply you are ignorant of the subject. In fact,
 apart from the few areas in which I've used them, and found the pitfalls,


I

 would state that I'm pretty ignorant of them.

 (btw, I think I was using the wrong terminology earlier. I was using fiber
 for fiber-group. Naturally one can have many fibers; to only have one


would

 be a bit pointless.)


 Indeed. Fibers are great, and have lots of unforeseen uses. If only






they

 were implemented a bit better in Win32, or, better, were a standard






part

 of

 C.


 what more need to be done ?
 with win32 fibers you can have several and jump between them :) see the
 example pulled from MS's own docs on the subject


 Your example is, with respect, a little trivial. What if I want to create


a

 fiber group with a certain peice of information, and use the (cooperative)
 multi-tasking nature to carry out some operation that must be done that


way,

 and then later I want to do the same again. There's no SetFiberData(), so
 there's no way to start off a new fiber "task" (i.e a job involving a


fiber

 group), unless I kick off a new thread (hardly the solution).

 One option would be to have a fiber-data indirection, and then you can
 change the task information at some later stage. But in that case we would
 have to ensure that all fibers in our "group" would have finished with


their

 use of the fiber data, and were back at an initialisable position. Not
 impossible, but not necessarily trivial.

 Finally, all that is perhaps feasible when you're totally in control of


all

 the code, but what about if it's in a library (either header, or via some
 API). You cannot turn a thread into a fiber twice, so if you have two libs
 making use of fibers the second one's going to die. (I can't remember
 whether the problem comes not with the second call to


ConvertThreadToFiber()

 or on the next context switch, but I can tell you it does die.)


 I started a SourceForge project last year -
 http://fiberspp.sourceforge.net/ - but have yet to do anything. I'm


 looking

 for talented people who know more about asm stuff than me to help me






out

 Anyway, as I said, I'm not an asm-head, but am happy to learn


from/cooperate

 with someone who is.

 If anyone wants to create a cross-platform multi-fiber-group API (if such


a

 thing is possible, of course, there're all those nasty stack guard page
 issues on Win32), then please shout up. It doesn't have to be on
 SourceForge, it could be a DMC/DMD thing.

 Matthew
Aug 10 2003
parent reply "Walter" <walter digitalmars.com> writes:
It sounds like fibers have an ill-designed implementation, and are not ready
for prime time. Besides, I may have found a neato way to do iterators
without need of coroutines.

"Matthew Wilson" <matthew stlsoft.org> wrote in message
news:bh7d5m$8ab$1 digitaldaemon.com...

 Just found the bits out of my article (it was written quite a long while
 back):

 "Unfortunately there is no way to close all the fibers without closing the
 thread (i.e. there is no CloseThreadFiber() function), in other
 words one cannot "de-fiber-ise" the thread. It is possible (at least on
 Windows 2000) to ignore a previous initialisation
 and call ConvertThreadToFiber() again, but the previously allocated fiber
 management block remains allocated.
 Although the MSDN knowledge base article Q185231 implies that one can


simply

 call LocalFree() on this block, this is
 sailing far too close to the wind for me to recommend as a serious


technique

 to others (since it is not part of the
 documented API, and therefore could be subject to breaking changes), and


is

 certainly not suitable for a robust and
 flexible library.

 Neither is it possible to call GetFiberData() in order to determine


whether

 fiber support has been initialised, as this
 raises an access violation if it has not (and I don't like the hack of
 catching that exception as the indicator), rather than
 returning NULL; even if it were, the problems of interposing one fiber
 sub-system on another would be inhibitive.

 The hitch is, of course, what if some other part of the process is already
 using fibers, or starts to use them during/after
 the initialisation of our use of them? The simple and unappealing answer


is

 that it will crash. If two fiber sub-systems
 run concurrently (within the scope defined by ConvertThreadToFiber() and


the

 exit from the calling thread) then the
 second one to be initialised will destroy the execution contexts of the
 first. If they run consecutively, then there will be a
 memory leak (although it is only experience, as opposed to explicit
 documentation on the subject, that lends this
 understanding)"

 Make sense? (I hope so, because that's what actually happens on all the
 Win32 OSs that I've tested it on.)

 Matthew

 "Matthew Wilson" <matthew stlsoft.org> wrote in message
 news:bh7cqu$7v3$1 digitaldaemon.com...

 This is not true. (Don't ask me to explain, as this is all in an






 article

 that's yet-to-be-published, and I want to keep my powder dry. I'm








know

 that's an *incredibly* lame thing to say, so I'm not going to








attempt

 to

 press any points on this thread further.)


 fine if your not going to explain I'll contine in ignorance of which






bit

 of

 my conceptual overview of fiber, threads, process and sessions is




 wrong!!

 Oh no, I knew I'd come accross sounding like a tool. <blush>. I'm not


 trying

 to be a princess, but once you've signed over the rights to your




article,

 you're not at liberty to go splashing the details in public until it




comes

 out. I wish I'd never said anything now .... <glum>.

 I certainly don't wish to imply you are ignorant of the subject. In




fact,

 apart from the few areas in which I've used them, and found the




pitfalls,

 I

 would state that I'm pretty ignorant of them.

 (btw, I think I was using the wrong terminology earlier. I was using




fiber

 for fiber-group. Naturally one can have many fibers; to only have one


 would

 be a bit pointless.)


 Indeed. Fibers are great, and have lots of unforeseen uses. If only






 they

 were implemented a bit better in Win32, or, better, were a standard






 part

 of

 C.


 what more need to be done ?
 with win32 fibers you can have several and jump between them :) see






the

 example pulled from MS's own docs on the subject


 Your example is, with respect, a little trivial. What if I want to




create

 a

 fiber group with a certain peice of information, and use the




(cooperative)

 multi-tasking nature to carry out some operation that must be done that


 way,

 and then later I want to do the same again. There's no SetFiberData(),




so

 there's no way to start off a new fiber "task" (i.e a job involving a


 fiber

 group), unless I kick off a new thread (hardly the solution).

 One option would be to have a fiber-data indirection, and then you can
 change the task information at some later stage. But in that case we




would

 have to ensure that all fibers in our "group" would have finished with


 their

 use of the fiber data, and were back at an initialisable position. Not
 impossible, but not necessarily trivial.

 Finally, all that is perhaps feasible when you're totally in control of


 all

 the code, but what about if it's in a library (either header, or via




some

 API). You cannot turn a thread into a fiber twice, so if you have two




libs

 making use of fibers the second one's going to die. (I can't remember
 whether the problem comes not with the second call to


 ConvertThreadToFiber()

 or on the next context switch, but I can tell you it does die.)


 I started a SourceForge project last year -
 http://fiberspp.sourceforge.net/ - but have yet to do anything. I'm


 looking

 for talented people who know more about asm stuff than me to help me






 out

 Anyway, as I said, I'm not an asm-head, but am happy to learn


 from/cooperate

 with someone who is.

 If anyone wants to create a cross-platform multi-fiber-group API (if




such

 a

 thing is possible, of course, there're all those nasty stack guard page
 issues on Win32), then please shout up. It doesn't have to be on
 SourceForge, it could be a DMC/DMD thing.

 Matthew
Aug 11 2003
next sibling parent reply "Sean L. Palmer" <palmer.sean verizon.net> writes:
I'll bite.

Sean

"Walter" <walter digitalmars.com> wrote in message
news:bha2fs$5dq$1 digitaldaemon.com...

 It sounds like fibers have an ill-designed implementation, and are not


ready

 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.
Aug 12 2003
parent reply "Walter" <walter digitalmars.com> writes:
I want to try and implement it first to see if it works. I have my best
ideas when out jogging, including this one. What I like to do is 'load up on
context', which means do all my research at the computer, with this
newsgroup, google, my books, etc. Then out the door for a run. I think all
the pounding seems to throw all the bits of data around in the air, shaking
them around until they fit together <g>.

"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bha9su$dpk$1 digitaldaemon.com...

 I'll bite.

 Sean

 "Walter" <walter digitalmars.com> wrote in message
 news:bha2fs$5dq$1 digitaldaemon.com...

 It sounds like fibers have an ill-designed implementation, and are not


 ready

 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.
Aug 12 2003
parent reply Ilya Minkov <midiclub 8ung.at> writes:
Walter wrote:

 I want to try and implement it first to see if it works. I have my best
 ideas when out jogging, including this one. What I like to do is 'load up on
 context', which means do all my research at the computer, with this
 newsgroup, google, my books, etc. Then out the door for a run. I think all
 the pounding seems to throw all the bits of data around in the air, shaking
 them around until they fit together <g>.


This is also one of the key symptomes of addiction - when a person keeps 
thinking about it when away from it. :) i.e. Internet addicts are said 
to have a "dense mindstream" as soon as they disconnect.

-i.
Aug 12 2003
parent "Walter" <walter digitalmars.com> writes:
"Ilya Minkov" <midiclub 8ung.at> wrote in message
news:bhbrfg$1vr0$1 digitaldaemon.com...

 Walter wrote:

 I want to try and implement it first to see if it works. I have my best
 ideas when out jogging, including this one. What I like to do is 'load




up on

 context', which means do all my research at the computer, with this
 newsgroup, google, my books, etc. Then out the door for a run. I think




all

 the pounding seems to throw all the bits of data around in the air,




shaking

 them around until they fit together <g>.


 This is also one of the key symptomes of addiction - when a person keeps
 thinking about it when away from it. :) i.e. Internet addicts are said
 to have a "dense mindstream" as soon as they disconnect.


What else is there to think about?
Aug 12 2003
prev sibling parent reply Bill Cox <bill viasic.com> writes:
Walter wrote:

 It sounds like fibers have an ill-designed implementation, and are not ready
 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.


I'm all ears.  I'd love to here about D's iterators.

Bill
Aug 12 2003
next sibling parent nobody yahoo.com writes:
Coroutine is more than just iterators:

http://www.cs.utexas.edu/users/wilson/schintro/schintro_75.html

" ... is far more powerful than normal procedure calling and returning, and
allows you to implement advanced control structures such as backtracking,
cooperative multitasking, and custom exception-handling."


In article <bharr3$vcu$2 digitaldaemon.com>, Bill Cox says...

Walter wrote:

 It sounds like fibers have an ill-designed implementation, and are not ready
 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.


I'm all ears.  I'd love to here about D's iterators.

Bill
Aug 12 2003
prev sibling parent reply nobody yahoo.com writes:
Coroutine is more than just iterators:

http://www.cs.utexas.edu/users/wilson/schintro/schintro_75.html

" ... is far more powerful than normal procedure calling and returning, and
allows you to implement advanced control structures such as backtracking,
cooperative multitasking, and custom exception-handling."


In article <bharr3$vcu$2 digitaldaemon.com>, Bill Cox says...

Walter wrote:

 It sounds like fibers have an ill-designed implementation, and are not ready
 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.


I'm all ears.  I'd love to here about D's iterators.

Bill
Aug 12 2003
parent "Mike Wynn" <mike.wynn l8night.co.uk> writes:
yes, but it sounds like Walters worked out how to do iterators without
coroutines

iterators don't need the ability to unwind the stack back beyond the
original point of creation
iterators can be implemented as an object with a single member function
which has no locals or params but instead all locals are member of the
object (kind of like the innerfunction model which considers a function to
be a member function of a stack frame). then either a switch on state or a
way to save the location of the last yeild. (yeild either updates a state
int, or saves the address of the next instruction in the "start from here"
pointer)
although this does not limit the iterated function to be only leaf function
it does limit the yeild point to only being within that function (you can
not yeild from a function that the itterating function calls).

<nobody yahoo.com> wrote in message news:bhbeht$1k53$1 digitaldaemon.com...

 Coroutine is more than just iterators:

 http://www.cs.utexas.edu/users/wilson/schintro/schintro_75.html

 " ... is far more powerful than normal procedure calling and returning,


and

 allows you to implement advanced control structures such as backtracking,
 cooperative multitasking, and custom exception-handling."


 In article <bharr3$vcu$2 digitaldaemon.com>, Bill Cox says...

Walter wrote:

 It sounds like fibers have an ill-designed implementation, and are not






ready

 for prime time. Besides, I may have found a neato way to do iterators
 without need of coroutines.


I'm all ears.  I'd love to here about D's iterators.

Bill
Aug 12 2003
prev sibling parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
 Your example is, with respect, a little trivial. What if I want to create


a

 fiber group with a certain peice of information, and use the (cooperative)
 multi-tasking nature to carry out some operation that must be done that


way,

 and then later I want to do the same again. There's no SetFiberData(), so
 there's no way to start off a new fiber "task" (i.e a job involving a


fiber

 group), unless I kick off a new thread (hardly the solution).

 One option would be to have a fiber-data indirection, and then you can
 change the task information at some later stage. But in that case we would
 have to ensure that all fibers in our "group" would have finished with


their

 use of the fiber data, and were back at an initialisable position. Not
 impossible, but not necessarily trivial.

 Finally, all that is perhaps feasible when you're totally in control of


all

 the code, but what about if it's in a library (either header, or via some
 API). You cannot turn a thread into a fiber twice, so if you have two libs
 making use of fibers the second one's going to die. (I can't remember
 whether the problem comes not with the second call to


ConvertThreadToFiber()

 or on the next context switch, but I can tell you it does die.)


the solution on Win32 is to use a thread local (contains the info to say
that the thread has been fibered), and everyone use the same fiber library.

by fiber group, I assume you mean a fiber that creates other fibers that
"yield" to it rather than back to main ?
(easy to do).
Aug 11 2003
parent reply "Matthew Wilson" <matthew stlsoft.org> writes:
 Finally, all that is perhaps feasible when you're totally in control of


 all

 the code, but what about if it's in a library (either header, or via




some

 API). You cannot turn a thread into a fiber twice, so if you have two




libs

 making use of fibers the second one's going to die. (I can't remember
 whether the problem comes not with the second call to


 ConvertThreadToFiber()

 or on the next context switch, but I can tell you it does die.)


 the solution on Win32 is to use a thread local (contains the info to say
 that the thread has been fibered), and everyone use the same fiber


library.

So you're saying the answer to the problem of the Win32 fiber implementation
not supporting multiple "fiber-groups" is to not use multiple
"fiber-groups". Well, yeah! That solves the problem, but means that the
usefulness of Win32 fibers is dramatically curtailed.

Isn't that just a clearer statement of my proposition that Win32 fibers are
not all that useful?


 by fiber group, I assume you mean a fiber that creates other fibers that
 "yield" to it rather than back to main ?
 (easy to do).


No, I mean that in your code you call some API function - we assume 3rd
party - which merrily does some fiber stuff. The code then returns to you,
and you carry on merrily, unaware that your thread has been "fiberised". At
a later time you call another function - perhaps one of your own - that also
makes use of fibers.

The minimum harm here is that you will have a memory leak (at least as far
as current versions of Win32 is concerned; future ones may well crash at
this point).

But what if the call to the 3rd party fiber-using API occurs after your
fiber initialisation, and within the time that you are switching between the
fibers. Crash!

Are you telling me that this is desirable functionality?
Aug 11 2003
parent "Mike Wynn" <mike.wynn l8night.co.uk> writes:
"Matthew Wilson" <matthew stlsoft.org> wrote in message
news:bh84p7$11vh$1 digitaldaemon.com...

 by fiber group, I assume you mean a fiber that creates other fibers that
 "yield" to it rather than back to main ?
 (easy to do).


 No, I mean that in your code you call some API function - we assume 3rd
 party - which merrily does some fiber stuff. The code then returns to you,
 and you carry on merrily, unaware that your thread has been "fiberised".


At

 a later time you call another function - perhaps one of your own - that


also

 makes use of fibers.


this is the same as using GWL_USERDATA, only one person can. if you write a
program that uses a library that uses it, you can't in your main app etc..

it does seem that GetCurrentFiber() is o.k. to call before you've converted
your thread to a fiber .....
strangly I get the value 0x00001E00 had expected null.

I see XP has `ConvertFiberToThread`
and server 2003 has ConvertThreadToFiberEx to allow the x86 fpu state to be
saved
as the NT fibers don't store FPU state (now that might be an issue, but I
would have though you would have flushed any FPU op's before yielding
anyway)
server 2003 also has Fiber local storage.

and from the win32 headers I deduced that FS:0x10 was the current fiber data
pointer and that the first enty is a pointer to the FiberData so wrote
SetFiberData and it works on Win2K (see test code)

I could see nothing in the Docs that implied that you can not call
ConvertThreadToFiber twice
but I made the assumption that the fiber state is only stored when you call
SwitchToFiber
and I was right, you can call convertThreadToFiber twice, and switch between
then ...
I've even tried manually setting the current fiber and the switching you can
create some fun loops :) the docs do mention that switching to current fiber
is not advisable but it does work as a way to set the return position of a
fiber. see the test doc. I felt like an afternoon of being abusive to win2K,
better than reading exchange docs on how write code to backup a server over
a network.

it would seem that if you call a library that uses fibers, and when it
returns the current fiber is not the same as it was, you can either manually
convert back to your main fiber and delete the one it was changed to, or
delete your created fiber and continue with the one you have now.
thus as I see it you can protect yourself for a lib that does not check that
the thread has been fiberised first
the docs mention that you can switch to a fiber of a different thread ...
now that opens up some fun doors.

I personally don't see anything that implies co-routines should not be
written using fibers, just like any other interaction with the OS there are
issuee for the user to consider.



begin 666 testco.d

M:F5C=#L-" T*97AT97)N*%=I;F1O=W,I('L-" T*86QI87, =F]I9"!F=6YC
M=&EO;B  3%!63TE$("D 3%!&24)%4E]35$%25%]23U5424Y%.PT*3%!63TE$
M($-R96%T949I8F5R*$173U)$(&1W4W1A8VM3:7IE+$Q01DE"15)?4U1!4E1?
M4D]55$E.12!L<%-T87)T061D<F5S<RQ,4%9/240 ;'!087)A;65T97(I.PT*

M=F5R=%1H<F5A9%1O1FEB97(H3%!63TE$(&QP4&%R86UE=&5R*3L-"E9/240 






M(#T 8V%S="A,4%9/240J*4=E=$-U<G)E;G1&:6)E<B I.PT*"2IF9'  /2!N
M97=F9#L-




M97-T(&9U;F, *')E=&X =&\ )7 I7&XB+"!C87-T*&EN="EP87)A;2 I.PT*




M"0EP<FEN=&8H(")F:6)E<B E>%QN(BP 8V%S="AI;G0I9F( *3L-" D)3%!6
M3TE$(&9D(#T

M;B!E("D >PT*"0EP<FEN=&8H(")C875G:'0 82!E>&-E<'1I;VY<;B( *3L-



M96YT1FEB97(H*3L-" D)<')I;G1F*" B9FEB97( )7A<;B(L(&-A<W0H:6YT

M=&8H(")F:6)E<B!D871A("5X7&XB+"!C87-T*&EN="EF9" I.PT*"7T 8V%T
M8V  *"!%>&-E<'1I;VX 92 I('L-" D)<')I;G1F*" B8V%U9VAT(&$ 97AC

M3TE$*3!X.3D *3L-" D)3%!63TE$(&9B(#T 1V5T0W5R<F5N=$9I8F5R*"D[

M4%9/240 9F0 /2!'971&:6)E<D1A=&$H*3L-" D)<')I;G1F*" B9FEB97( 

M=&5&:6)E<B  ,38Y.3DL('1E<W1?9G5N8RP 82 I.PT*"0E3=VET8VA4;T9I


M>&-E<'1I;VX 92 I('L-" D)<')I;G1F*" B8V%U9VAT(&$ 97AC97!T:6]N



M96YT1FEB97(H(&-A<W0H3%!63TE$*3!X,44P," I.PT*"0EB(#T 0V]N=F5R
M=%1H<F5A9%1O1FEB97(H(&-A<W0H3%!63TE$*3$ *3L-" D)<')I;G1F*" B
M03HE>"P 0CHE>%QN(BP 8V%S="AI;G0I82P 8V%S="AI;G0I8B I.PT*"0ET
M<GD >PT*"0D)3%!63TE$(&9B(#T
M<FEN=&8H(")F:6)E<B E>%QN(BP 8V%S="AI;G0I9F( *3L-" D)"4Q05D])
M1"!F9" ]($=E=$9I8F5R1&%T82 I.PT*"0D)<')I;G1F*" B9FEB97( 9&%T
M82 E>%QN(BP 8V%S="AI;G0I9F0 *3L-" D)?2!C871C:" H($5X8V5P=&EO
M;B!E("D >PT*"0D)<')I;G1F*" B8V%U9VAT(&$ 97AC97!T:6]N("( *3L-
M" D)?0T*"0E,4%9/240 8R ]($-R96%T949I8F5R*" Q-CDY.2P =&5S=%]F





M"5-W:71C:%1O1FEB97(H(&( *3L-" D)+R\ 82!W:6QL(&YO=R!C;VUE(&)A
M8VL =&\ :&5R90T*"0EP<FEN=&8H(")F:6YA;&QY(&$ =V%S(&-A;&QE9"!A
M9V%I;B( *3L-" D)8V]U;G1E<B ](#DY.PT*"7T-" EI9B H(&-O=6YT97( 
M/" Q," I('L-" D)=')Y('L-" D)"2\O(')E86P 86)U<VEV92!D;V-S('=A






M"0D)4W=I=&-H5&]&:6)E<B  8B I.PT*"0D)"7!R:6YT9B  (G-W:71C:"!T
M;R!D('=O=6QD(&AA=F4 9V]T('EO=2!H97)E(&)U="!I="!W:6QL(&YO=%QN
M(B I.PT*"0D)?0T*"0D)4W=I=&-H5&]&:6)E<B  92 I.PT*"0D)<')I;G1F
M*" B9%LR75QN(B I.PT*"0D):68 *"!C;W5N=&5R(#X ,3 P("D >PT*"0D)
M"7!R:6YT9B  (F1;,ET +3X 85QN(B I.PT*"0D)"5-W:71C:%1O1FEB97(H
M(&$ *3L-" D)"0EP<FEN=&8H(")A("T^(&1;,UU<;B( *3L-" D)"7T-" D)
M"7!R:6YT9B  (F1;,UU<;B( *3L-" D)?2!C871C:" H($5X8V5P=&EO;B!E
M("D >PT*"0D)<')I;G1F*" B8V%U9VAT(&$ 97AC97!T:6]N("( *3L-" D)


M9B H($=E=$-U<G)E;G1&:6)E<B I("$]/2!A("D >PT*"0D)<')I;G1F*")W

M1V5T0W5R<F5N=$9I8F5R*"D[("\O(&-F('=I;&P =6YW:6YD(&)A8VL =&\ 
M=VAE;F-E(&ET(&-A;64-" D)"5-E=$-U<G)E;G1&:6)E<B  82 I.PT*"0D)

M=&\ 82!S:&]U;&0 96YD('5P(&AE<F4-" D)"7!R:6YT9B  (G=E(&%R92!G


M>PT*"0EP<FEN=&8H(")C875G:'0 82!E>&-E<'1I;VX (B I.PT*"7T-" T*
M"7!R:6YT9B  (F1O;F4 03HE>"P 0CHE>%QN(BP 8V%S="AI;G0I82P 8V%S

`
end
Aug 11 2003
prev sibling parent reply "Walter" <walter digitalmars.com> writes:
"Sean L. Palmer" <palmer.sean verizon.net> wrote in message
news:bh6euu$2ci7$1 digitaldaemon.com...

 Fibers are as lightweight as you want to make them.  They have almost zero
 overhead compared with threads.  All it is is swapping to a different


stack;

 very similar to cooperative multitasking.

 You can implement them yourself using nothing more than setjmp/longjmp (so
 long as you're able to figure out what the stack pointer is in the


registers

 structure, which is platform dependent).  Or you can use Windows fibers,


but

 you gain nothing by it except avoiding the hairy details of switching


stacks

 and registers.

 The compiler can internally implement fibers however it wants;  it's


really

 quite simple and I can give you advice on implementation if you'd like.


You

 would want a custom implementation for each platform to get the most
 performance.

 My coworkers used to call this "stack twiddling".  ;)  It can sure clear


up

 the logic behind a state machine like you wouldn't believe.

 Coroutines are just language syntax that enables transparent utilization


of

 fibers.  "Yield" keyword just calls out to the fiber manager and saves the
 state of this fiber and lets the next fiber run, or switches back to the
 main fiber.  Calling a coroutine starts a new fiber or switches to one
 that's already running, resuming at just past where the previous yield


left

 off.

 You don't need language support to do this, but it would be cleaner, and


the

 portability hassle of switching stacks would be best done in the compiler


or

 standard library.


I'd like to eventually add coroutines to D, but there are other things that
need to be done first. If you'd like to do a coroutine class, sort of like
some of the C++ ones out there, that could become part of Phobos. It could
hide the portability issues.
Aug 10 2003
parent reply "Mike Wynn" <mike.wynn l8night.co.uk> writes:
import c.stdio;
import windows;
import object;

extern(Windows) {

alias void function( LPVOID ) LPFIBER_START_ROUTINE;
LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
lpStartAddress,LPVOID lpParameter);
VOID DeleteFiber(LPVOID lpFiber);
LPVOID ConvertThreadToFiber(LPVOID lpParameter);
VOID SwitchToFiber(LPVOID lpFiber);
}

// T is the return type from the co-routine
// U is the user context for the co-routine
class CoManagerBase {
 static LPVOID mainFiber = null;
 Exception throw_me = null;
 LPVOID this_fiber  = null;
 bit delete_me      = false;

 static void initForCoRoutines() {
  if ( mainFiber === null ) {
   mainFiber = ConvertThreadToFiber( null );
  }
 }
 ~this() {
  if ( this_fiber ) { endCoRoutine(); }
 }
 void createCoRoutine( LPFIBER_START_ROUTINE start_up ) {
  assert( this_fiber === null );
  this_fiber = CreateFiber( 16384, start_up, cast(LPVOID)this );
 }
 void yeildCoRoutine() {
  assert( CoManagerBase.mainFiber );
  SwitchToFiber( CoManagerBase.mainFiber );   // fiber->main fiber
(original)
 }

 void deleteCoRoutine() {
  assert( this_fiber );
  DeleteFiber( this_fiber );
  this_fiber = null;
  delete_me  = false;
 }

 void endCoRoutine() {
  delete_me = true;
  SwitchToFiber( CoManagerBase.mainFiber );
 }

 void processCoRoutine() {
  assert( this_fiber );
  SwitchToFiber( this_fiber );
 }

 bit isActive() { return (this_fiber !== null); }
}

template coroutine( T, U ) {
 alias void (*user_func)( CoManagerBase cm, U uservalue );

 class CoManager : CoManagerBase {
  user_func func;
  U         value;
  T         return_value;
 public:
  this() {
   initForCoRoutines();
  }
  this( user_func func0, U user_value ) {
   this();
   begin( func0, user_value );
  }
  void begin( user_func func0, U user_value ) {
   assert( isActive() == false );
   func = func0;
   value = user_value;
   createCoRoutine( fiberCoRoutine );
  }

  void yield( T rv ) {
   return_value = rv;
   yeildCoRoutine();
  }

  void end( T rv ) {
   return_value = rv;
   endCoRoutine();
  }

  T process() {
   processCoRoutine();
   if ( throw_me )  { throw throw_me; }
   if ( delete_me ) { deleteCoRoutine(); }
   return return_value;
  }
 }
 extern (Windows) void
 fiberCoRoutine( LPVOID param )
 {
  CoManager cm = cast(CoManager)param;
  cm.func( cm, cm.value );
  while( true ){
   cm.throw_me = new Exception( "CoRoutine overrun" );
   SwitchToFiber( CoManagerBase.mainFiber );   // fiber->main fiber
(original)
  }
 }
// alias void function( CoManager mgr, U uservalue ) user_func;
}

struct Range { int start, end; }
instance coroutine( int, Range ) cr_I_R;
instance coroutine( int, int ) cr_I_I;

void inc_by( CoManagerBase mgr, int incr ) {
 int i = 0;
 while(true) {
  i += incr;
  (cast(cr_I_I.CoManager)mgr).yield( i );
 }
}

void counter( CoManagerBase mgr, Range r ) {
 int i = r.start;
 while( i < r.end ) {
  (cast(cr_I_R.CoManager)mgr).yield( i );
  i++;
 }
 (cast(cr_I_R.CoManager)mgr).end( i );
}

int main( char[][] argv )
{
 static Range r = { start:4, end:9 };
 cr_I_I.CoManager coI = new cr_I_I.CoManager( cast(cr_I_I.user_func)&inc_by,
11 );
 cr_I_R.CoManager coR = new cr_I_R.CoManager( cast(cr_I_R.user_func)counter,
r );

 printf( "being some corountines\n" );
 while( coR.isActive() ) {
  printf( "   incrementer : %d\n", coI.process() );
  printf( "counter[range] : %d\n", coR.process() );
 }
 printf( "*done*\n" );

 return 0;
}
----------------

how's that for proof that fibers are A easy and B do the job.


 I'd like to eventually add coroutines to D, but there are other things


that

 need to be done first. If you'd like to do a coroutine class, sort of like
 some of the C++ ones out there, that could become part of Phobos. It could
 hide the portability issues.


I've attached the versions I'd rather use (less casting) but form some
reason (looks like a forward ref problem)
I cant define two mutually dependant types within a template :(



begin 666 testco_fails.d

M:F5C=#L-" T*97AT97)N*%=I;F1O=W,I('L-" T*86QI87, =F]I9"!F=6YC
M=&EO;B  3%!63TE$("D 3%!&24)%4E]35$%25%]23U5424Y%.PT*3%!63TE$
M($-R96%T949I8F5R*$173U)$(&1W4W1A8VM3:7IE+$Q01DE"15)?4U1!4E1?
M4D]55$E.12!L<%-T87)T061D<F5S<RQ,4%9/240 ;'!087)A;65T97(I.PT*

M=F5R=%1H<F5A9%1O1FEB97(H3%!63TE$(&QP4&%R86UE=&5R*3L-"E9/240 

M=&AE(')E='5R;B!T>7!E(&9R;VT =&AE(&-O+7)O=71I;F4-"B\O(%4 :7, 
M=&AE('5S97( 8V]N=&5X="!F;W( =&AE(&-O+7)O=71I;F4-"F-L87-S($-O
M36%N86=E<D)A<V4 >PT*"7-T871I8R!,4%9/240 ;6%I;D9I8F5R(#T ;G5L
M;#L-" E%>&-E<'1I;VX =&AR;W=?;64 /2!N=6QL.PT*"4Q05D])1"!T:&ES





M?0T*"79O:60 8W)E871E0V]2;W5T:6YE*"!,4$9)0D527U-405)47U)/551)

M;" I.PT*"0ET:&ES7V9I8F5R(#T
M=%]U<"P 8V%S="A,4%9/240I=&AI<R I.PT*"7T-" EV;VED('EE:6QD0V]2
M;W5T:6YE*"D >PT*"0EA<W-E<G0H($-O36%N86=E<D)A<V4N;6%I;D9I8F5R

M("D[("  +R\ 9FEB97(M/FUA:6X 9FEB97( *&]R:6=I;F%L*0T*"7T-" EV
M;VED(&1E;&5T94-O4F]U=&EN92 I('L-" D)87-S97)T*"!T:&ES7V9I8F5R

M97( /2!N=6QL.PT*"0ED96QE=&5?;64 (#T






M" EA;&EA<R!V;VED(" J=7-E<E]F=6YC*2  0V]-86YA9V5R(&UG<BP 52!U
M<V5R=F%L=64 *3L-" T*"6-L87-S($-O36%N86=E<B Z($-O36%N86=E<D)A
M<V4 >PT*"0EU<V5R7V9U;F, 9G5N8SL-" D)52  ("  ("  ('9A;'5E.PT*
M"0E4("  ("  ("  <F5T=7)N7W9A;'5E.PT*"7!U8FQI8SH-" D)=&AI<R I


M"0EB96=I;B  9G5N8S L('5S97)?=F%L=64 *3L-" D)?0T*"0EV;VED(&)E
M9VEN*"!U<V5R7V9U;F, 9G5N8S L(%4 =7-E<E]V86QU92 I('L-" D)"6%S
M<V5R="  :7-!8W1I=F4H*2 ]/2!F86QS92 I.PT*"0D)9G5N8R ](&9U;F,P
M.PT*"0D)=F%L=64 /2!U<V5R7W9A;'5E.PT*"0D)8W)E871E0V]2;W5T:6YE



M7W9A;'5E(#T

M=&AR;W=?;64 *2  >R!T:')O=R!T:')O=U]M93L ?0T*"0D):68 *"!D96QE


M"69I8F5R0V]2;W5T:6YE*"!,4%9/240 <&%R86T *0T*"7L-" D)0V]-86YA
M9V5R(&-M(#T
M+"!C;2YV86QU92 I.PT*"0EW:&EL92  =')U92 I>PT*"0D)8VTN=&AR;W=M

M"5-W:71C:%1O1FEB97(H($-O36%N86=E<D)A<V4N;6%I;D9I8F5R("D[("  

M86QI87, =F]I9"!F=6YC=&EO;B  0V]-86YA9V5R(&UG<BP 52!U<V5R=F%L




M;F-R("D >PT*"6EN="!I(#T ,#L-


M(#T <BYS=&%R=#L-" EW:&EL92  :2 \('(N96YD("D >PT*"0EM9W(N>6EE
M;&0H(&D *3L-" D):2LK.PT*"7T-"GT-" T*:6YT(&UA:6XH(&-H87);75M=
M(&%R9W8 *0T*>PT*"7-T871I8R!286YG92!R(#T >R!S=&%R=#HT+"!E;F0Z


M8W)?25]2+D-O36%N86=E<B!C;U( /2!N97< 8W)?25]2+D-O36%N86=E<B  
M8V%S="AC<E])7U(N=7-E<E]F=6YC*6-O=6YT97(L('( *3L-" T*"7!R:6YT
M9B  (F)E:6YG('-O;64 8V]R;W5N=&EN97-<;B( *3L-" EW:&EL92  8V]2

M)61<;B(L(&-O22YP<F]C97-S*"D *3L-" D)<')I;G1F*" B8V]U;G1E<EMR
M86YG95T .B E9%QN(BP 8V]2+G!R;V-E<W,H*2 I.PT*"7T-" EP<FEN=&8H

`
end
Aug 10 2003
parent "Walter" <walter digitalmars.com> writes:
I'm going to have to study it a bit. Can you make it work on linux? Write a
doc file for it?

"Mike Wynn" <mike.wynn l8night.co.uk> wrote in message
news:bh711f$2td5$1 digitaldaemon.com...

 import c.stdio;
 import windows;
 import object;

 extern(Windows) {

 alias void function( LPVOID ) LPFIBER_START_ROUTINE;
 LPVOID CreateFiber(DWORD dwStackSize,LPFIBER_START_ROUTINE
 lpStartAddress,LPVOID lpParameter);
 VOID DeleteFiber(LPVOID lpFiber);
 LPVOID ConvertThreadToFiber(LPVOID lpParameter);
 VOID SwitchToFiber(LPVOID lpFiber);
 }

 // T is the return type from the co-routine
 // U is the user context for the co-routine
 class CoManagerBase {
  static LPVOID mainFiber = null;
  Exception throw_me = null;
  LPVOID this_fiber  = null;
  bit delete_me      = false;

  static void initForCoRoutines() {
   if ( mainFiber === null ) {
    mainFiber = ConvertThreadToFiber( null );
   }
  }
  ~this() {
   if ( this_fiber ) { endCoRoutine(); }
  }
  void createCoRoutine( LPFIBER_START_ROUTINE start_up ) {
   assert( this_fiber === null );
   this_fiber = CreateFiber( 16384, start_up, cast(LPVOID)this );
  }
  void yeildCoRoutine() {
   assert( CoManagerBase.mainFiber );
   SwitchToFiber( CoManagerBase.mainFiber );   // fiber->main fiber
 (original)
  }

  void deleteCoRoutine() {
   assert( this_fiber );
   DeleteFiber( this_fiber );
   this_fiber = null;
   delete_me  = false;
  }

  void endCoRoutine() {
   delete_me = true;
   SwitchToFiber( CoManagerBase.mainFiber );
  }

  void processCoRoutine() {
   assert( this_fiber );
   SwitchToFiber( this_fiber );
  }

  bit isActive() { return (this_fiber !== null); }
 }

 template coroutine( T, U ) {
  alias void (*user_func)( CoManagerBase cm, U uservalue );

  class CoManager : CoManagerBase {
   user_func func;
   U         value;
   T         return_value;
  public:
   this() {
    initForCoRoutines();
   }
   this( user_func func0, U user_value ) {
    this();
    begin( func0, user_value );
   }
   void begin( user_func func0, U user_value ) {
    assert( isActive() == false );
    func = func0;
    value = user_value;
    createCoRoutine( fiberCoRoutine );
   }

   void yield( T rv ) {
    return_value = rv;
    yeildCoRoutine();
   }

   void end( T rv ) {
    return_value = rv;
    endCoRoutine();
   }

   T process() {
    processCoRoutine();
    if ( throw_me )  { throw throw_me; }
    if ( delete_me ) { deleteCoRoutine(); }
    return return_value;
   }
  }
  extern (Windows) void
  fiberCoRoutine( LPVOID param )
  {
   CoManager cm = cast(CoManager)param;
   cm.func( cm, cm.value );
   while( true ){
    cm.throw_me = new Exception( "CoRoutine overrun" );
    SwitchToFiber( CoManagerBase.mainFiber );   // fiber->main fiber
 (original)
   }
  }
 // alias void function( CoManager mgr, U uservalue ) user_func;
 }

 struct Range { int start, end; }
 instance coroutine( int, Range ) cr_I_R;
 instance coroutine( int, int ) cr_I_I;

 void inc_by( CoManagerBase mgr, int incr ) {
  int i = 0;
  while(true) {
   i += incr;
   (cast(cr_I_I.CoManager)mgr).yield( i );
  }
 }

 void counter( CoManagerBase mgr, Range r ) {
  int i = r.start;
  while( i < r.end ) {
   (cast(cr_I_R.CoManager)mgr).yield( i );
   i++;
  }
  (cast(cr_I_R.CoManager)mgr).end( i );
 }

 int main( char[][] argv )
 {
  static Range r = { start:4, end:9 };
  cr_I_I.CoManager coI = new cr_I_I.CoManager(


cast(cr_I_I.user_func)&inc_by,

 11 );
  cr_I_R.CoManager coR = new cr_I_R.CoManager(


cast(cr_I_R.user_func)counter,

 r );

  printf( "being some corountines\n" );
  while( coR.isActive() ) {
   printf( "   incrementer : %d\n", coI.process() );
   printf( "counter[range] : %d\n", coR.process() );
  }
  printf( "*done*\n" );

  return 0;
 }
 ----------------

 how's that for proof that fibers are A easy and B do the job.


 I'd like to eventually add coroutines to D, but there are other things


 that

 need to be done first. If you'd like to do a coroutine class, sort of




like

 some of the C++ ones out there, that could become part of Phobos. It




could

 hide the portability issues.


 I've attached the versions I'd rather use (less casting) but form some
 reason (looks like a forward ref problem)
 I cant define two mutually dependant types within a template :(
Aug 10 2003