• Richard (Rikki) Andrew Cattermole (21/21) Dec 12 2024 Stackless coroutines, is a way to enable asynchronous
• Atila Neves (12/33) Jan 13 I had trouble understanding the proposal. If I didn't already
• Richard (Rikki) Andrew Cattermole (37/42) Jan 13 what a coroutine is, I wouldn't have found out by reading the abstract.
• Atila Neves (19/47) Jan 15 I haven't read it before; I'm also not an LLM.
• Richard (Rikki) Andrew Cattermole (96/143) Jan 15 Before reading all this, I have something I want to make clear about
• Jin (16/18) Jan 15 I note that with the advent of async/await in JS, development for

Richard (Rikki) Andrew Cattermole <richard cattermole.co.nz> writes:

Stackless coroutines, is a way to enable asynchronous 
programming, for lesser skilled and less knowledgable people 
whilst offering efficient processing of events, safely.

This version of the proposal has been rewritten to account for a 
lack of understanding on the separation of library code versus 
what the language is offering.

And a few changes related to yielding. Yielding is no longer 
guaranteed to be implicit. You may explicitly yield using an 
``await`` statement should you wish to. The library type must 
support implicit yielding if you wish to use it. Both may be used 
on the same type, it is entirely dependent upon the called 
methods attributes.

Lastly, the changes have been made to simplify the descriptor to 
make the implementation within the compiler a little bit easier. 
It does mean that you as a library author have no way to know 
about the functions in the state machine (not that you could have 
done much with them).

Current: 
https://gist.github.com/rikkimax/fe2578e1dfbf66346201fd191db4bdd4/649a5a6cc68c4bfe9f5a62f746a3a90f6b4beaf4

Latest: 
https://gist.github.com/rikkimax/fe2578e1dfbf66346201fd191db4bdd4

Atila Neves <atila.neves gmail.com> writes:

On Thursday, 12 December 2024 at 10:36:50 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 Stackless coroutines, is a way to enable asynchronous 
 programming, for lesser skilled and less knowledgable people 
 whilst offering efficient processing of events, safely.

 This version of the proposal has been rewritten to account for 
 a lack of understanding on the separation of library code 
 versus what the language is offering.

 And a few changes related to yielding. Yielding is no longer 
 guaranteed to be implicit. You may explicitly yield using an 
 ``await`` statement should you wish to. The library type must 
 support implicit yielding if you wish to use it. Both may be 
 used on the same type, it is entirely dependent upon the called 
 methods attributes.

 Lastly, the changes have been made to simplify the descriptor 
 to make the implementation within the compiler a little bit 
 easier. It does mean that you as a library author have no way 
 to know about the functions in the state machine (not that you 
 could have done much with them).

 Current: 
 https://gist.github.com/rikkimax/fe2578e1dfbf66346201fd191db4bdd4/649a5a6cc68c4bfe9f5a62f746a3a90f6b4beaf4

 Latest: 
 https://gist.github.com/rikkimax/fe2578e1dfbf66346201fd191db4bdd4

I had trouble understanding the proposal. If I didn't already 
know what a coroutine is, I wouldn't have found out by reading 
the abstract. There are a few sentences I didn't understand in 
their entirety either such as "If it causes an error, this error 
is guaranteed to be wrong in a multi-threaded application of it.".

My main issue is that I don't think the DIP justifies the need 
for stackless coroutines (which I think are a good idea). It also 
seems more complicated than what other languages have, and I'm 
not sure why that is. Why ` async return` instead of `yield`? Why 
have to add ` async` to the grammar if it looks like an attribute?

"Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:

On 14/01/2025 6:59 AM, Atila Neves wrote:
 I had trouble understanding the proposal. If I didn't already know 

what a coroutine is, I wouldn't have found out by reading the abstract. 
There are a few sentences I didn't understand in their entirety either 
such as "If it causes an error, this error is guaranteed to be wrong in 
a multi-threaded application of it.".

I do not understand why you are having trouble with it.
It has not come up before and Gemini understood it.
It is short, precise and complete to what I mean.

If you want me to change it, I need a lot more feedback including:

- How you are interpreting it
- What questions you had after reading it
- what did you expect it to contain

As it currently stands this is not constructive feedback, there is 
nothing I can do with it. I do not understand what problems you are 
having with it.

 It also seems more complicated than what other languages have, and I'm 
 not sure why that is.

Its not more complicated, but I can understand that it may appear that way.

Other languages can tie the feature to a specific library, which will 
not work for us.

Consider why we cannot: a coroutine language feature is tied to its 
representation in library, which is tied to is eventloop, which is tied 
to sockets, windowing, pipes, processes, thread pool ext.

None of which can be in druntime, has to be Phobos. But we cannot tie a 
language feature to Phobos, and if we do that I cannot experiment prior 
to PhobosV3 to ensure it both works as expected and to learn if any 
further expansion is needed.

Also coroutines are used in both generative and event handling basis, 
they are not the same library wise. Tieing it to just one is going to be 
hell for someone. Most likely me as I'm responsible for the user experience.

 Why | async return| instead of |yield|?

Then ``yield`` would be a keyword, which in turn breaks code which is 
known to exist.

There is no benefit to doing this. But we _could_ do it.

However there is a good question here, why not combine ``await`` 
statement with `` async return``? Well the answer is you may want to 
return a coroutine, which couldn't be differentiated by the compiler.

 Why have to add | async| to the grammar if it looks like an attribute?

All language attributes are in the grammar, there is nothing special 
going on there.

https://dlang.org/spec/grammar.html#attributes

Atila Neves <atila.neves gmail.com> writes:

On Monday, 13 January 2025 at 18:51:27 UTC, Richard (Rikki) 
Andrew Cattermole wrote:
 On 14/01/2025 6:59 AM, Atila Neves wrote:
 I had trouble understanding the proposal. If I didn't already


 I do not understand why you are having trouble with it.
 It has not come up before and Gemini understood it.

I haven't read it before; I'm also not an LLM.

 It is short, precise and complete to what I mean.

I don't think that's the case.

 If you want me to change it, I need a lot more feedback 
 including:

 - How you are interpreting it
 - What questions you had after reading it
 - what did you expect it to contain

Sure. I think the feedback would be quite long, though.

I wonder if it would be better to have a coroutine library first; 
I know that it would be a lot more clumsy to use than it would be 
with language support. But maybe having a library prove itself 
useful first would be the way forward.

 Other languages can tie the feature to a specific library, 
 which will not work for us.

Why is that?

 Consider why we cannot: a coroutine language feature is tied to 
 its representation in library, which is tied to is eventloop, 
 which is tied to sockets, windowing, pipes, processes, thread 
 pool ext.

How is this different in other languages?

 None of which can be in druntime, has to be Phobos.

Why is that?

 Also coroutines are used in both generative and event handling 
 basis, they are not the same library wise. Tieing it to just 
 one is going to be hell for someone. Most likely me as I'm 
 responsible for the user experience.

Again, how is this different in other languages?

 Why | async return| instead of |yield|?

 Then ``yield`` would be a keyword, which in turn breaks code 
 which is known to exist.

C++ got around that with `co_yield`.

 There is no benefit to doing this. But we _could_ do it.

Familiarity would be a benefit.

 All language attributes are in the grammar, there is nothing 
 special going on there.

 https://dlang.org/spec/grammar.html#attributes

For historical reasons, yes. I'm aware one can't attach an 
attribute to `return` otherwise right now, but wherever they 
already work I would argue that `core.attributes` is the way to 
go.

"Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:

Before reading all this, I have something I want to make clear about 
coroutines that probably should be said earlier contextually to it.

There will be people who are not happy with our library design and 
implementation. It will NOT matter what choices we will make, we cannot 
make everyone happy if we limit the language feature to one solution.

This group includes me, due to -betterC (and some other misc concerns).

Alternatively, which is what I've gone with, we can just not do that. We 
can make it work for any library. Then people can do their own thing or 
pick someone elses. This is a strength of D not a weakness.

And the best part? It is not more complicated. It is not more work to 
implement, if anything it is a subset of what you would need to have 
instead. Nor does it give a worse user experience. It is a different 
design with better tradeoffs for us, that is all.

On 15/01/2025 10:10 PM, Atila Neves wrote:
 If you want me to change it, I need a lot more feedback including:

 - How you are interpreting it
 - What questions you had after reading it
 - what did you expect it to contain

 
 Sure. I think the feedback would be quite long, though.

I cannot see a problem with it and I've given evidence that I have good 
reason to not, so a statement like "I don't understand it" is not 
helpful if the goal is to see changes.

So yes please, give me more information that I can take action on!

It may be a good idea to ask Mike for help, this kind of feedback is 
something he is good at (considering his job).

 I wonder if it would be better to have a coroutine library first; I know 
 that it would be a lot more clumsy to use than it would be with language 
 support. But maybe having a library prove itself useful first would be 
 the way forward.

Been there done that.

https://github.com/Project-Sidero/eventloop/tree/master/source/sidero/eventloop/coroutine

It is absolutely hell to write them by hand.


async/await, yes they wrote the library before the language feature and 
were stuck with some less than desirable choices (at least in terms of D 
they are anyway).

This also happens to be why my library is so much hell to write the 
coroutines for currently, because you are effectively replacing the 
language and that is intentional.

 Other languages can tie the feature to a specific library, which will 
 not work for us.

 
 Why is that?

Well for one thing, I am not putting experimental code into Phobos let 
alone druntime for an event loop. This needs to work outside of it.

I need to be able to modify my existing event loop that is designed for 
coroutines in -betterC, to use it. Otherwise I will not be able to find 
any problems it may have or other tunings that will give a better user 
experience once we turn it on.

Plus I see no reason to start tieing this language feature to a specific 
library. We do not box. But we do use templates. We love templates. We 
love generating types and symbols to do this kind of thing. It is both a 
well loved aspect of D, and a very well understood one. Lean into it, 
not against it.

 Consider why we cannot: a coroutine language feature is tied to its 
 representation in library, which is tied to is eventloop, which is 
 tied to sockets, windowing, pipes, processes, thread pool ext.

 
 How is this different in other languages?

As far as I'm aware it is not, but you do have to acknowledge it to 
understand the decision on this front.

 None of which can be in druntime, has to be Phobos.

 
 Why is that?

It is an absolute massive project.

With a ton of platform and runtime specific things.

Trust me, it does not belong in druntime.

You cannot convince me that it is the right place.

 Also coroutines are used in both generative and event handling basis, 
 they are not the same library wise. Tieing it to just one is going to 
 be hell for someone. Most likely me as I'm responsible for the user 
 experience.

 
 Again, how is this different in other languages?

C++ has a massive proposal to handle generative data handling side of 
things. It includes scheduler support, (note that this proposal does not 
need the language to be aware of such things).

I cannot find the paper in question, otherwise I would link it.


data (multiple value returns), the focus is upon event handling.

They are sadly different use cases and are going to result in different 
libraries.

Rust literally ties the language to POSIX specific event loop function 
calls, that end up requiring them to use undocumented API's on Windows 
to make work.

At some point you gotta admit, having the compiler produce a state 
struct with a handle method with everything a library needs to work with 
the language feature looks quite simple in comparison ;)

Building up the state machine and extraction of information such as what 
is returned, how it completes with what types (including exceptions 
ext.) happens in all languages. But they tend to go a step further and 
start messing around with library code, this doesn't, nor would it be to 
our advantage.

 Why | async return| instead of |yield|?

 Then ``yield`` would be a keyword, which in turn breaks code which is 
 known to exist.

 
 C++ got around that with `co_yield`.
 
 There is no benefit to doing this. But we _could_ do it.

 
 Familiarity would be a benefit.

To C++ that has had them for only a couple of years.

 From my perspective, C++ has an ugly solution to the problem, that need 
not exist in terms of syntax.

Now compare it to what I proposed:

- Uses an attribute that exists for the same concept, but in a different 
place in grammar.
- It would still be read in a way that is understood control flow wise, 
even if you did not understand coroutines.
- Does not risk breaking code.

To me this is a much better solution that fits D, rather than blindingly 
copying another language with very different needs in terms of syntax 
than we have.

We don't need to copy C++, nor do we have the same baggage as C++ so our 
choices can be different on this, so why should we?

 All language attributes are in the grammar, there is nothing special 
 going on there.

 https://dlang.org/spec/grammar.html#attributes

 
 For historical reasons, yes. I'm aware one can't attach an attribute to 
 `return` otherwise right now, but wherever they already work I would 
 argue that `core.attributes` is the way to go.

Which has to be imported.

I argue similarly, but there are target audience and language awareness 
to what I recommend.

`` async`` is special, it is used to trigger a head line language 
feature with a very large target audience. Therefore it goes in language.

All of the library attributes in the DIP that the average developer 
doesn't need to know exists, they are in ``core.attributes``.

Plus, coroutines really need to have support for slicing and dicing at 
the parser level. I worked really hard to make that possible for Walter 
due to his issues with ``opApply``. It took weeks of back and forth with 
Adam, for me to come up with the second draft. Just so I could make it 
easier on Walter, but at the same time prevent any of the very large 

teams, they can get nasty.

In a DIP this size, there is a lot of contextual information that 
shouldn't be in it. This is a great example of it.

Jin <nin-jin ya.ru> writes:

On Wednesday, 15 January 2025 at 16:19:37 UTC, Richard (Rikki) 
Andrew Cattermole wrote:

 added async/await

I note that with the advent of async/await in JS, development for 
the browser turned into hell. And when node-fibers (a native 
nodejs extension that adds support for coroutines with a stack at 
runtime) was broken, all hell broke loose on the servers. Briefly 
about async/await problems:

- [Low performance due to the inability to properly optimize the 
code.](https://page.hyoo.ru/#!=btunlj_fp1tum/View'btunlj_fp1tum'.Details=%D0%90%D1%81%D0%B8%D0%BD%D1%85%D1%80%D0%BE%D0%BD%D0%BD%D1%8B%D0%B9%20%D0%BA%D0%B5%D0%B9%D1%81)
- [Different colors of functions that virally affect the call 
stack.](https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/)
- [Inability to abort a deep subtasks without manually drawing a 
CancellationToken since await is not 
owning.](https://hackernoon.com/why-do-you-need-a-cancellation-token-in-c-for-tasks)
- [The need to reinvent the stack as an 
AsyncContext.](https://github.com/tc39/proposal-async-context)