digitalmars.D - Fiber implementation questions
- Ilya Yaroshenko (11/11) Jun 20 2016 Hello,
- =?UTF-8?Q?S=c3=b6nke_Ludwig?= (12/19) Jun 20 2016 I'm currently working on a successor of vibe.d's core module [1], which
- Dicebot (6/17) Jun 20 2016 No, not without special (expensive) runtime and possibly compiler
- deadalnix (5/14) Jun 20 2016 No, and there are no way to make it safe without significant
- Ilya Yaroshenko (2/7) Jun 21 2016 I thought the code should be added to the GC roots.
- Dicebot (6/11) Jun 21 2016 https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4328
- Steven Schveighoffer (4/15) Jun 21 2016 It appears that Fibers interact with the GC differently:
- Dicebot (6/25) Jun 21 2016 I suspected something like this, it explains a lot. Do you know
- Steven Schveighoffer (3/26) Jun 21 2016 There must be a reason, I think maybe Martin or Sean would know more.
- David Nadlinger (4/20) Jun 21 2016 You don't want to be scanning the unused part of the stack
- Steven Schveighoffer (3/17) Jun 21 2016 OK, that's... pretty obvious, thanks :)
Hello, I am thinking about faster event loop that vibe.d has (does not matter what driver are used). I want to implement (at least DIP) advanced Fiber yielding loop that can allow to eliminate context switching for all kind of event waiters. Q1: Is it save to detach Fiber context from one thread and attach it to another thread? Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why? Best regards, Ilya
Jun 20 2016
=?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> Am 20.06.2016 um 17:43 schrieb Ilya Yaroshenko:Hello, I am thinking about faster event loop that vibe.d has (does not matter what driver are used). I want to implement (at least DIP) advanced Fiber yielding loop that can allow to eliminate context switching for all kind of event waiters.I'm currently working on a successor of vibe.d's core module [1], which is more or less an internal rewrite to yield maximum performance. The vibe.d side definitely plays a part there, but most of the improvements are based on a rewritten event loop abstraction [2], which gives roughly a 3x speedup over both, libevent and libasync*, for a simple HTTP server request benchmark. This event loop abstraction is currently just a proof of concept and still experimental, though.Q1: Is it save to detach Fiber context from one thread and attach it to another thread?It will at least always break TLS variables in one way or another. [1]: https://github.com/vibe-d/vibe-core [2]: https://github.com/vibe-d/eventcore * May have changed in the past weeks/months
Jun 20 2016
On 06/20/2016 06:43 PM, Ilya Yaroshenko wrote:Hello, I am thinking about faster event loop that vibe.d has (does not matter what driver are used). I want to implement (at least DIP) advanced Fiber yielding loop that can allow to eliminate context switching for all kind of event waiters. Q1: Is it save to detach Fiber context from one thread and attach it to another thread?No, not without special (expensive) runtime and possibly compiler support. Primarily because TLS.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Don't know. It indeed sounds wrong though I have never experienced premature collection with fibers so the matter maybe more complicated than it seems.
Jun 20 2016
On Monday, 20 June 2016 at 15:43:27 UTC, Ilya Yaroshenko wrote:Hello, I am thinking about faster event loop that vibe.d has (does not matter what driver are used). I want to implement (at least DIP) advanced Fiber yielding loop that can allow to eliminate context switching for all kind of event waiters. Q1: Is it save to detach Fiber context from one thread and attach it to another thread?No, and there are no way to make it safe without significant penalty for ALL code (not just code using fibers) and breaking the type system.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 20 2016
On Monday, 20 June 2016 at 20:17:04 UTC, deadalnix wrote:On Monday, 20 June 2016 at 15:43:27 UTC, Ilya Yaroshenko wrote:I thought the code should be added to the GC roots.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 21 2016
On 06/20/2016 11:17 PM, deadalnix wrote:https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4328 allocates fiber stack using mmap / malloc / VirtualAlloc and doesn't seem to register that memory in GC. But that may be indirect result of https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4439 call, which is why I am not sure.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 21 2016
On 6/21/16 6:14 AM, Dicebot wrote:On 06/20/2016 11:17 PM, deadalnix wrote:It appears that Fibers interact with the GC differently: https://github.com/dlang/druntime/blob/master/src/core/thread.d#L1611 -Stevehttps://github.com/dlang/druntime/blob/master/src/core/thread.d#L4328 allocates fiber stack using mmap / malloc / VirtualAlloc and doesn't seem to register that memory in GC. But that may be indirect result of https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4439 call, which is why I am not sure.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 21 2016
On Tuesday, 21 June 2016 at 12:18:37 UTC, Steven Schveighoffer wrote:On 6/21/16 6:14 AM, Dicebot wrote:I suspected something like this, it explains a lot. Do you know why this approach was chosen instead of simply registering allocated memory for GC scan? Does GC has special approach for stack scanning which differs from normal memory?On 06/20/2016 11:17 PM, deadalnix wrote:It appears that Fibers interact with the GC differently: https://github.com/dlang/druntime/blob/master/src/core/thread.d#L1611 -Stevehttps://github.com/dlang/druntime/blob/master/src/core/thread.d#L4328 allocates fiber stack using mmap / malloc / VirtualAlloc and doesn't seem to register that memory in GC. But that may be indirect result of https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4439 call, which is why I am not sure.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 21 2016
On 6/21/16 11:43 AM, Dicebot wrote:On Tuesday, 21 June 2016 at 12:18:37 UTC, Steven Schveighoffer wrote:There must be a reason, I think maybe Martin or Sean would know more. -SteveOn 6/21/16 6:14 AM, Dicebot wrote:I suspected something like this, it explains a lot. Do you know why this approach was chosen instead of simply registering allocated memory for GC scan? Does GC has special approach for stack scanning which differs from normal memory?On 06/20/2016 11:17 PM, deadalnix wrote:It appears that Fibers interact with the GC differently: https://github.com/dlang/druntime/blob/master/src/core/thread.d#L1611https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4328 allocates fiber stack using mmap / malloc / VirtualAlloc and doesn't seem to register that memory in GC. But that may be indirect result of https://github.com/dlang/druntime/blob/master/src/core/thread.d#L4439 call, which is why I am not sure.Q2: core.thread.Fiber implementation looks like GC does not scan function stacks for yielded contexts. Why?Doesn't it ?
Jun 21 2016
On Tuesday, 21 June 2016 at 16:12:13 UTC, Steven Schveighoffer wrote:On 6/21/16 11:43 AM, Dicebot wrote:You don't want to be scanning the unused part of the stack region. -DavidOn Tuesday, 21 June 2016 at 12:18:37 UTC, Steven Schveighoffer wrote:There must be a reason, I think maybe Martin or Sean would know more.On 6/21/16 6:14 AM, Dicebot wrote: httpthub.com/dlang/druntime/blob/master/src/core/thread.d#L1611I suspected something like this, it explains a lot. Do you know why this approach was chosen instead of simply registering allocated memory for GC scan? Does GC has special approach for stack scanning which differs from normal memory?
Jun 21 2016
On 6/21/16 3:03 PM, David Nadlinger wrote:On Tuesday, 21 June 2016 at 16:12:13 UTC, Steven Schveighoffer wrote:OK, that's... pretty obvious, thanks :) -SteveOn 6/21/16 11:43 AM, Dicebot wrote:You don't want to be scanning the unused part of the stack region. -DavidOn Tuesday, 21 June 2016 at 12:18:37 UTC, Steven Schveighoffer wrote:There must be a reason, I think maybe Martin or Sean would know more.On 6/21/16 6:14 AM, Dicebot wrote: httpthub.com/dlang/druntime/blob/master/src/core/thread.d#L1611I suspected something like this, it explains a lot. Do you know why this approach was chosen instead of simply registering allocated memory for GC scan? Does GC has special approach for stack scanning which differs from normal memory?
Jun 21 2016