www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Hotfix release vibe.d 0.7.28

reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
This is a small bugfix release that mainly fixes two critical regressions:

  - FreeListRef!T, which is used heavily in the HTTP server code, stored
    its reference count in an unallocated memory region, leading to
    possible memory leaks or memory corruption

  - A TCP connection with a non-empty write buffer that got closed by
    the remote peer and locally at the same time could result in the
    calling task to starve (i.e. it got never resumed after yielding
    execution). In particular, this could happen when accessing HTTPS
    servers with the HTTP client in conjunction with "Connection: close".

http://vibed.org/blog/posts/vibe-release-0.7.28
Feb 27 2016
next sibling parent reply Sebastiaan Koppe <mail skoppe.eu> writes:
On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig wrote:

 This is a small bugfix release that mainly fixes two critical 
 regessions


Great. Thanks for the quick release!
Feb 27 2016
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 28.02.2016 um 02:51 schrieb Sebastiaan Koppe:

 On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig wrote:

 This is a small bugfix release that mainly fixes two critical regessions


 Great. Thanks for the quick release!


Thanks for taking the time to do an in-depth analysis!
Feb 28 2016
prev sibling parent reply sigod <sigod.mail gmail.com> writes:
On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig wrote:

 This is a small bugfix release that mainly fixes two critical 
 regressions:

  - FreeListRef!T, which is used heavily in the HTTP server 
 code, stored
    its reference count in an unallocated memory region, leading 
 to
    possible memory leaks or memory corruption

  - A TCP connection with a non-empty write buffer that got 
 closed by
    the remote peer and locally at the same time could result in 
 the
    calling task to starve (i.e. it got never resumed after 
 yielding
    execution). In particular, this could happen when accessing 
 HTTPS
    servers with the HTTP client in conjunction with 
 "Connection: close".

 http://vibed.org/blog/posts/vibe-release-0.7.28


You forgot to update site header.

Is there any plans on when big split will happen?
Feb 28 2016
parent reply =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 29.02.2016 um 00:47 schrieb sigod:

 On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig wrote:

 This is a small bugfix release that mainly fixes two critical
 regressions:

  - FreeListRef!T, which is used heavily in the HTTP server code, stored
    its reference count in an unallocated memory region, leading to
    possible memory leaks or memory corruption

  - A TCP connection with a non-empty write buffer that got closed by
    the remote peer and locally at the same time could result in the
    calling task to starve (i.e. it got never resumed after yielding
    execution). In particular, this could happen when accessing HTTPS
    servers with the HTTP client in conjunction with "Connection: close".

 http://vibed.org/blog/posts/vibe-release-0.7.28


 You forgot to update site header.


Thanks, also forgot the documentation (even if nothing has changed).


 Is there any plans on when big split will happen?


It will be a step-by-step process. I'm currently working on a new 
version of the `vibe.core` package that contains some large changes 
under the hood. Once that is in a functional state, I'll look into how 
to enable optional replacement of the existing vibe:core package by this 
new, separately hosted vibe-core package. vibe:core, at that point, will 
only receive bug fixes and continues to live for a while (let's say a 
year or one and a half).

The same procedure will then happen for vibe:http (the new package will 
include HTTP/2 support) and the other sub packages.

All of the new packages will get a version number of 1.0.0, once they 
can be considered reasonably stable.

One unfortunate aspect of my current work on vibe-core is that I'm 
building on a new event loop abstraction that I built as a prototype to 
see where the performance bottlenecks of the current system are. 
libasync was too slow and it had a too complicated structure to make 
quick tests for improving performance. Now I'm leaning towards 
finalizing the new prototype library and proposing it for Phobos 
inclusion at some point.
Feb 28 2016
next sibling parent reply Jonas Drewsen <nospam4321 hotmail.com> writes:
On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 Now I'm leaning towards finalizing the new prototype library 
 and proposing it for Phobos inclusion at some point.


Would that library support the same event sources as libasync ie. 
filesystem, notification, sockets etc?

I really think this kind of thing is missing in phobos atm. no 
matter if it is a new lib from you or libasync+optimizations.
Feb 29 2016
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 29.02.2016 um 09:36 schrieb Jonas Drewsen:

 On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 Now I'm leaning towards finalizing the new prototype library and
 proposing it for Phobos inclusion at some point.


 Would that library support the same event sources as libasync ie.
 filesystem, notification, sockets etc?


Yes, it would be sockets, files, file/directory change watchers, timers, 
manual events, posix signals and DNS lookups.


 I really think this kind of thing is missing in phobos atm. no matter if
 it is a new lib from you or libasync+optimizations.
Feb 29 2016
prev sibling next sibling parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 Am 29.02.2016 um 00:47 schrieb sigod:

 On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig 
 wrote:

 This is a small bugfix release that mainly fixes two critical
 regressions:

  - FreeListRef!T, which is used heavily in the HTTP server 
 code, stored
    its reference count in an unallocated memory region, 
 leading to
    possible memory leaks or memory corruption

  - A TCP connection with a non-empty write buffer that got 
 closed by
    the remote peer and locally at the same time could result 
 in the
    calling task to starve (i.e. it got never resumed after 
 yielding
    execution). In particular, this could happen when 
 accessing HTTPS
    servers with the HTTP client in conjunction with 
 "Connection: close".

 http://vibed.org/blog/posts/vibe-release-0.7.28


 You forgot to update site header.


 Thanks, also forgot the documentation (even if nothing has 
 changed).


 Is there any plans on when big split will happen?


 It will be a step-by-step process. I'm currently working on a 
 new version of the `vibe.core` package that contains some large 
 changes under the hood. Once that is in a functional state, 
 I'll look into how to enable optional replacement of the 
 existing vibe:core package by this new, separately hosted 
 vibe-core package. vibe:core, at that point, will only receive 
 bug fixes and continues to live for a while (let's say a year 
 or one and a half).

 The same procedure will then happen for vibe:http (the new 
 package will include HTTP/2 support) and the other sub packages.

 All of the new packages will get a version number of 1.0.0, 
 once they can be considered reasonably stable.

 One unfortunate aspect of my current work on vibe-core is that 
 I'm building on a new event loop abstraction that I built as a 
 prototype to see where the performance bottlenecks of the 
 current system are. libasync was too slow and it had a too 
 complicated structure to make quick tests for improving 
 performance. Now I'm leaning towards finalizing the new 
 prototype library and proposing it for Phobos inclusion at some 
 point.


Hi Sonke,

I'm really interested in your work on a new event loop 
abstraction. One of the big issues for the project I'm working on 
is that the current implementation is not
 nogc and nothrow (while most of my code that doesn't interact 
with vibe.d is nothrow,  nogc and where possible pure).
Another thing that I would like to request is support for 
std.experimental.allocator. I need to be able to provide an 
allocator through which all vibe-core allocations should happen.
Just to clarify, I'm only interested in having a  nogc/nothrow 
event loop, as my project is a rather low-level (it is meant to 
be used both from C and D code) and I won't need the other parts 
of the framework (like web, db, etc.). And I think it's OK to use 
the GC for application-level logic.
Feb 29 2016
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 29.02.2016 um 11:20 schrieb ZombineDev:

 Hi Sonke,

 I'm really interested in your work on a new event loop abstraction. One
 of the big issues for the project I'm working on is that the current
 implementation is not
  nogc and nothrow (while most of my code that doesn't interact with
 vibe.d is nothrow,  nogc and where possible pure).
 Another thing that I would like to request is support for
 std.experimental.allocator. I need to be able to provide an allocator
 through which all vibe-core allocations should happen.
 Just to clarify, I'm only interested in having a  nogc/nothrow event
 loop, as my project is a rather low-level (it is meant to be used both
 from C and D code) and I won't need the other parts of the framework
 (like web, db, etc.). And I think it's OK to use the GC for
 application-level logic.


Everything is  safe and nothrow as far as possible. The plan is also to 
basically have no dynamic memory allocations after the warmup phase. 
However,  nogc is still difficult, because still a lot of Phobos and 
Druntime are not annotated, and because that would mean that all 
callbacks would have to be  nogc, too. Especially the latter can only be 
solved by converting the API to take callbacks as an alias template 
parameter, but if possible I'd like to keep the possibility of passing 
an `interface` around to hide the implementation...
Feb 29 2016
prev sibling next sibling parent reply Georgi D <georgid outlook.com> writes:
On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 The same procedure will then happen for vibe:http (the new 
 package will include HTTP/2 support) and the other sub packages.


This is great news. Will the new HTTP package support an endpoint 
address and transport abstraction so it can be used over Unix 
Domain sockets, Named/Annonymous pipes and other stream types?

I have looked into implementing Unix Domain sockets support for 
vibe.d(more specifically the reverse proxy module) and noticed 
that enabling HTTP over UDS would be a fairly big and complicated 
change.

Working on it is still on my todo list but if the the abstraction 
is going to change anyway I might wait until it has settled a bit.

I am also willing to look and help with the abstractions before 
they are included in an official release if it would not 
interfere too much with your work.
Feb 29 2016
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 29.02.2016 um 11:26 schrieb Georgi D:

 On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 The same procedure will then happen for vibe:http (the new package
 will include HTTP/2 support) and the other sub packages.


 This is great news. Will the new HTTP package support an endpoint
 address and transport abstraction so it can be used over Unix Domain
 sockets, Named/Annonymous pipes and other stream types?

 I have looked into implementing Unix Domain sockets support for
 vibe.d(more specifically the reverse proxy module) and noticed that
 enabling HTTP over UDS would be a fairly big and complicated change.

 Working on it is still on my todo list but if the the abstraction is
 going to change anyway I might wait until it has settled a bit.

 I am also willing to look and help with the abstractions before they are
 included in an official release if it would not interfere too much with
 your work.


It would be pretty straight forward to add a low-level HTTP protocol 
layer that works on an existing ConnectionStream. I'll keep that in mind.
Mar 01 2016
prev sibling parent reply Eugene Wissner <belka caraus.de> writes:
On Monday, 29 February 2016 at 07:54:09 UTC, Sönke Ludwig wrote:

 Am 29.02.2016 um 00:47 schrieb sigod:

 On Saturday, 27 February 2016 at 16:21:05 UTC, Sönke Ludwig 
 wrote:

 This is a small bugfix release that mainly fixes two critical
 regressions:

  - FreeListRef!T, which is used heavily in the HTTP server 
 code, stored
    its reference count in an unallocated memory region, 
 leading to
    possible memory leaks or memory corruption

  - A TCP connection with a non-empty write buffer that got 
 closed by
    the remote peer and locally at the same time could result 
 in the
    calling task to starve (i.e. it got never resumed after 
 yielding
    execution). In particular, this could happen when 
 accessing HTTPS
    servers with the HTTP client in conjunction with 
 "Connection: close".

 http://vibed.org/blog/posts/vibe-release-0.7.28


 You forgot to update site header.


 Thanks, also forgot the documentation (even if nothing has 
 changed).


 Is there any plans on when big split will happen?


 It will be a step-by-step process. I'm currently working on a 
 new version of the `vibe.core` package that contains some large 
 changes under the hood. Once that is in a functional state, 
 I'll look into how to enable optional replacement of the 
 existing vibe:core package by this new, separately hosted 
 vibe-core package. vibe:core, at that point, will only receive 
 bug fixes and continues to live for a while (let's say a year 
 or one and a half).

 The same procedure will then happen for vibe:http (the new 
 package will include HTTP/2 support) and the other sub packages.

 All of the new packages will get a version number of 1.0.0, 
 once they can be considered reasonably stable.

 One unfortunate aspect of my current work on vibe-core is that 
 I'm building on a new event loop abstraction that I built as a 
 prototype to see where the performance bottlenecks of the 
 current system are. libasync was too slow and it had a too 
 complicated structure to make quick tests for improving 
 performance. Now I'm leaning towards finalizing the new 
 prototype library and proposing it for Phobos inclusion at some 
 point.


Sönke, is your current work on core available somewhere?
Since my vibe.d related work isn't stable it wouldn't a problem 
to use unstable core-component. It is better than rewriting later 
if a lot of things change.
Mar 10 2016
parent =?UTF-8?Q?S=c3=b6nke_Ludwig?= <sludwig outerproduct.org> writes:
Am 10.03.2016 um 20:40 schrieb Eugene Wissner:

 Sönke, is your current work on core available somewhere?
 Since my vibe.d related work isn't stable it wouldn't a problem to use
 unstable core-component. It is better than rewriting later if a lot of
 things change.


It's still in a very early state (barely enough to run a TCP based 
server) and the user facing API will stay more or less backwards 
compatible. The main exceptions are that some callbacks will now have to 
be nothrow and/or  safe and that some types change from class to struct. 
So at this point I'd not recommend this for anything more than quick 
experiments and possibly as the basis for early feedback about certain 
API decisions:

https://github.com/vibe-d/
Mar 11 2016