www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Announcing libasync, a cross-platform D event loop

reply Etienne <etcimon gmail.com> writes:
It's finally here: https://github.com/etcimon/libasync

We all know how event loops are the foundation of more popular libraries 
Qt and Nodejs.. we now have a natively compiling async library entirely 
written in D.

This event library was tested on Win32, Linux x64, Mac OS x64, with DMD 
2.066, offers the more low-level async objects: timers, file i/o, dns 
resolver, tcp, udp, listeners, signals (cross-thread), notifications 
(same thread), and more recently (and with great efforts for 
implementing with OS X / BSD) a directory watcher.

e.g. You can run a timer with:

	import std.datetime; import std.stdio; import libasync.all;
	EventLoop evl = new EventLoop;
	auto timer = new AsyncTimer(evl);
	timer.duration(2.seconds).periodic().run({ writeln("Another 2 seconds 
have passed"); });
	while(evl.loop()) continue;

The tests may be most revealing: 
https://github.com/etcimon/libasync/blob/master/source/libasync/test.d

A (lightly tested) vibe.d driver using all those async objects is also 
available and currently ongoing a pull request:

https://github.com/etcimon/vibe.d/tree/native-events

The incentive was to make vibe.d compile in completely native D, I'm now 
moving onto a botan C++ => D wrapper for it, I plan on moving objects to 
D over the years until the TLS library can be completely native. I thank 
Walter for the efforts on extern(C++)

Finally, I release this on the basis of an MIT license, looking forward 
to seeing our community flourishing with yet more native libraries. Code on
Sep 24 2014
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/24/14, 6:13 AM, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync


This is fantastic! We really need something like this in a Facebook project.

Would be appropriate for you to consider making a bid for adding it to 
Phobos? In that case one issue to address is integrating std.concurrency 
with the event loop, i.e. registering an event handler should be 
possible for a thread message as well. The Boost license would need to 
be added. What do you think?


Andrei
Sep 24 2014
parent reply Etienne <etcimon gmail.com> writes:
 This is fantastic! We really need something like this in a Facebook
 project.


That's pleasing to hear, although I'm pretty sure Facebook is far from 
being the only organization who will benefit from this ;)


 Would be appropriate for you to consider making a bid for adding it 


to Phobos?

Thanks for the invitation, I'll start working on a phobos fork with this 
added into std.async and re-licensed to Boost. Would that be an 
appropriate namespace?


 In that case one issue to address is integrating std.concurrency
 with the event loop, i.e. registering an event handler should be
 possible for a thread message as well.


Of course, the libasync.threads module already has a good foothold on 
the strategy, so a newer/better std.concurrency could deprecate parts of 
this module by offering an adapted message queue that allows threads 
with an event loop to communicate with threads that have one or not, in 
both directions.


 Do I understand correctly that it is similar library to boost.asio 


(EventLoop being equivalent of asio::io_service)?

That's right, but instead of io_service::run() you call 
eventloop.loop(). In both cases, it's a wrapper around epoll_wait 
(linux), kqueue (osx/bsd) or MsgWaitForMultipleObjects (windows) with a 
timeout parameter. It waits for the objects supplied by the library. The 
"strands" feature is pretty much covered by the vibe.d tasks if you're 
using the driver I wrote for it.
Sep 24 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/24/14, 9:30 AM, Etienne wrote:

  > This is fantastic! We really need something like this in a Facebook
  > project.

 That's pleasing to hear, although I'm pretty sure Facebook is far from
 being the only organization who will benefit from this ;)

  > Would be appropriate for you to consider making a bid for adding it
 to Phobos?

 Thanks for the invitation, I'll start working on a phobos fork with this
 added into std.async and re-licensed to Boost. Would that be an
 appropriate namespace?


Yes. Thanks!


  > In that case one issue to address is integrating std.concurrency
  > with the event loop, i.e. registering an event handler should be
  > possible for a thread message as well.

 Of course, the libasync.threads module already has a good foothold on
 the strategy, so a newer/better std.concurrency could deprecate parts of
 this module by offering an adapted message queue that allows threads
 with an event loop to communicate with threads that have one or not, in
 both directions.


Yah, I think that's where the most design effort around porting would go.

Andrei
Sep 24 2014
prev sibling parent reply Martin Nowak <code dawg.eu> writes:
On 09/24/2014 06:30 PM, Etienne wrote:

  > This is fantastic! We really need something like this in a Facebook
  > project.

 That's pleasing to hear, although I'm pretty sure Facebook is far from
 being the only organization who will benefit from this ;)


Who doesn't need something like this?


  > Would be appropriate for you to consider making a bid for adding it
 to Phobos?

 Thanks for the invitation, I'll start working on a phobos fork with this
 added into std.async and re-licensed to Boost. Would that be an
 appropriate namespace?


I still have to try out the library, but I'd like to see that.

One thing that always bothers me with async libraries is that now every 
IO class has it async cousin, so there is Socket and AsyncSocket, 
resolveDNS and asyncResolveDNS.
With Fibers and a Scheduler it's actually possible to present the same 
API to asynchronous and synchronous code. I'd really like to see this at 
some point, but a cross-platform event loop in phobos is a great first step.
Sep 25 2014
parent Etienne Cimon <etcimon gmail.com> writes:
On 2014-09-25 19:34, Martin Nowak wrote:

 One thing that always bothers me with async libraries is that now every
 IO class has it async cousin, so there is Socket and AsyncSocket,
 resolveDNS and asyncResolveDNS.
 With Fibers and a Scheduler it's actually possible to present the same
 API to asynchronous and synchronous code. I'd really like to see this at
 some point, but a cross-platform event loop in phobos is a great first
 step.


Exactly, though I'm pretty sure this could be solved by moving vibe core 
to std.vibe, and then adding it as a dependency for std.socket and 
std.concurrency. It doesn't feel right having the future/promise aside 
of fibers with a scheduler, it needs to be done correctly from the start.
Sep 25 2014
prev sibling next sibling parent "Szymon Gatner" <noemail gmail.com> writes:
On Wednesday, 24 September 2014 at 13:13:34 UTC, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular 
 libraries Qt and Nodejs.. we now have a natively compiling 
 async library entirely written in D.

 This event library was tested on Win32, Linux x64, Mac OS x64, 
 with DMD 2.066, offers the more low-level async objects: 
 timers, file i/o, dns resolver, tcp, udp, listeners, signals 
 (cross-thread), notifications (same thread), and more recently 
 (and with great efforts for implementing with OS X / BSD) a 
 directory watcher.

 e.g. You can run a timer with:

 	import std.datetime; import std.stdio; import libasync.all;
 	EventLoop evl = new EventLoop;
 	auto timer = new AsyncTimer(evl);
 	timer.duration(2.seconds).periodic().run({ writeln("Another 2 
 seconds have passed"); });
 	while(evl.loop()) continue;

 The tests may be most revealing: 
 https://github.com/etcimon/libasync/blob/master/source/libasync/test.d

 A (lightly tested) vibe.d driver using all those async objects 
 is also available and currently ongoing a pull request:

 https://github.com/etcimon/vibe.d/tree/native-events

 The incentive was to make vibe.d compile in completely native 
 D, I'm now moving onto a botan C++ => D wrapper for it, I plan 
 on moving objects to D over the years until the TLS library can 
 be completely native. I thank Walter for the efforts on 
 extern(C++)

 Finally, I release this on the basis of an MIT license, looking 
 forward to seeing our community flourishing with yet more 
 native libraries. Code on


Do I understand correctly that it is similar library to 
boost.asio (EventLoop being equivalent of asio::io_service)?
Sep 24 2014
prev sibling next sibling parent =?UTF-8?B?IlRow6lv?= Bueno" <munrek gmx.com> writes:
On Wednesday, 24 September 2014 at 13:13:34 UTC, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular 
 libraries Qt and Nodejs.. we now have a natively compiling 
 async library entirely written in D.


I am very excited about this. Thank you for this awesome 
contribution, you made my day :)
Sep 24 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/24/14, 6:13 AM, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync


http://www.reddit.com/r/programming/comments/2hcm9n/announcing_libasync_a_crossplatform_d_event_loop/

Andrei
Sep 24 2014
prev sibling next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2014-09-24 15:13, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular libraries
 Qt and Nodejs.. we now have a natively compiling async library entirely
 written in D.

 This event library was tested on Win32, Linux x64, Mac OS x64, with DMD
 2.066, offers the more low-level async objects: timers, file i/o, dns
 resolver, tcp, udp, listeners, signals (cross-thread), notifications
 (same thread), and more recently (and with great efforts for
 implementing with OS X / BSD) a directory watcher.


Shouldn't the directory watcher use FSEvents on OS X?

-- 
/Jacob Carlborg
Sep 24 2014
parent reply Etienne <etcimon gmail.com> writes:
On 2014-09-24 3:15 PM, Jacob Carlborg wrote:

 Shouldn't the directory watcher use FSEvents on OS X?


I've thought about it but it isn't compatible with other BSD platforms 
and has no docs about using it with kqueue, it ended up looking more 
complicated and unstable because I could read complaints everywhere I 
looked. It ended up putting me in a direction where I needed separate 
threads with their own event loops and communication through signals, 
which didn't fit in with what I was used to doing from what the other 
platforms offered.

I found the solution with kqueue's vnode, which acts like inotify. It 
doesn't have recursion nor file modification events when watching 
folders so individual files had to be watched and the directories 
checked against cache data every time an event came in. It seems a lot 
more flexible on the long run, and making feature additions / 
compatibility adjustments didn't sound like a nightmare on the long run.
Sep 24 2014
parent reply Jacob Carlborg <doob me.com> writes:
On 24/09/14 22:09, Etienne wrote:


 I've thought about it but it isn't compatible with other BSD platforms
 and has no docs about using it with kqueue, it ended up looking more
 complicated and unstable because I could read complaints everywhere I
 looked. It ended up putting me in a direction where I needed separate
 threads with their own event loops and communication through signals,
 which didn't fit in with what I was used to doing from what the other
 platforms offered.


The only complains I've seen with FSEvents is that it doesn't support 
notifying about file changes, only directory changes. But that has been 
fixed in OS X 10.7 so that complain is moot.


 I found the solution with kqueue's vnode, which acts like inotify. It
 doesn't have recursion nor file modification events when watching
 folders so individual files had to be watched and the directories
 checked against cache data every time an event came in.


That sounds like a quite big limitation.

-- 
/Jacob Carlborg
Sep 24 2014
parent Etienne <etcimon gmail.com> writes:
On 2014-09-25 2:51 AM, Jacob Carlborg wrote:

 The only complains I've seen with FSEvents is that it doesn't support
 notifying about file changes, only directory changes. But that has been
 fixed in OS X 10.7 so that complain is moot.


Good, and according to the stats I've seen, 95% of users are on the 2 
most recent OS X versions!


 I found the solution with kqueue's vnode, which acts like inotify. It
 doesn't have recursion nor file modification events when watching
 folders so individual files had to be watched and the directories
 checked against cache data every time an event came in.


 That sounds like a quite big limitation.


It is, but I managed to get around it. Although I haven't yet 
implemented the renaming/move operation detection. I might as well plan 
on writing the FSEvent implementation for Mac OS X and state some 
limitations with FreeBSD in that case =) Thanks for the hint!
Sep 25 2014
prev sibling next sibling parent ketmar via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Wed, 24 Sep 2014 09:13:31 -0400
Etienne via Digitalmars-d <digitalmars-d puremagic.com> wrote:


 It's finally here: https://github.com/etcimon/libasync
=20
 We all know how event loops are the foundation of more popular
 libraries Qt and Nodejs.. we now have a natively compiling async
 library entirely written in D.


it's great! and it will be even greater ;-) to have this in phobos.

thank you for your work.
Sep 24 2014
prev sibling next sibling parent reply "Zhao Puming" <zhaopuming gmail.com> writes:
Great work Etienne!

will libasync make it into phobos?

On Wednesday, 24 September 2014 at 13:13:34 UTC, Etienne wrote:

 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular 
 libraries Qt and Nodejs.. we now have a natively compiling 
 async library entirely written in D.

 This event library was tested on Win32, Linux x64, Mac OS x64, 
 with DMD 2.066, offers the more low-level async objects: 
 timers, file i/o, dns resolver, tcp, udp, listeners, signals 
 (cross-thread), notifications (same thread), and more recently 
 (and with great efforts for implementing with OS X / BSD) a 
 directory watcher.

 e.g. You can run a timer with:

 	import std.datetime; import std.stdio; import libasync.all;
 	EventLoop evl = new EventLoop;
 	auto timer = new AsyncTimer(evl);
 	timer.duration(2.seconds).periodic().run({ writeln("Another 2 
 seconds have passed"); });
 	while(evl.loop()) continue;

 The tests may be most revealing: 
 https://github.com/etcimon/libasync/blob/master/source/libasync/test.d

 A (lightly tested) vibe.d driver using all those async objects 
 is also available and currently ongoing a pull request:

 https://github.com/etcimon/vibe.d/tree/native-events

 The incentive was to make vibe.d compile in completely native 
 D, I'm now moving onto a botan C++ => D wrapper for it, I plan 
 on moving objects to D over the years until the TLS library can 
 be completely native. I thank Walter for the efforts on 
 extern(C++)

 Finally, I release this on the basis of an MIT license, looking 
 forward to seeing our community flourishing with yet more 
 native libraries. Code on
Sep 24 2014
parent reply "Sean Kelly" <sean invisibleduck.org> writes:
On Thursday, 25 September 2014 at 03:29:03 UTC, Zhao Puming wrote:

 Great work Etienne!

 will libasync make it into phobos?


I certainly hope so.  We need async functionality if we're to 
ever have a decent socket package in Phobos.
Sep 25 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Thu, Sep 25, 2014 at 11:48:29PM +0000, Sean Kelly via Digitalmars-d wrote:

 On Thursday, 25 September 2014 at 03:29:03 UTC, Zhao Puming wrote:

Great work Etienne!

will libasync make it into phobos?


 
 I certainly hope so.  We need async functionality if we're to ever
 have a decent socket package in Phobos.


Adam has also written an event loop (arsd.eventloop), which also has
timers and a signals-and-slots subsystem:

	https://github.com/adamdruppe/arsd/blob/master/eventloop.d

I'm not sure if it supports multithreading, but its event registration
API is very cool. I think we should pick the best of both projects for
inclusion in Phobos.


T

-- 
Let X be the set not defined by this sentence...
Sep 25 2014
prev sibling next sibling parent reply Andrej Mitrovic via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 9/24/14, Etienne via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 It's finally here: https://github.com/etcimon/libasync


Small nitpick with the spelling: asynchroneous => asynchronous. I
personally don't care about spelling but many people tend to
(unfortunately) look the other way when they find typos.

Btw, have you had a look at http://wiki.dlang.org/Event_system ? I'm
just wondering how much of that idea page / proposal is covered.

Anyway, this is a solid initiative. I look forward to seeing it in Phobos!
Sep 26 2014
parent Etienne <etcimon gmail.com> writes:
On 2014-09-26 5:29 AM, Andrej Mitrovic via Digitalmars-d wrote:

 Small nitpick with the spelling: asynchroneous => asynchronous. I
 personally don't care about spelling but many people tend to
 (unfortunately) look the other way when they find typos.


Hm, my french messed with my mind on that one :-P we say asynchrone or 
asynchroné


 Btw, have you had a look at http://wiki.dlang.org/Event_system ? I'm
 just wondering how much of that idea page / proposal is covered.

 Anyway, this is a solid initiative. I look forward to seeing it in Phobos!


I originally developed it with vibe.d's usage in mind which implicitely 
forced me to cover most of this list, but currently it's lacking 
AsyncObject which would allow to monitor file descriptors and cover a 
good chunk of this list. I'm sure it will be a very useful resource for 
upcoming decisions.
Sep 26 2014
prev sibling next sibling parent reply "Adam Wilson" <flyboynw gmail.com> writes:
On Wed, 24 Sep 2014 06:13:31 -0700, Etienne <etcimon gmail.com> wrote:


 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular libraries  
 Qt and Nodejs.. we now have a natively compiling async library entirely  
 written in D.

 This event library was tested on Win32, Linux x64, Mac OS x64, with DMD  
 2.066, offers the more low-level async objects: timers, file i/o, dns  
 resolver, tcp, udp, listeners, signals (cross-thread), notifications  
 (same thread), and more recently (and with great efforts for  
 implementing with OS X / BSD) a directory watcher.

 e.g. You can run a timer with:

 	import std.datetime; import std.stdio; import libasync.all;
 	EventLoop evl = new EventLoop;
 	auto timer = new AsyncTimer(evl);
 	timer.duration(2.seconds).periodic().run({ writeln("Another 2 seconds  
 have passed"); });
 	while(evl.loop()) continue;

 The tests may be most revealing:  
 https://github.com/etcimon/libasync/blob/master/source/libasync/test.d

 A (lightly tested) vibe.d driver using all those async objects is also  
 available and currently ongoing a pull request:

 https://github.com/etcimon/vibe.d/tree/native-events

 The incentive was to make vibe.d compile in completely native D, I'm now  
 moving onto a botan C++ => D wrapper for it, I plan on moving objects to  
 D over the years until the TLS library can be completely native. I thank  
 Walter for the efforts on extern(C++)

 Finally, I release this on the basis of an MIT license, looking forward  
 to seeing our community flourishing with yet more native libraries. Code  
 on


You mentioned Botan. I already have a C++ => D Wrapper project going over  
here: https://github.com/ellipticbit/titanium

I am working out a bug where the memory corrupts itself when passing data  
back to D but it works and most of the leg-work is done. And I am  
definitely open to pull-requests.

-- 
Adam Wilson
GitHub/IRC: LightBender
Aurora Project Coordinator
Sep 26 2014
parent reply Etienne <etcimon gmail.com> writes:
On 2014-09-27 12:25 AM, Adam Wilson wrote:

 You mentioned Botan. I already have a C++ => D Wrapper project going
 over here: https://github.com/ellipticbit/titanium

 I am working out a bug where the memory corrupts itself when passing
 data back to D but it works and most of the leg-work is done. And I am
 definitely open to pull-requests.


Your wrapper will probably have to wrap a new Botan. I decided to 
translate everything to D because I need the complete interface.

https://github.com/etcimon/botan
Sep 27 2014
parent reply "Joakim" <dlang joakim.fea.st> writes:
On Saturday, 27 September 2014 at 17:06:53 UTC, Etienne wrote:

 On 2014-09-27 12:25 AM, Adam Wilson wrote:

 You mentioned Botan. I already have a C++ => D Wrapper project 
 going
 over here: https://github.com/ellipticbit/titanium

 I am working out a bug where the memory corrupts itself when 
 passing
 data back to D but it works and most of the leg-work is done. 
 And I am
 definitely open to pull-requests.


 Your wrapper will probably have to wrap a new Botan. I decided 
 to translate everything to D because I need the complete 
 interface.

 https://github.com/etcimon/botan


How long do you think that's going to take?  What do you plan to 
do about ongoing C++ patches added to the original C++ botan 
version?  Maybe developing something like Daniel Murphy's DDMD 
magicport for botan would save you some time from doing it all 
manually.
Sep 27 2014
parent reply Etienne <etcimon gmail.com> writes:
On 2014-09-27 2:07 PM, Joakim wrote:

 How long do you think that's going to take?  What do you plan to do
 about ongoing C++ patches added to the original C++ botan version?
 Maybe developing something like Daniel Murphy's DDMD magicport for botan
 would save you some time from doing it all manually.


I see it done in a week before the testing, search and replace does the 
trick for most of it, the longest part is merging the .h and .cpp files. 
The patches don't seem very frequent (last update was in april, mostly 
minor), but I can also apply those manually if necessary. However, I 
intend on branching off completely and maintaining it myself with new 
algorithms, a certificate factory and a better BER/DER serialization 
engine (I have an ASN1 library in the works as well).
Sep 27 2014
parent reply Etienne <etcimon gmail.com> writes:
On 2014-09-27 2:13 PM, Etienne wrote:

 engine (I have an ASN1 library in the works as well).


It's nearly finished, it will allow BER/DER serialization to take place 
from UDAs and native types at compile-time:

https://github.com/globecsys/asn1.d
Sep 27 2014
parent reply "Suliman" <evermind live.ru> writes:
On Saturday, 27 September 2014 at 18:21:18 UTC, Etienne wrote:

 On 2014-09-27 2:13 PM, Etienne wrote:

 engine (I have an ASN1 library in the works as well).


 It's nearly finished, it will allow BER/DER serialization to 
 take place from UDAs and native types at compile-time:

 https://github.com/globecsys/asn1.d


/// Usage: run() the object, start watching directories, receive 
an event in your handler,
/// read the changes by draining the buffer.

Could you show how to use it for monitoring FS?

I did
import libasync.watcher;

So now I need simply call run()? But how to pass folder path to 
it?


void main()
{
	bool run()
	{
		return 0;
	}

}


It's seems that I missing something...
Jun 28 2015
parent reply "Etienne" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 15:09:25 UTC, Suliman wrote:

 On Saturday, 27 September 2014 at 18:21:18 UTC, Etienne wrote:

 On 2014-09-27 2:13 PM, Etienne wrote:

 engine (I have an ASN1 library in the works as well).


 It's nearly finished, it will allow BER/DER serialization to 
 take place from UDAs and native types at compile-time:

 https://github.com/globecsys/asn1.d


 /// Usage: run() the object, start watching directories, 
 receive an event in your handler,
 /// read the changes by draining the buffer.

 Could you show how to use it for monitoring FS?

 I did
 import libasync.watcher;

 So now I need simply call run()? But how to pass folder path to 
 it?


 void main()
 {
 	bool run()
 	{
 		return 0;
 	}

 }


 It's seems that I missing something...


You can see an example in libasync.tests
Jun 28 2015
parent reply "Suliman" <evermind live.ru> writes:
 You can see an example in libasync.tests


thanks! I have got few questions
g_cbCheck = new shared bool[19];
what does it do?

AsyncDirectoryWatcher g_watcher = new 
AsyncDirectoryWatcher(getThreadEventLoop());
Am I right understand that it's run new eventloop inside 
AsyncDirectoryWatcher function?
Jun 28 2015
next sibling parent reply "Suliman" <evermind live.ru> writes:
It's particularity of unit-test blocks, or why function are 
specified without return type like:

g_evl = getThreadEventLoop();


Also next code that I take from example after run absolutely do 
not do nothing:

void main()
{

	auto g_evl = getThreadEventLoop();
	auto g_cbCheck = new shared bool[19];

	auto g_watcher = new AsyncDirectoryWatcher(g_evl);
	g_watcher.run({
		DWChangeInfo[1] change;
		DWChangeInfo[] changeRef = change.ptr[0..1];
		while(g_watcher.readChanges(changeRef)){
			g_cbCheck[18] = true;
			writeln(change);
		}
	});
	g_watcher.watchDir(".");
	AsyncTimer tm = new AsyncTimer(g_evl);
	tm.duration(1.seconds).run({
		writeln("Creating directory ./hey");
		mkdir("./hey");
		assert(g_watcher.watchDir("./hey/"));
		tm.duration(1.seconds).run({
			writeln("Writing to ./hey/tmp.tmp for the first time");
			std.file.write("./hey/tmp.tmp", "some string");
			tm.duration(100.msecs).run({
				writeln("Removing ./hey/tmp.tmp");
				remove("./hey/tmp.tmp");
			});
		});
	});


}
Jun 28 2015
parent reply "Etienne Cimon" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 16:57:44 UTC, Suliman wrote:

 Also next code that I take from example after run absolutely do 
 not do nothing:


This code will register a directory watcher in the working 
directory, on the thread's event loop, and then use timers to 
create file/folder activity to trigger this directory watcher.

The only thing needed to make this work is r/w access to the 
working directory and a running event loop.

So, adding the line:

g_evl.run(10.seconds);

If you don't have an event loop running, your application is 
basically not going to receive callback events.
Jun 28 2015
parent reply "Etienne Cimon" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 17:10:16 UTC, Etienne Cimon wrote:

 g_evl.run(10.seconds);


Hmm, sorry that would be g_evl.loop(10.seconds) or 
getThreadEventLoop().loop(10.seconds). I use the vibe.d driver 
most often.
Jun 28 2015
parent reply "Suliman" <evermind live.ru> writes:
 Hmm, sorry that would be g_evl.loop(10.seconds) or 
 getThreadEventLoop().loop(10.seconds). I use the vibe.d driver 
 most often.


I changed paths to hardcoded to prevent issue with them, but next 
code after run only create "hey" folder and nothing more. No 
files in it:

{

	void dirWatcher()
	{
		auto g_watcher = new 
AsyncDirectoryWatcher(getThreadEventLoop());
		g_watcher.run({
			DWChangeInfo[1] change;
			DWChangeInfo[] changeRef = change.ptr[0..1];
			while(g_watcher.readChanges(changeRef)){
				writeln(change);
			}
		});
		g_watcher.watchDir(".");
		AsyncTimer tm = new AsyncTimer(getThreadEventLoop());
		tm.duration(1.seconds).run({
			writeln("Creating directory ./hey");
			mkdir(`D:\code\DirWathcher\hey`);
			assert(g_watcher.watchDir(`D:\code\DirWathcher\hey`));
			tm.duration(4.seconds).run({
				writeln("Writing to ./hey/tmp.tmp for the first time");
				std.file.write(`D:\code\DirWathcher\hey\tmp.tmp`, "some 
string");
				tm.duration(100.msecs).run({
					writeln("Removing ./hey/tmp.tmp");
					remove(`D:\code\DirWathcher\hey\tmp.tmp`);
				});
			});
		});
		getThreadEventLoop().loop(10.seconds);
	}

dirWatcher();
destroyAsyncThreads();

}


--------------
And how to detect if some there was some manipulation inside 
monitoring folder? How I can recive name of file that was added 
to folder?
Jun 28 2015
parent reply "Etienne Cimon" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 18:33:28 UTC, Suliman wrote:

 Hmm, sorry that would be g_evl.loop(10.seconds) or 
 getThreadEventLoop().loop(10.seconds). I use the vibe.d driver 
 most often.


 I changed paths to hardcoded to prevent issue with them, but 
 next code after run only create "hey" folder and nothing more. 
 No files in it:


Try putting the timers outside the callbacks in the meantime. 
I'll test it tomorrow when I'm at the office =)


 --------------
 And how to detect if some there was some manipulation inside 
 monitoring folder? How I can recive name of file that was added 
 to folder?


There's a path in the DWFileChange change structure, you can 
access it with change.path. You'll have to put it in a path 
parser to access the filename, such as std.path
Jun 28 2015
next sibling parent "Suliman" <evermind live.ru> writes:
 Try putting the timers outside the callbacks in the meantime. 
 I'll test it tomorrow when I'm at the office =)


Ok. Thanks. I did:
dirWatcher();
getThreadEventLoop().loop(10.seconds);
destroyAsyncThreads();
same result.
Jun 28 2015
prev sibling next sibling parent reply "Suliman" <evermind live.ru> writes:
void main()
{

	void dirWatcher()
	{
		auto g_watcher = new 
AsyncDirectoryWatcher(getThreadEventLoop());
		g_watcher.run({
			DWChangeInfo[1] change;
			DWChangeInfo[] changeRef = change.ptr[0..1];
			while(g_watcher.readChanges(changeRef)){
				writeln(change);
			}
		});
		g_watcher.watchDir(".");
		getThreadEventLoop();
		
	}

dirWatcher();

//destroyAsyncThreads();

}
Is this code enough to monitoring folder?

If I run it it's terminating, it's seems that eventloop not 
starting or I placed it in wrong place?
Jun 28 2015
parent reply "Etienne Cimon" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 19:34:46 UTC, Suliman wrote:

 void main()
 {

 	void dirWatcher()
 	{
 		auto g_watcher = new 
 AsyncDirectoryWatcher(getThreadEventLoop());
 		g_watcher.run({
 			DWChangeInfo[1] change;
 			DWChangeInfo[] changeRef = change.ptr[0..1];
 			while(g_watcher.readChanges(changeRef)){
 				writeln(change);
 			}
 		});
 		g_watcher.watchDir(".");
 		getThreadEventLoop();
 		
 	}

 dirWatcher();

 //destroyAsyncThreads();

 }
 Is this code enough to monitoring folder?

 If I run it it's terminating, it's seems that eventloop not 
 starting or I placed it in wrong place?


You should do getThreadEventLoop().loop(); and it will monitor 
the changes forever. Try making changes in the filesystem 
manually :)
Jun 28 2015
parent reply "Suliman" <evermind live.ru> writes:
Ehm, there is issue on Windows or I am again doing something 
wrong?

void main()
{

  	void dirWatcher()
  	{
  		auto g_watcher = new 
AsyncDirectoryWatcher(getThreadEventLoop());
  		g_watcher.run({
  			DWChangeInfo[1] change;
  			DWChangeInfo[] changeRef = change.ptr[0..1];
  			while(g_watcher.readChanges(changeRef)){
  				writeln(change);
  			}
  		});
  		g_watcher.watchDir(".");
  		getThreadEventLoop().loop();
  	}

  dirWatcher();

  destroyAsyncThreads();

}


Without destroyAsyncThreads(); it's fail with exception.
Jun 29 2015
parent reply "Etienne" <etcimon gmail.com> writes:
On Monday, 29 June 2015 at 20:34:11 UTC, Suliman wrote:

 Ehm, there is issue on Windows or I am again doing something 
 wrong?

 void main()
 {

  	void dirWatcher()
  	{
  		auto g_watcher = new 
 AsyncDirectoryWatcher(getThreadEventLoop());
  		g_watcher.run({
  			DWChangeInfo[1] change;
  			DWChangeInfo[] changeRef = change.ptr[0..1];
  			while(g_watcher.readChanges(changeRef)){
  				writeln(change);
  			}
  		});
  		g_watcher.watchDir(".");
  		getThreadEventLoop().loop();
  	}

  dirWatcher();

  destroyAsyncThreads();

 }


 Without destroyAsyncThreads(); it's fail with exception.


Yes that line is required
Jun 29 2015
parent reply "suliman" <Evermind live.ru> writes:
Ok, but code still do exit after it's run. Look like loop is not 
started
Jun 29 2015
parent "Etienne" <etcimon gmail.com> writes:
On Tuesday, 30 June 2015 at 04:38:31 UTC, suliman wrote:

 Ok, but code still do exit after it's run. Look like loop is 
 not started


Yes, you need to watch for the return type which will be false in 
case of error

while(getThreadEventLoop().loop(10.seconds))
continue;

writeln(getThreadEventLoop().error);
Jun 30 2015
prev sibling parent "Suliman" <evermind live.ru> writes:
 There's a path in the DWFileChange change structure, you can 
 access it with change.path. You'll have to put it in a path 
 parser to access the filename, such as std.path


Could you give an example of usage? And where DWFileChange 
structure is located? I do not see it in watcher.d
Jul 11 2015
prev sibling parent "Etienne" <etcimon gmail.com> writes:
On Sunday, 28 June 2015 at 16:32:24 UTC, Suliman wrote:

 You can see an example in libasync.tests


 thanks! I have got few questions
 g_cbCheck = new shared bool[19];
 what does it do?

 AsyncDirectoryWatcher g_watcher = new 
 AsyncDirectoryWatcher(getThreadEventLoop());
 Am I right understand that it's run new eventloop inside 
 AsyncDirectoryWatcher function?


The g_cbCheck is specific to this unit test, everytime a callback 
succeeds it will set its index to true in the array.

The event loop must be passed to the async object because the 
watcher will register itself to it when it is run.

When you've registered your callback in the watcher and run it 
through the event loop, you have to run the event loop and events 
will be pushed to your callback
Jun 28 2015
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
24-Sep-2014 17:13, Etienne пишет:

 It's finally here: https://github.com/etcimon/libasync

 We all know how event loops are the foundation of more popular libraries
 Qt and Nodejs.. we now have a natively compiling async library entirely
 written in D.

 This event library was tested on Win32, Linux x64, Mac OS x64, with DMD
 2.066, offers the more low-level async objects: timers, file i/o, dns
 resolver, tcp, udp, listeners, signals (cross-thread), notifications
 (same thread), and more recently (and with great efforts for
 implementing with OS X / BSD) a directory watcher.


Awesome! Bookmarked for future ;)

-- 
Dmitry Olshansky
Sep 27 2014