www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - yet another event loop

reply Eugene Wissner <belka caraus.de> writes:
https://github.com/caraus-ecms/tanya

Ok there are not so many event loops in D and here an another one 
and its name is "tanya".
I want it to become not an event loop only but a general purpose 
library that has an event loop.

What once started as a libev rewrite, hasn't much common with 
libev now except some general concepts like watchers.


Regarding libev:
1) tanya is very, very basic and it hasn't a lot of important 
features yet like signals, UDP, threads and so on. I had to begin 
somewhere and stripped out everything that isn't relevant for a 
basic event loop. Features will be added with the time.

2) Only epoll is currently supported. But I tried to create an 
API that can be easily extended, so you have to extend one class 
and implement a few methods to add other backends.

3) In another thread chmike (many thanks again!) pointed me to 
Windows IOCP. I'm not completely sure I understand how the 
completion ports work, but I implemented the loop in the way that 
you haven't to care about file descriptors and sockets but get 
notified if the data are really available. And you write aswell 
not to a socket but into a buffer, and the event loop takes care 
of passing it then to the socket. I hope it can make the work 
with the loop more pleasant and can make it possible to create a 
performant Windows implementation.


Other points:
1) The library is 100%  nogc. I know there were some discussions 
that this  nogc is pure marketing thing, but I find it helpful 
that the compiler can say if you allocate somewhere in a language 
where GC allocations can happen behind the scenes.

2) The loop throws a few exceptions that should be freed, but I'm 
thinking to switch to some data type "either exception or return 
value" and make the loop nothrow. It has nothing to do with 
 nogc. It is just kind of not very cool if an exception can kill 
the event loop if something goes wrong.

3) The library isn't thread safe. I will work on it later.

4) libev wasn't the only source of inspiration. tanya is a mix of 
libev and asyncio and asynchronous. I took over the concept of 
protocols and transports from asyncio/asynchronous since I 
believe they make the writing of applications really pleasant. 
The difference is that they aren't a kind of "wrapper" around the 
actual event loop, but are first-class citizens. It could make it 
difficult to write such wrappers like that ones that exist for 
libasync, but on the other side it kills some unneeded 
abstractions and makes the code structure simplier, that could 
also give some additional performance.

5) I tried to write unittests and short descriptions everywhere, 
so there is some documentation and examples. For an usage example 
skip the crap I'm writing here and look at the end of this 
message.


There are already some "extras":
tanya.memory: has a simple allocator (Ullocator) that uses 
mmap/munmap (tested on Linux, will theoretically work on other 
platforms aswell). "allocator" package has some functions like 
"finalize" that can be used in  nogc code instead of dispose or 
"resizeArray" that is similar to shrinkArray/expandArray from 
std.experimental.allocator, but doesn't take a delta as argument 
but just the length, that the array should have. The allocator 
was the most difficult part of the library for me, but very 
interesting. I had to rewrite it 3 times till I got something 
working. I just advice everyone to write their own malloc/free 
implementaion, it is a frustrating, but awsome experience!

tanya.container: Queue, Singly-linked list and In-/Output Buffer 
(useful in C-style functions that take a void pointer and the 
length as argument and return bytes read/written). I wrote them 
for the event loop, not sure they are good as general-purpose 
containers, but I would be anyway interested to make them 
suitable for other use-cases. They are also differently concepted 
than phobos containers. Phobos containers as far as I've seen are 
containers that implement ranges functionality in 
substructs/subclasses. tanya's containers are a mix of containers 
and ranges.

tanya.math: has "pow" function that calculates x**y mod z. The 
algorithm is similar to the one used by phobos. The return type 
and arguments are currently ulong but it will change, I will need 
larger numbers probably.

tanya.random: has an "Entropy" class that can generate 64-byte 
blocks of random data (uses getrandom syscall). The generic logic 
is stolen from mbedtls.

tanya.crypto.padding: implements some algorithms to pad 
128/192/258-byte blocks of data. But you cannot remove the 
padding :) Sorry, it will be added soon. Just started.


I made some tests with an echo-client written in Go (just found 
one benchmarking one-page Go echo-client in the internet). Here 
is an usage example, just to give some feeling how the library 
works (Examples and description will be added to the repository 
soon):

import tanya.memory;
import tanya.event.loop;
import tanya.event.protocol;
import tanya.event.transport;
import tanya.event.watcher;
import core.stdc.stdio;
import std.exception;
import core.stdc.string;
import core.sys.posix.netinet.in_;
import core.sys.posix.fcntl;
import core.sys.posix.unistd : close;

class EchoProtocol : TransmissionControlProtocol
{
 nogc:
	private DuplexTransport transport;

	void received(ubyte[] data)
	{
		transport.write(data);
		printf("%.*s", data.length, data.ptr);
	}

	void connected(DuplexTransport transport)
	{
		this.transport = transport;
		printf("Got connection.\n");
	}

	void disconnected()
	{
         printf("Disconnected.\n");
	}
}

void main()
{
	sockaddr_in addr;
	int s = socket(AF_INET, SOCK_STREAM, 0);
	auto loop = getDefaultLoop();

	// Echo server
	printf("Listening on port 8192\n");

	addr.sin_family = AF_INET;
	addr.sin_port = htons(cast(ushort)8192);
	addr.sin_addr.s_addr = INADDR_ANY;

	if (bind(s, cast(sockaddr *)&addr, addr.sizeof) != 0)
	{
		throw make!Exception(defaultAllocator, "bind");
	}

	fcntl(s, F_SETFL, fcntl(s, F_GETFL, 0) | O_NONBLOCK);
	listen(s, 5);

	auto io =  make!ConnectionWatcher(defaultAllocator,
	    () => cast(Protocol) make!EchoProtocol(defaultAllocator),
	    s);

	loop.start(io);

	loop.run();

	shutdown(s, SHUT_RDWR);
	close(s);
}


Sorry for that sockaddr_in stuff, I want to add a Socket class in 
the next commits, that would eliminate the need of this 
impossible boilerplate.

So far.. I would say it is the first test release and I'm 
beginning with testing and continue to write. Enjoy, I hope it 
doesn't leak too much memory...
Aug 24 2016
next sibling parent reply mogu <mogucpp 163.com> writes:
On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
wrote:

 https://github.com/caraus-ecms/tanya

 Ok there are not so many event loops in D and here an another 
 one and its name is "tanya".
 ...


Nice works, thanks.
Aug 24 2016
parent Eugene Wissner <belka caraus.de> writes:
On Thursday, 25 August 2016 at 00:24:59 UTC, mogu wrote:

 On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
 wrote:

 https://github.com/caraus-ecms/tanya

 Ok there are not so many event loops in D and here an another 
 one and its name is "tanya".
 ...


 Nice works, thanks.


Hey,

Thanks for your feedback. I'm very interested if someone would 
like the idea and maybe participate.

It would be also nice to have some benchmarking but I couldn't 
find an appropriate tool for this. I found some small programs 
but everything in languages I don't know, so I couldn't find out 
if my implementation doesn't work in some cases or the 
benchmarking tool. Writing an own bencmarking suite would take 
some time that I would better spend writing the library.
Maybe I should have first a simple http server. There a lot of 
matured tools for testing http.
Aug 25 2016
prev sibling next sibling parent reply Bill Hicks <billhicks gmail.com> writes:
On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
wrote:

 https://github.com/caraus-ecms/tanya

 Ok there are not so many event loops in D and here an another 
 one and its name is "tanya".
 


Could you change the name to something more recognizable to help 
with D's popularity?  Something like 'kardashian' might be 
better.  Nobody really knows 'tanya', do you know what I mean?  
Maybe even 'madonna', so people will know that D is just as 
talented as others languages.

Marketing is going to be key for D, so let's do our best.
Aug 25 2016
parent reply Eugene Wissner <belka caraus.de> writes:
On Friday, 26 August 2016 at 02:22:54 UTC, Bill Hicks wrote:

 On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
 wrote:

 https://github.com/caraus-ecms/tanya

 Ok there are not so many event loops in D and here an another 
 one and its name is "tanya".
 


 Could you change the name to something more recognizable to 
 help with D's popularity?  Something like 'kardashian' might be 
 better.  Nobody really knows 'tanya', do you know what I mean?  
 Maybe even 'madonna', so people will know that D is just as 
 talented as others languages.

 Marketing is going to be key for D, so let's do our best.


I think I disagree. For example I didn't know who is kardashian 
and the name is pretty difficult. And I think neither kardashian 
nor madonna are worth to name something after them. I also doubt 
that naming after famous people gave more popularity to kafka, 
picasa, ada, pascal and so on
Aug 26 2016
parent reply bachmeier <no spam.net> writes:
On Friday, 26 August 2016 at 10:38:59 UTC, Eugene Wissner wrote:

 I think I disagree. For example I didn't know who is kardashian 
 and the name is pretty difficult. And I think neither 
 kardashian nor madonna are worth to name something after them. 
 I also doubt that naming after famous people gave more 
 popularity to kafka, picasa, ada, pascal and so on


The person you responded to is a troll that has been cluttering 
the forum. No need to even read what they are posting.
Aug 26 2016
parent reply Eugene Wissner <belka caraus.de> writes:
On Friday, 26 August 2016 at 10:54:17 UTC, bachmeier wrote:

 The person you responded to is a troll that has been cluttering 
 the forum. No need to even read what they are posting.


Didn't know it, thanks
Aug 26 2016
parent reply Bill Hicks <billhicks gmail.com> writes:
On Friday, 26 August 2016 at 14:19:55 UTC, Eugene Wissner wrote:

 On Friday, 26 August 2016 at 10:54:17 UTC, bachmeier wrote:

 The person you responded to is a troll that has been 
 cluttering the forum. No need to even read what they are 
 posting.


 Didn't know it, thanks


Don't listen to him, he's trying to sharpen your mind.

If you don't want to use a famous name, then at least use a 
non-female name.  The sad reality is that most guys in the 
industry perceive females as inferior, so using a name like 
'tanya' will have a poor effect on your project.  You want to 
reach as many minds as possible, regardless of how good or bad 
your library is, so pick a manly name.
Aug 26 2016
parent Johnjo Willoughby <who me.com> writes:
On Saturday, 27 August 2016 at 05:20:40 UTC, Bill Hicks wrote:

 On Friday, 26 August 2016 at 14:19:55 UTC, Eugene Wissner wrote:

 On Friday, 26 August 2016 at 10:54:17 UTC, bachmeier wrote:

 The person you responded to is a troll that has been 
 cluttering the forum. No need to even read what they are 
 posting.


 Didn't know it, thanks


 Don't listen to him, he's trying to sharpen your mind.

 If you don't want to use a famous name, then at least use a 
 non-female name.  The sad reality is that most guys in the 
 industry perceive females as inferior, so using a name like 
 'tanya' will have a poor effect on your project.  You want to 
 reach as many minds as possible, regardless of how good or bad 
 your library is, so pick a manly name.


The name should reflect what you're doing, the service, the 
quality. I mean sure using the Bill Hicks moniker will get you 
attention at the start, but people will quickly realise it's a 
scam, you're advertising 28 days matured sirloin but giving them 
shoe leather soaked in piss.

F - Must try harder.
Aug 29 2016
prev sibling next sibling parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-08-24 18:03:39 +0000, Eugene Wissner said:


 I want it to become not an event loop only but a general purpose 
 library that has an event loop.


Hi, well, I would re-think this since the danger is, that everything is 
so connected that I can use either all or nothing at all.

I'm searching for a configurable (network, timer, GUI) stand-alone 
event-loop library, that abstracts away platform differences for years. 
Those that I found are so tighly deep coupled with the rest, that you 
can't get them out.

IMO general purpose libraries building their own context and world-view 
are waste of time...

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Aug 26 2016
parent reply Eugene Wissner <belka caraus.de> writes:
On Friday, 26 August 2016 at 10:03:22 UTC, Robert M. Münch wrote:

 On 2016-08-24 18:03:39 +0000, Eugene Wissner said:


 I want it to become not an event loop only but a general 
 purpose library that has an event loop.


 Hi, well, I would re-think this since the danger is, that 
 everything is so connected that I can use either all or nothing 
 at all.

 I'm searching for a configurable (network, timer, GUI) 
 stand-alone event-loop library, that abstracts away platform 
 differences for years. Those that I found are so tighly deep 
 coupled with the rest, that you can't get them out.

 IMO general purpose libraries building their own context and 
 world-view are waste of time...


Do you mean that the library can have different modules but they 
should be independent of each other as much as possible (like 
phobos) or that every part that can be separated belongs to its 
own repository?
I absolutely agree with the first but not sure about the second 
point.
Aug 26 2016
parent =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2016-08-26 19:37:19 +0000, Eugene Wissner said:


 Do you mean that the library can have different modules but they should 
 be independent of each other as much as possible (like phobos) or that 
 every part that can be separated belongs to its own repository?


I don't see both related. The 1st property is about library design, the 
2nd about code management.

If I can somehow only use the event part without having to get all 
other stuff in that's perfect. Putting everything into one repository 
is OK for me as well, makes handling the lib simpler.

-- 
Robert M. Münch
http://www.saphirion.com
smarter | better | faster
Aug 26 2016
prev sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
wrote:

 https://github.com/caraus-ecms/tanya


Please make documentation easily available for your library. I 
wish to use event loops in D, but I have no desire to wade though 
someone else's code in order to figure out how use the library.
Aug 26 2016
parent Eugene Wissner <belka caraus.de> writes:
On Friday, 26 August 2016 at 15:02:42 UTC, Jack Stouffer wrote:

 On Wednesday, 24 August 2016 at 18:03:39 UTC, Eugene Wissner 
 wrote:

 https://github.com/caraus-ecms/tanya


 Please make documentation easily available for your library. I 
 wish to use event loops in D, but I have no desire to wade 
 though someone else's code in order to figure out how use the 
 library.


Sure. I will try then that it appears as soon as possible either 
as a few web pages or github-wiki. Thanks for this note
Aug 26 2016