www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Low-overhead components

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
Not really an article or anything - this was planned as just a 
post to this newsgroup, but I decided to put it somewhere 
suitable for larger blocks of text with formatting:

http://blog.thecybershadow.net/2013/07/28/low-overhead-components/
Jul 27 2013
next sibling parent reply "Kagamin" <spam here.lot> writes:
For a hashtable to know about its container is probably more 
wasteful than having a pointer to the allocator. If you don't 
like deep nesting, write a shortcut function, which will do the 
entire job for you similar to the read function:
Jul 28 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 July 2013 at 12:31:46 UTC, Kagamin wrote:

 For a hashtable to know about its container is probably more 
 wasteful than having a pointer to the allocator.


Why? The difference is one indirection. Are you referring to the 
impact of template bloat and code cache misses?


 If you don't like deep nesting, write a shortcut function, 
 which will do the entire job for you similar to the read 
 function:



Yes, but you still need to write it. With mixins, each layer is 
declared separately, and you can access it directly.
Jul 28 2013
parent reply "Kagamin" <spam here.lot> writes:
On Sunday, 28 July 2013 at 21:03:43 UTC, Vladimir Panteleev wrote:

 Why? The difference is one indirection. Are you referring to 
 the impact of template bloat and code cache misses?


bloat


 Yes, but you still need to write it.


That's what we do.
You need to write it anyway to provide simple interface to load 
xml files, no matter whether you use mixins or nesting.


 With mixins, each layer is declared separately, and you can 
 access it directly.


Don't you fear namespace pollution? You probably don't want to 
call arbitrary methods on xmlParser and later try to figure out, 
what did you call exactly.
Jul 30 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 30 July 2013 at 15:57:50 UTC, Kagamin wrote:

 On Sunday, 28 July 2013 at 21:03:43 UTC, Vladimir Panteleev 
 wrote:

 Why? The difference is one indirection. Are you referring to 
 the impact of template bloat and code cache misses?


 bloat


In most cases, only one template instance will be "hot" at one 
time, so I don't think it would be a problem in general.


 With mixins, each layer is declared separately, and you can 
 access it directly.


 Don't you fear namespace pollution? You probably don't want to 
 call arbitrary methods on xmlParser and later try to figure 
 out, what did you call exactly.


If xmlParser would be a mixin, referring to it directly could 
only refer to the symbols declared in the mixin itself.
Jul 30 2013
parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 30 July 2013 at 16:48:42 UTC, Vladimir Panteleev 
wrote:

 In most cases, only one template instance will be "hot" at one 
 time, so I don't think it would be a problem in general.


You need to load templates from disk. I'm thinking about Adobe 
products, though I don't know why they're so huge and why they 
start so slow.


 If xmlParser would be a mixin, referring to it directly could 
 only refer to the symbols declared in the mixin itself.


Do you mean referring to the mixin doesn't include mixins nested 
into it?
Jul 31 2013
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 31 July 2013 at 08:04:35 UTC, Kagamin wrote:

 On Tuesday, 30 July 2013 at 16:48:42 UTC, Vladimir Panteleev 
 wrote:

 In most cases, only one template instance will be "hot" at one 
 time, so I don't think it would be a problem in general.


 You need to load templates from disk.


Loaded from a memory-mapped file, to be precise. Linkers can 
optimize the layout of your program to put rarely-used templates 
somewhere where they won't be loaded from the disk until they're 
needed. (Do you think a 500MB executable installer gets loaded 
into RAM entirely before it can run?) But it's not like there 
will be a thousand XML parser instantiations in the same program.


 I'm thinking about Adobe products, though I don't know why 
 they're so huge and why they start so slow.


I doubt it has anything to do with template bloat.


 Do you mean referring to the mixin doesn't include mixins 
 nested into it?


Why would you nest mixins? That's the worst of both worlds.
Jul 31 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/27/13 11:20 PM, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a post to
 this newsgroup, but I decided to put it somewhere suitable for larger
 blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


Nice! All - please don't reddit until Monday morning.

Thanks,

Andrei
Jul 28 2013
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
BTW, there's no fast way to boundcheck two-ptr range. It should 
work similar to opSlice:

T opIndex(size_t index)
{
   static if (CHECKED)
     assert(index < end-ptr);
   return *(ptr + index);
}
Jul 28 2013
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Sunday, 28 July 2013 at 18:51:15 UTC, Kagamin wrote:

 BTW, there's no fast way to boundcheck two-ptr range. It should 
 work similar to opSlice:

 T opIndex(size_t index)
 {
   static if (CHECKED)
     assert(index < end-ptr);
   return *(ptr + index);
 }


That's a bug, thanks. But non-release-build performance is not a 
concern for FastArrayRange anyway.
Jul 28 2013
prev sibling next sibling parent =?UTF-8?B?Ikx1w61z?= Marques" <luismarques gmail.com> writes:
On Sunday, 28 July 2013 at 06:20:29 UTC, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a 
 post to this newsgroup, but I decided to put it somewhere 
 suitable for larger blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


BTW, slightly off-topic, but could you clarify something for me? 
In the phrase "I understand that STL allocators are stateless, 
which is boring ", does the expression "I understand that" mean 
"I think the following is true, but I'm not sure" or "I know the 
following is true, and I acknowledge it, (but...)", or something 
else?
Jul 29 2013
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/27/13 11:20 PM, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a post to
 this newsgroup, but I decided to put it somewhere suitable for larger
 blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


Vote up!

http://www.reddit.com/r/programming/comments/1jap9d/lowoverhead_components/


Andrei
Jul 29 2013
prev sibling next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/27/2013 11:20 PM, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a post to this
 newsgroup, but I decided to put it somewhere suitable for larger blocks of text
 with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


Please include your name as author on this!
Jul 29 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 29 July 2013 at 20:08:19 UTC, Walter Bright wrote:

 On 7/27/2013 11:20 PM, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a 
 post to this
 newsgroup, but I decided to put it somewhere suitable for 
 larger blocks of text
 with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


 Please include your name as author on this!


Fixed, thanks. I wasn't sure if the post was ready for Reddit, as 
it's mainly some thoughts written down for advanced D users.
Jul 29 2013
parent "Jonathan A Dunlap" <jdunlap outlook.com> writes:
 Fixed, thanks. I wasn't sure if the post was ready for Reddit, 
 as it's mainly some thoughts written down for advanced D users.


Thanks for writing this. Aside from the primary purpose of the 
article, I was able to learn along the way more about D's mixins 
because of it.

As a game architect, I love discovering ways to further abstract, 
reuse, improve speed, and aid readability... all stuff that 
mixins can help with. ;)
Jul 29 2013
prev sibling next sibling parent reply Faux Amis <faux amis.com> writes:
On 28-7-2013 08:20, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a post to
 this newsgroup, but I decided to put it somewhere suitable for larger
 blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


I do not understand what this code should help with:

struct LAYER(BASE)
{
     BASE base;
     // ... use base ...
}

Any advice on what I should read to get it?
(no C++ exp)
Jul 30 2013
parent reply "Dicebot" <public dicebot.lv> writes:
On Tuesday, 30 July 2013 at 12:37:26 UTC, Faux Amis wrote:

 I do not understand what this code should help with:

 struct LAYER(BASE)
 {
     BASE base;
     // ... use base ...
 }

 Any advice on what I should read to get it?
 (no C++ exp)


It is meant to do "has-a" (http://en.wikipedia.org/wiki/Has-a) 
inheritance with only exception - it should not be as is not an 
inheritance. :) Common way to achieve code re-usage without 
defining meaningless hierarchies.
Jul 30 2013
parent reply Faux Amis <faux amis.com> writes:
On 30-7-2013 14:48, Dicebot wrote:

 On Tuesday, 30 July 2013 at 12:37:26 UTC, Faux Amis wrote:

 I do not understand what this code should help with:

 struct LAYER(BASE)
 {
     BASE base;
     // ... use base ...
 }

 Any advice on what I should read to get it?
 (no C++ exp)


 It is meant to do "has-a" (http://en.wikipedia.org/wiki/Has-a)
 inheritance with only exception - it should not be as is not an
 inheritance. :) Common way to achieve code re-usage without defining
 meaningless hierarchies.


like this:?

struct LAYER(BASE)
{
	BASE base;
	// ... use base ...
	void func(){};
}

struct Base
{
	alias LAYER!(Base) Layer;
	Layer layer;
	layer.base = this;
	layer.func();
     // ...
}
Jul 30 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 30 July 2013 at 14:33:59 UTC, Faux Amis wrote:

 like this:?

 struct LAYER(BASE)
 {
 	BASE base;
 	// ... use base ...
 	void func(){};
 }

 struct Base
 {
 	alias LAYER!(Base) Layer;
 	Layer layer;
 	layer.base = this;
 	layer.func();
     // ...
 }


Not quite.

Let's say that, for the sake of example, we want to create a 
pipeline for doing simple operations for integers using this 
technique.

First, let's define an interface by convention. Each layer will 
have a method that handles the int value. Let's call that method 
"process". It will take one int argument and return void.

So, one layer to add 1 to the result and pass it to the next 
layer would look like this:

struct Incrementer(BASE)
{
	BASE next;

	void process(int value)
	{
		next.process(value + 1);
	}
}

If we want to multiply numbers by 2, same thing:

struct Doubler(BASE)
{
	BASE next;

	void process(int value)
	{
		next.process(value * 2);
	}
}

At the end of the chain, we'll want to save or print the result. 
This layer does not have a BASE, so it doesn't even need to be a 
template:

struct Printer
{
	void process(int value)
	{
		writeln(value);
	}
}

And here's how to use everything together, if we want to print 
x*2+1:

import std.stdio;

alias Printer Layer0;
alias Incrementer!Layer0 Layer1;
alias Doubler!Layer1 Layer2;

void main()
{
	Layer2 chain;
	chain.process(3); // will print 7
}
Jul 30 2013
parent Faux Amis <faux amis.com> writes:
On 30-7-2013 17:22, Vladimir Panteleev wrote:

 On Tuesday, 30 July 2013 at 14:33:59 UTC, Faux Amis wrote:

 like this:?

 struct LAYER(BASE)
 {
     BASE base;
     // ... use base ...
     void func(){};
 }

 struct Base
 {
     alias LAYER!(Base) Layer;
     Layer layer;
     layer.base = this;
     layer.func();
     // ...
 }


 Not quite.

 Let's say that, for the sake of example, we want to create a pipeline
 for doing simple operations for integers using this technique.

 First, let's define an interface by convention. Each layer will have a
 method that handles the int value. Let's call that method "process". It
 will take one int argument and return void.

 So, one layer to add 1 to the result and pass it to the next layer would
 look like this:

 struct Incrementer(BASE)
 {
      BASE next;

      void process(int value)
      {
          next.process(value + 1);
      }
 }

 If we want to multiply numbers by 2, same thing:

 struct Doubler(BASE)
 {
      BASE next;

      void process(int value)
      {
          next.process(value * 2);
      }
 }

 At the end of the chain, we'll want to save or print the result. This
 layer does not have a BASE, so it doesn't even need to be a template:

 struct Printer
 {
      void process(int value)
      {
          writeln(value);
      }
 }

 And here's how to use everything together, if we want to print x*2+1:

 import std.stdio;

 alias Printer Layer0;
 alias Incrementer!Layer0 Layer1;
 alias Doubler!Layer1 Layer2;

 void main()
 {
      Layer2 chain;
      chain.process(3); // will print 7
 }


Thanks!

For teaching purposes I would suggest to make the flow more obvious by 
writing it like this:

struct Incrementer(BASE)
{
     BASE next;

     void process(int value)
     {
         value += 1;
         next.process(value);
     }
}
Jul 30 2013
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Sunday, 28 July 2013 at 06:20:29 UTC, Vladimir Panteleev wrote:

 Not really an article or anything - this was planned as just a 
 post to this newsgroup, but I decided to put it somewhere 
 suitable for larger blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


On topic of proposals: what to you think about embedding context 
pointer into struct when such alias is used? That will require 
some enforcement from compiler if template instance will be 
actually used as member field in same struct but avoid template 
bloat and fits with existing behavior of nested aggregates.
Jul 30 2013
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 30 July 2013 at 13:28:40 UTC, Dicebot wrote:

 On Sunday, 28 July 2013 at 06:20:29 UTC, Vladimir Panteleev 
 wrote:

 Not really an article or anything - this was planned as just a 
 post to this newsgroup, but I decided to put it somewhere 
 suitable for larger blocks of text with formatting:

 http://blog.thecybershadow.net/2013/07/28/low-overhead-components/


 On topic of proposals: what to you think about embedding 
 context pointer into struct when such alias is used? That will 
 require some enforcement from compiler if template instance 
 will be actually used as member field in same struct but avoid 
 template bloat and fits with existing behavior of nested 
 aggregates.


While not a bad proposal in itself, it's not very interesting in 
terms of performance. If we are to do pointers, then pointers to 
the lower layer aren't much harder to do by hand and save you the 
ECX (this pointer) adjustment when calling the lower layer's 
methods.
Jul 30 2013
parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 30 July 2013 at 14:21:55 UTC, Vladimir Panteleev 
wrote:

 While not a bad proposal in itself, it's not very interesting 
 in terms of performance. If we are to do pointers, then 
 pointers to the lower layer aren't much harder to do by hand 
 and save you the ECX (this pointer) adjustment when calling the 
 lower layer's methods.


My guess (guess!) it will actually have better performance than 
generating new template body for every member field. That 
template bloat can easily get out of control to the point of 
negative performance impact. As I have already mentioned, we 
don't do that for nested aggregates - and this one is not that 
different.

I do favor you first proposal actually but it requires some 
considerations about fitting concept into language design 
(despite being easy to implement).
Jul 30 2013