www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Discussion on Go and D

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
A few more samples of people's perception of the two languages:

http://news.ycombinator.com/item?id=3805302


Andrei
Apr 06 2012
next sibling parent reply deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302


 Andrei


I did some measurement on that point for D lately : 
http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/
Apr 06 2012
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/12 11:20 AM, deadalnix wrote:

 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/


Sweet, will take a look when I'll get better. Found a typo when 
skimming: Mesure.

Andrei
Apr 06 2012
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/6/12 11:20 AM, deadalnix wrote:

 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/


Also "Lets". You may want to just pass the text through a spellchecker, 
the impact of typos on perception is higher than most authors think.

Andrei
Apr 06 2012
prev sibling next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302


 Andrei


 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/


GC issues like this are currently blocking development of Visual D (a 
Win32 project): when just adding spaces to a file, parsing the new file 
every other second often needs 10 or more parsings until an equal amount 
of memory is collected compared to the allocated memory. AFAICT Visual D 
just keeps a reference to the root of the most recent AST of a source file.

What's even worse: when the allocated memory gets larger (say > 200MB), 
the garbage collection itself takes more than a second stalling the 
application, which is a real pain if it happens while you are typing 
source text (it does happen quite often).

I've benchmarked testgc3.d from the dmd test suite a little: it 
allocates about 200 MB of memory, never collects anything, still the 
latest garbage collections takes about a second on an i7 boosted to 3GHz.

Compiling testgc3.d for x64 with GDC, it allocates twice as much memory, 
still the garbage collection takes about a second.

Most of the time is spent in the mark phase of the collection, which 
could probably be optimized to some extend, but that would have to be a 
very good optimization to not running into problems with a little more 
allocated memory.

So my current feeling is that conservative garbage collection with 
stop-the-world mechanics is unusable in an interactive application 
allocating a decent amount of memory (which is rather small in today's 
computers). False pointers add to the problem, but they are not the 
worst issue.

I hope there is something wrong with my reasoning, and that you could 
give me some hints to avoid the memory bloat and the application stalls.

Rainer
Apr 06 2012
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could give me
 some hints to avoid the memory bloat and the application stalls.


A couple of things you can try (they are workarounds, not solutions):

1. Actively delete memory you no longer need, rather than relying on the gc to 
catch it. Yes, this is as unsafe as using C's free().

2. Null out pointers & references when you are done with them. This helps
reduce 
unwanted pinning of unused gc memory.

3. Minimize use of global memory, as that is a major source of source of roots.
Apr 06 2012
next sibling parent deadalnix <deadalnix gmail.com> writes:
Le 06/04/2012 20:01, Walter Bright a écrit :

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().

 2. Null out pointers & references when you are done with them. This
 helps reduce unwanted pinning of unused gc memory.

 3. Minimize use of global memory, as that is a major source of source of
 roots.


Additionally, recycle objects (this is a common practice in java when 
performance matters).
Apr 06 2012
prev sibling next sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().


Actually, having to deal with lifetime issues myself takes away the 
biggest plus of the GC, so I am a bit reluctant to do this.


 2. Null out pointers & references when you are done with them. This
 helps reduce unwanted pinning of unused gc memory.

 3. Minimize use of global memory, as that is a major source of source of
 roots.


I don't think there are many places in the code where these hints might 
apply. Are there known ways of hunting down false references?

Still, my main concern are the slow collections that stall the 
application when a decent amount of memory is used. Removing false 
pointers won't change that, just make it happen a little later.
Apr 06 2012
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 7 April 2012 01:08, Rainer Schuetze <r.sagitario gmx.de> wrote:


 On 4/6/2012 8:01 PM, Walter Bright wrote:


 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:


 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.




 2. Null out pointers & references when you are done with them. This
 helps reduce unwanted pinning of unused gc memory.

 3. Minimize use of global memory, as that is a major source of source of
 roots.


 I don't think there are many places in the code where these hints might
 apply. Are there known ways of hunting down false references?

 Still, my main concern are the slow collections that stall the application
 when a decent amount of memory is used. Removing false pointers won't
 change that, just make it happen a little later.


An obvious best-practise is to allocate in fewer-larger blocks. Ie, more
compounds and aggregates where possible.
I also expect you are doing a lot of string processing. Using D strings
directly? I presume you have a stack-string class? Put as many working
strings on the stack as possible...
Apr 06 2012
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/7/2012 12:44 AM, Manu wrote:

 On 7 April 2012 01:08, Rainer Schuetze <r.sagitario gmx.de
 <mailto:r.sagitario gmx.de>> wrote:

     I don't think there are many places in the code where these hints
     might apply. Are there known ways of hunting down false references?

     Still, my main concern are the slow collections that stall the
     application when a decent amount of memory is used. Removing false
     pointers won't change that, just make it happen a little later.


 An obvious best-practise is to allocate in fewer-larger blocks. Ie, more
 compounds and aggregates where possible.
 I also expect you are doing a lot of string processing. Using D strings
 directly? I presume you have a stack-string class? Put as many working
 strings on the stack as possible...


There isn't a lot of string processing involved: tokens take a slice on 
the original text, and nodes of the AST seldomly save more than the 
identifier which just the same slice. So the full text always remains in 
memory, but this is only small part of the actual footprint, the AST is 
a lot bigger.

The nodes have child and parent references, so you keep the whole AST 
once there is a false pointer to any node. I could try breaking up this 
dependencies when I think the AST is no longer used, but that brings me 
back to manual memory management and thread synchronization (parsing 
uses std.parallelism).
Apr 07 2012
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 07.04.2012 2:08, Rainer Schuetze wrote:

 On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


How about this:
http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/

Or you can wrap-up something similar along the same lines.

-- 
Dmitry Olshansky
Apr 06 2012
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/7/2012 8:24 AM, Dmitry Olshansky wrote:

 On 07.04.2012 2:08, Rainer Schuetze wrote:

 On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


 How about this:
 http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/


 Or you can wrap-up something similar along the same lines.


Thanks for your and other's hints on reducing garbage collected memory, 
but I find it hard to isolate larger blocks of memory for manual 
management. Most of the structs and classes are rather small.

I'm rather unhappy to sell D with the hint "Go back to manual memory 
management if you need more than 64MB of memory and want your 
application to be responsive."
Apr 07 2012
next sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 07.04.2012 18:43, Rainer Schuetze wrote:

 On 4/7/2012 8:24 AM, Dmitry Olshansky wrote:

 On 07.04.2012 2:08, Rainer Schuetze wrote:

 On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on
 the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


 How about this:
 http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/



 Or you can wrap-up something similar along the same lines.


 Thanks for your and other's hints on reducing garbage collected memory,
 but I find it hard to isolate larger blocks of memory for manual
 management. Most of the structs and classes are rather small.


Then clearly you need a custom allocator/allocation scheme. Most likely 
a mark/release or a free list kind of thing. Like say TempAlloc by 
David. As standard allocator design is still in motion you'd have to do 
your own thing ATM.

Parsers and lexers are notable examples where doing custom allocation 
pays off nicely.


 I'm rather unhappy to sell D with the hint "Go back to manual memory
 management if you need more than 64MB of memory and want your
 application to be responsive."


I totally understand this sentiment, and unless GC improves by an order 
of magnitude it is not going to work well with large to medium-scale 
apps. Particularly long running ones, I once had been running VisualD 
for about 16 hours straight (back in the days of GSOC 2011) ;)

-- 
Dmitry Olshansky
Apr 07 2012
next sibling parent reply Manu <turkeyman gmail.com> writes:
On 7 April 2012 19:04, Dmitry Olshansky <dmitry.olsh gmail.com> wrote:


 On 07.04.2012 18:43, Rainer Schuetze wrote:


 On 4/7/2012 8:24 AM, Dmitry Olshansky wrote:


 On 07.04.2012 2:08, Rainer Schuetze wrote:


 On 4/6/2012 8:01 PM, Walter Bright wrote:


 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:


 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on
 the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


 How about this:
 http://blog.thecybershadow.**net/2010/07/15/data-d-**
 unmanaged-memory-wrapper-for-**d/<http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/>



 Or you can wrap-up something similar along the same lines.


 Thanks for your and other's hints on reducing garbage collected memory,
 but I find it hard to isolate larger blocks of memory for manual
 management. Most of the structs and classes are rather small.


 Then clearly you need a custom allocator/allocation scheme. Most likely a
 mark/release or a free list kind of thing. Like say TempAlloc by David. As
 standard allocator design is still in motion you'd have to do your own
 thing ATM.

 Parsers and lexers are notable examples where doing custom allocation pays
 off nicely.


  I'm rather unhappy to sell D with the hint "Go back to manual memory

 management if you need more than 64MB of memory and want your
 application to be responsive."


 I totally understand this sentiment, and unless GC improves by an order of
 magnitude it is not going to work well with large to medium-scale apps.
 Particularly long running ones, I once had been running VisualD for about
 16 hours straight (back in the days of GSOC 2011) ;)



Yeeesss.. I run VisualD for days at a time, and it just leaks memory until
my computer chokes and crashes.
It hovers between 1gb and 2gb usage under 'normal' usage for me, on a
relatively small project (only 20-ish files).
I am now in the habit if killing and restarting it regularly, but that's
clearly not a good sign for real-world D apps...
Apr 07 2012
parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 07.04.2012 20:51, Manu wrote:
[snip]

     I totally understand this sentiment, and unless GC improves by an
     order of magnitude it is not going to work well with large to
     medium-scale apps. Particularly long running ones, I once had been
     running VisualD for about 16 hours straight (back in the days of
     GSOC 2011) ;)


 Yeeesss.. I run VisualD for days at a time, and it just leaks memory
 until my computer chokes and crashes.
 It hovers between 1gb and 2gb usage under 'normal' usage for me, on a
 relatively small project (only 20-ish files).
 I am now in the habit if killing and restarting it regularly, but that's
 clearly not a good sign for real-world D apps...


I just turned off a couple of cool things like highlighting syntax 
errors. Seems to be fast and fluid afterwards.

-- 
Dmitry Olshansky
Apr 07 2012
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Apr 7, 2012, at 9:04 AM, Dmitry Olshansky <dmitry.olsh gmail.com> wrote:

=20
 I totally understand this sentiment, and unless GC improves by an order of=


 magnitude it is not going to work well with large to medium-scale apps. Par=
ticularly long running ones, I once had been running VisualD for about 16 ho=
urs straight (back in the days of GSOC 2011) ;)

I keep all my apps open pretty much indefinitely, and restart when things sl=
ow down too much. Safari, for example, tends to leak memory and needs to be r=
estarted every few days on the outside.=20=
Apr 07 2012
prev sibling next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 4/7/2012 7:43 AM, Rainer Schuetze wrote:

 I'm rather unhappy to sell D with the hint "Go back to manual memory management
 if you need more than 64MB of memory and want your application to be
responsive."


Sure, that's why I said it was a workaround, not a solution.
Apr 07 2012
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 04/07/2012 04:43 PM, Rainer Schuetze wrote:

 On 4/7/2012 8:24 AM, Dmitry Olshansky wrote:

 On 07.04.2012 2:08, Rainer Schuetze wrote:

 On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on
 the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


 How about this:
 http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/



 Or you can wrap-up something similar along the same lines.


 Thanks for your and other's hints on reducing garbage collected memory,
 but I find it hard to isolate larger blocks of memory for manual
 management. Most of the structs and classes are rather small.


As you apparently just re-parse the whole source and throw the old AST 
away, wouldn't it be rather simple? You could just create a region 
allocator and free all the memory at once after the re-parse.


 I'm rather unhappy to sell D with the hint "Go back to manual memory
 management if you need more than 64MB of memory and want your
 application to be responsive."


I think it is actually awesome that manual memory management is possible.
Apr 07 2012
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/8/2012 12:04 AM, Timon Gehr wrote:

 On 04/07/2012 04:43 PM, Rainer Schuetze wrote:

 On 4/7/2012 8:24 AM, Dmitry Olshansky wrote:

 On 07.04.2012 2:08, Rainer Schuetze wrote:

 On 4/6/2012 8:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on
 the
 gc to catch it. Yes, this is as unsafe as using C's free().


 Actually, having to deal with lifetime issues myself takes away the
 biggest plus of the GC, so I am a bit reluctant to do this.


 How about this:
 http://blog.thecybershadow.net/2010/07/15/data-d-unmanaged-memory-wrapper-for-d/




 Or you can wrap-up something similar along the same lines.


 Thanks for your and other's hints on reducing garbage collected memory,
 but I find it hard to isolate larger blocks of memory for manual
 management. Most of the structs and classes are rather small.


 As you apparently just re-parse the whole source and throw the old AST
 away, wouldn't it be rather simple? You could just create a region
 allocator and free all the memory at once after the re-parse.


If you only use the syntax error highlighting feature that might work. 
As soon as semantic analysis kicks in modifications on a copy of the 
parse tree are done lazily, and as symbols get resolved, references into 
other trees are remembered. It can get rather involved to figure out 
when it is safe to delete a memory block manually.

The region allocator has the advantage that it does not need the 
alignment to the power of 2 for each allocation, so memory could be 
saved, too.

I guess, I'll try something along the line with some additional 
functions to avoid bad references after deletion.


 I'm rather unhappy to sell D with the hint "Go back to manual memory
 management if you need more than 64MB of memory and want your
 application to be responsive."


 I think it is actually awesome that manual memory management is possible.
Apr 08 2012
prev sibling parent reply Chad J <chadjoan __spam.is.bad__gmail.com> writes:
On 04/06/2012 02:01 PM, Walter Bright wrote:

 On 4/6/2012 10:37 AM, Rainer Schuetze wrote:

 I hope there is something wrong with my reasoning, and that you could
 give me
 some hints to avoid the memory bloat and the application stalls.


 A couple of things you can try (they are workarounds, not solutions):

 1. Actively delete memory you no longer need, rather than relying on the
 gc to catch it. Yes, this is as unsafe as using C's free().


I keep reading that 'delete' is going to go away.  Is this even 
future-proof, or is code written in this fashion going to suffer a 
reckoning later on?

As an aside: I think this point generalizes to "avoid using the GC in 
cases where it is easy to do so".  D is very good at this due to having 
expressive structs, scope keyword, array slices, and a number of other 
features that synergize to track ownership/lifetime of memory without GC 
help.


 2. Null out pointers & references when you are done with them. This
 helps reduce unwanted pinning of unused gc memory.

 3. Minimize use of global memory, as that is a major source of source of
 roots.
Apr 07 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/7/12 4:26 PM, Chad J wrote:

 I keep reading that 'delete' is going to go away. Is this even
 future-proof, or is code written in this fashion going to suffer a
 reckoning later on?


Biggest confusions in the history of humankind:

1. Pyramids have been built by aliens;

2. Flying with a device heavier than air is impossible;

3. The ability to dispose of memory will disappear along with the delete 
keyword.


Andrei
Apr 07 2012
next sibling parent Chad J <chadjoan __spam.is.bad__gmail.com> writes:
On 04/07/2012 05:42 PM, Andrei Alexandrescu wrote:

 On 4/7/12 4:26 PM, Chad J wrote:

 I keep reading that 'delete' is going to go away. Is this even
 future-proof, or is code written in this fashion going to suffer a
 reckoning later on?


 Biggest confusions in the history of humankind:

 1. Pyramids have been built by aliens;

 2. Flying with a device heavier than air is impossible;

 3. The ability to dispose of memory will disappear along with the delete
 keyword.


 Andrei


Oh.  Whoops.

Thanks!
Apr 07 2012
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 4/7/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 3. The ability to dispose of memory will disappear along with the delete
 keyword.


Pull this and hopefully that myth will come to an end:
https://github.com/D-Programming-Language/d-programming-language.org/pull/112
Apr 08 2012
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/8/12 4:20 AM, Andrej Mitrovic wrote:

 On 4/7/12, Andrei Alexandrescu<SeeWebsiteForEmail erdani.org>  wrote:

 3. The ability to dispose of memory will disappear along with the delete
 keyword.


 Pull this and hopefully that myth will come to an end:
 https://github.com/D-Programming-Language/d-programming-language.org/pull/112


Done.

Andrei
Apr 08 2012
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Apr 08, 2012 at 09:42:42AM -0500, Andrei Alexandrescu wrote:

 On 4/8/12 4:20 AM, Andrej Mitrovic wrote:

On 4/7/12, Andrei Alexandrescu<SeeWebsiteForEmail erdani.org>  wrote:

3. The ability to dispose of memory will disappear along with the
delete keyword.


Pull this and hopefully that myth will come to an end:
https://github.com/D-Programming-Language/d-programming-language.org/pull/112




[...]

I have to say, that is one cool thing about having .init for all types.
So we can cleanup after an object (close files, release resources, etc.)
and still have the GC do whatever it needs to do, when it needs to. I
love it!!


T

-- 
The day Microsoft makes something that doesn't suck is probably the day
they start making vacuum cleaners... -- Slashdotter
Apr 08 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-04-06 19:37, Rainer Schuetze wrote:

 GC issues like this are currently blocking development of Visual D (a
 Win32 project): when just adding spaces to a file, parsing the new file
 every other second often needs 10 or more parsings until an equal amount
 of memory is collected compared to the allocated memory. AFAICT Visual D
 just keeps a reference to the root of the most recent AST of a source file.

 What's even worse: when the allocated memory gets larger (say > 200MB),
 the garbage collection itself takes more than a second stalling the
 application, which is a real pain if it happens while you are typing
 source text (it does happen quite often).


Can you pause the GC when the user is typing? When you're finished with 
the processing you can start it again.

-- 
/Jacob Carlborg
Apr 07 2012
parent reply Manu <turkeyman gmail.com> writes:
On 7 April 2012 17:03, Jacob Carlborg <doob me.com> wrote:


 On 2012-04-06 19:37, Rainer Schuetze wrote:


 GC issues like this are currently blocking development of Visual D (a
 Win32 project): when just adding spaces to a file, parsing the new file
 every other second often needs 10 or more parsings until an equal amount
 of memory is collected compared to the allocated memory. AFAICT Visual D
 just keeps a reference to the root of the most recent AST of a source
 file.

 What's even worse: when the allocated memory gets larger (say > 200MB),
 the garbage collection itself takes more than a second stalling the
 application, which is a real pain if it happens while you are typing
 source text (it does happen quite often).


 Can you pause the GC when the user is typing? When you're finished with
 the processing you can start it again.



There's a bit of a problem there though, when you're coding, when are you
NOT typing? :)
I don't ever stop and sit there patiently for a few seconds for no reason.
Apr 07 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-04-07 17:41, Manu wrote:

 On 7 April 2012 17:03, Jacob Carlborg <doob me.com <mailto:doob me.com>>
 wrote:

     On 2012-04-06 19:37, Rainer Schuetze wrote:


         GC issues like this are currently blocking development of Visual
         D (a
         Win32 project): when just adding spaces to a file, parsing the
         new file
         every other second often needs 10 or more parsings until an
         equal amount
         of memory is collected compared to the allocated memory. AFAICT
         Visual D
         just keeps a reference to the root of the most recent AST of a
         source file.

         What's even worse: when the allocated memory gets larger (say >
         200MB),
         the garbage collection itself takes more than a second stalling the
         application, which is a real pain if it happens while you are typing
         source text (it does happen quite often).


     Can you pause the GC when the user is typing? When you're finished
     with the processing you can start it again.


 There's a bit of a problem there though, when you're coding, when are
 you NOT typing? :)
 I don't ever stop and sit there patiently for a few seconds for no reason.


It depends. I can do quite a lot of coding without typing. But yeah, I 
type a lot. It would basically turn on and off the GC, probably not good.

-- 
/Jacob Carlborg
Apr 07 2012
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302


 Andrei


 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/


I studied the GC a bit more and noticed a possible issue:

- memory allocations are aligned up to a power of 2 <= page size
- the memory area beyond the actually requested size is left untouched 
when allocating
- when the memory is collected, it is also untouched
- the marking of references during collection does not know the 
requested size, so it scans the full memory block

Result: When a collected memory block is reused by a smaller allocation, 
there might still be false pointers in the unused area.

When I clear this data, my first impression is that it has improved the 
situation, but not enough. I'll have to create some non-interactive test 
to verify.

Rainer
Apr 07 2012
next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Apr 7, 2012, at 9:45 AM, Rainer Schuetze <r.sagitario gmx.de> wrote:


=20
=20
 On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a =C3=A9crit :

 A few more samples of people's perception of the two languages:
=20
 http://news.ycombinator.com/item?id=3D3805302
=20
=20
 Andrei


=20
 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-=




non-precise-gc/

=20


=20
 I studied the GC a bit more and noticed a possible issue:
=20
 - memory allocations are aligned up to a power of 2 <=3D page size
 - the memory area beyond the actually requested size is left untouched whe=


n allocating

 - when the memory is collected, it is also untouched
 - the marking of references during collection does not know the requested s=


ize, so it scans the full memory block

=20
 Result: When a collected memory block is reused by a smaller allocation, t=


here might still be false pointers in the unused area.

=20
 When I clear this data, my first impression is that it has improved the si=


tuation, but not enough. I'll have to create some non-interactive test to ve=
rify.

The unused area is typically zeroed out on allocation. Check GC.mallocNoSync=
.=20=
Apr 07 2012
parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 4/7/2012 8:26 PM, Sean Kelly wrote:

 On Apr 7, 2012, at 9:45 AM, Rainer Schuetze<r.sagitario gmx.de>  wrote:


 On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a écrit :

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302


 Andrei


 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-using-non-precise-gc/


 I studied the GC a bit more and noticed a possible issue:

 - memory allocations are aligned up to a power of 2<= page size
 - the memory area beyond the actually requested size is left untouched when
allocating
 - when the memory is collected, it is also untouched
 - the marking of references during collection does not know the requested
size, so it scans the full memory block

 Result: When a collected memory block is reused by a smaller allocation, there
might still be false pointers in the unused area.

 When I clear this data, my first impression is that it has improved the
situation, but not enough. I'll have to create some non-interactive test to
verify.


 The unused area is typically zeroed out on allocation. Check GC.mallocNoSync..


That's where I added it ;-) But in fact it is in malloc/calloc and a few 
more places, though I haven't checked whether all the other calls of 
mallocNoSync handle the situation correctly. Almost all calls are to 
malloc anyway, so my patch doesn't change anything.
Apr 07 2012
parent Sean Kelly <sean invisibleduck.org> writes:
On Apr 7, 2012, at 12:37 PM, Rainer Schuetze <r.sagitario gmx.de> wrote:


=20
=20
 On 4/7/2012 8:26 PM, Sean Kelly wrote:

 On Apr 7, 2012, at 9:45 AM, Rainer Schuetze<r.sagitario gmx.de>  wrote:
=20

=20
=20
 On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a =C3=A9crit :

 A few more samples of people's perception of the two languages:
=20
 http://news.ycombinator.com/item?id=3D3805302
=20
=20
 Andrei


=20
 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-usin=








g-non-precise-gc/

=20


=20
 I studied the GC a bit more and noticed a possible issue:
=20
 - memory allocations are aligned up to a power of 2<=3D page size
 - the memory area beyond the actually requested size is left untouched w=






hen allocating

 - when the memory is collected, it is also untouched
 - the marking of references during collection does not know the requeste=






d size, so it scans the full memory block

=20
 Result: When a collected memory block is reused by a smaller allocation,=






 there might still be false pointers in the unused area.

=20
 When I clear this data, my first impression is that it has improved the s=






ituation, but not enough. I'll have to create some non-interactive test to v=
erify.

=20
 The unused area is typically zeroed out on allocation. Check GC.mallocNoS=




ync..

=20
 That's where I added it ;-) But in fact it is in malloc/calloc and a few m=


ore places, though I haven't checked whether all the other calls of mallocNo=
Sync handle the situation correctly. Almost all calls are to malloc anyway, s=
o my patch doesn't change anything.

Is it not already there?  Maybe it was removed in all the GC optimizations? =
 I certainly remember a memset(0) in there.=20=
Apr 07 2012
prev sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Sat, 07 Apr 2012 12:45:44 -0400, Rainer Schuetze <r.sagitario gmx.de>=
  =

wrote:


 On 4/6/2012 6:20 PM, deadalnix wrote:

 Le 06/04/2012 18:07, Andrei Alexandrescu a =C3=A9crit :

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3D3805302


 Andrei


 I did some measurement on that point for D lately :
 http://www.deadalnix.me/2012/03/05/impact-of-64bits-vs-32bits-when-us=




ing-non-precise-gc/


 I studied the GC a bit more and noticed a possible issue:

 - memory allocations are aligned up to a power of 2 <=3D page size
 - the memory area beyond the actually requested size is left untouched=


  =


 when allocating


No, it's zeroed if the block is requested without the NO_SCAN bit set.

see:  =

https://github.com/D-Programming-Language/druntime/blob/master/src/gc/gc=
x.d#L479

Note to Sean, it's not in the no-sync part (makes sense, why hold the lo=
ck  =

while you are memsetting).

-Steve



 Apr 09 2012



prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com>  writes:


Andrei Alexandrescu:


 A few more samples of people's perception of the two languages:
 
 http://news.ycombinator.com/item?id=3805302


Some of the comments in that ycombinator thread are a bit unnerving.

There is a similar thread on Reddit too:
http://www.reddit.com/r/programming/comments/rvwj0/go_severe_memory_problems_on_32bit_systems/

Two quotes from the Reddit page:


The Boehm GC attempts to mitigate this by detecting false references to free
blocks and blacklisting before they become references to live blocks,
restricting their use to low-impact situations.


Also Boehm supports typed allocation(gc_typed.h/GC_malloc_explicitly_typed)
where you tell GC that pointers are located only at specific offsets and
everything other should be ignored.<


tag data as no-pointers and allocate in separate section. The garbage collector
can avoid scanning this section, with reduces collection time as well as the
number of false positives.<


Bye,
bearophile



 Apr 06 2012



 next sibling parent  Sean Kelly <sean invisibleduck.org>  writes:


It shouldn't be hard to retrofit the Boehm collector into D if anyone cares t=
o make a comparison.=20

On Apr 6, 2012, at 11:36 AM, bearophile <bearophileHUGS lycos.com> wrote:


 Andrei Alexandrescu:
=20

 A few more samples of people's perception of the two languages:
=20
 http://news.ycombinator.com/item?id=3D3805302


=20
 Some of the comments in that ycombinator thread are a bit unnerving.
=20
 There is a similar thread on Reddit too:
 http://www.reddit.com/r/programming/comments/rvwj0/go_severe_memory_proble=


ms_on_32bit_systems/

=20
 Two quotes from the Reddit page:
=20

 The Boehm GC attempts to mitigate this by detecting false references to f=




ree blocks and blacklisting before they become references to live blocks, re=
stricting their use to low-impact situations.

 Also Boehm supports typed allocation(gc_typed.h/GC_malloc_explicitly_typed=


) where you tell GC that pointers are located only at specific offsets and e=
verything other should be ignored.<

=20

 tag data as no-pointers and allocate in separate section. The garbage col=




lector can avoid scanning this section, with reduces collection time as well=
 as the number of false positives.<

=20
 Bye,
 bearophile





 Apr 06 2012



prev sibling  parent  bearophile <bearophileHUGS lycos.com>  writes:


In the Reddit thread they have also linked this paper, "Precise Garbage
Collection for C", by Jon Rafkind, Adam Wick,  John Regehr and Matthew Flatt:

http://www.cs.utah.edu/~regehr/papers/ismm15-rafkind.pdf

It contains some ideas (and it seems my idea of a standard optioanl onGC()
method for unions/structs/classes is not so bad).

Bye,
bearophile



 Apr 06 2012



prev sibling  parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 4/6/2012 9:07 AM, Andrei Alexandrescu wrote:

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302



At least we don't have this issue: http://news.ycombinator.com/item?id=3814020

The D gc allocates smallish "chunks" as required, not one giant virtual arena.



 Apr 08 2012



  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/8/12 12:21 PM, Walter Bright wrote:

 On 4/6/2012 9:07 AM, Andrei Alexandrescu wrote:

 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/item?id=3805302



 At least we don't have this issue:
 http://news.ycombinator.com/item?id=3814020

 The D gc allocates smallish "chunks" as required, not one giant virtual
 arena.


On reddit, too: 
http://www.reddit.com/r/programming/comments/rzd19/do_not_use_go_for_32bit_development/

I find that behavior quite surprising because, unlike imprecise GC, is 
much easier to improve.

Anyhow, the recent discussions on Go clarify that we need to improve our 
collector's precision, and pronto. The only thing that didn't yet make 
the problem more painful in D is that D programs create a lot less garbage.


Andrei



 Apr 08 2012



  parent reply Manu <turkeyman gmail.com>  writes:


On 8 April 2012 23:44, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>wrote:


 On 4/8/12 12:21 PM, Walter Bright wrote:


 On 4/6/2012 9:07 AM, Andrei Alexandrescu wrote:


 A few more samples of people's perception of the two languages:

 http://news.ycombinator.com/**item?id=3805302<http://news.ycombinator.com/item?id=3805302>



 At least we don't have this issue:
 http://news.ycombinator.com/**item?id=3814020<http://news.ycombinator.com/item?id=3814020>

 The D gc allocates smallish "chunks" as required, not one giant virtual
 arena.


 On reddit, too: http://www.reddit.com/r/**programming/comments/rzd19/do_**
 not_use_go_for_32bit_**development/<http://www.reddit.com/r/programming/comments/rzd19/do_not_use_go_for_32bit_development/>

 I find that behavior quite surprising because, unlike imprecise GC, is
 much easier to improve.

 Anyhow, the recent discussions on Go clarify that we need to improve our
 collector's precision, and pronto. The only thing that didn't yet make the
 problem more painful in D is that D programs create a lot less garbage.



What do you base that statistic on? I'm not arguing that fact, just that I
haven't seen any evidence one way or the other. What causes Go to create
significantly more garbage than D? Are there benchmarks or test cases I
should be aware of on the topic?
I'm quite curious to know more about this stuff, but I haven't found that

interest to me, since most game devs are most familiar with that)



 Apr 08 2012



 next sibling parent reply Walter Bright <newshound2 digitalmars.com>  writes:


On 4/8/2012 3:57 PM, Manu wrote:

 What do you base that statistic on? I'm not arguing that fact, just that I
 haven't seen any evidence one way or the other. What causes Go to create
 significantly more garbage than D? Are there benchmarks or test cases I should
 be aware of on the topic?


The first ycombinator reference is a person who didn't run out of memory using 
D. That implies far less pressure on the gc.

My understanding of Go is that when it does structural conformance, it builds 
some of the necessary data at runtime on the gc heap.

Anyhow, D has a lot of facilities for putting things on the stack rather than 
the heap, immutable data doesn't need to get copied, and slices allow lots of 
reuse of existing objects.



 Apr 08 2012



 next sibling parent  "bearophile" <bearophileHUGS lycos.com>  writes:


Walter:


 Anyhow, D has a lot of facilities for putting things on the 
 stack rather than the heap,


In a system language heap allocations are often among the spots 
where the program is slower. So a good system language must offer 
good ways to use the stack as much as possible. D offers some 
ways to use the stack, but surely there is a good amount of space 
for improvements, some examples:

1) Some kind of optimization to avoid heap allocations for lines 
of code like this will improve the situation:
int[3] a = [10,20,30];

2) Some kind of (safer) stack-allocated variable-length arrays 
are handy (see Ada language too that uses stack-allocated arrays 
a lot):
http://d.puremagic.com/issues/show_bug.cgi?id=5348

3) Currently emplace has several problems and limits.

4) A compilation switch to list all points where a heap-allocetd 
closure is created will help avoid unwanted heap usage:
http://d.puremagic.com/issues/show_bug.cgi?id=5070

5 extra) Currently there are no ways to put an immutable 
associative array on the stack, or as global immutable variable 
:-) Many of my small associative arrays don't need to mutate 
after I have created them at the start of the program. Having a 
way to put them on the stack seems an interesting thing.

Bye,
bearophile



 Apr 08 2012



prev sibling  parent reply Manu <turkeyman gmail.com>  writes:


On 9 April 2012 02:24, Walter Bright <newshound2 digitalmars.com> wrote:


 On 4/8/2012 3:57 PM, Manu wrote:


 What do you base that statistic on? I'm not arguing that fact, just that I
 haven't seen any evidence one way or the other. What causes Go to create
 significantly more garbage than D? Are there benchmarks or test cases I
 should
 be aware of on the topic?


 The first ycombinator reference is a person who didn't run out of memory
 using D. That implies far less pressure on the gc.

 My understanding of Go is that when it does structural conformance, it
 builds some of the necessary data at runtime on the gc heap.

 Anyhow, D has a lot of facilities for putting things on the stack rather
 than the heap, immutable data doesn't need to get copied, and slices allow
 lots of reuse of existing objects.


"optimized D was slightly faster than Go at almost anything and consumed up
to 70% less memory"
Interesting... I don't know enough about Go to reason that finding, I guess
I assumed it has most of the same possibilities available to D. (no
immutable data? no stack structs? no references/pointers/slices? crazy...)

The only D program I have significant experience with is VisualD, and it
hogs 1-2gb of ram for me under general usage, and eventually crashes, after
paging heavily and bringing my computer to a crawl. Not a good sign from
the first and only productive D app I've run yet ;)
This seems a lot like his experience with Go... but comparisons aside, D
still clearly isn't there yet when it comes to the GC either, and I'm
amazed Google thing Go is production ready if that guys findings are true!



 Apr 08 2012



 next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <xtzgzorex gmail.com>  writes:


On 09-04-2012 02:18, Manu wrote:

 On 9 April 2012 02:24, Walter Bright <newshound2 digitalmars.com
 <mailto:newshound2 digitalmars.com>> wrote:

     On 4/8/2012 3:57 PM, Manu wrote:

         What do you base that statistic on? I'm not arguing that fact,
         just that I
         haven't seen any evidence one way or the other. What causes Go
         to create
         significantly more garbage than D? Are there benchmarks or test
         cases I should
         be aware of on the topic?


     The first ycombinator reference is a person who didn't run out of
     memory using D. That implies far less pressure on the gc.

     My understanding of Go is that when it does structural conformance,
     it builds some of the necessary data at runtime on the gc heap.

     Anyhow, D has a lot of facilities for putting things on the stack
     rather than the heap, immutable data doesn't need to get copied, and
     slices allow lots of reuse of existing objects.


 "optimized D was slightly faster than Go at almost anything and consumed
 up to 70% less memory"
 Interesting... I don't know enough about Go to reason that finding, I
 guess I assumed it has most of the same possibilities available to D.
 (no immutable data? no stack structs? no references/pointers/slices?
 crazy...)

 The only D program I have significant experience with is VisualD, and it
 hogs 1-2gb of ram for me under general usage, and eventually crashes,
 after paging heavily and bringing my computer to a crawl. Not a good
 sign from the first and only productive D app I've run yet ;)
 This seems a lot like his experience with Go... but comparisons aside, D
 still clearly isn't there yet when it comes to the GC either, and I'm
 amazed Google thing Go is production ready if that guys findings are true!


Google likes to invent random useless languages. See: Dart. Both 
languages are solutions looking for problems. ;)

And yes, precise GC is more essential than most people think.

-- 
- Alex



 Apr 08 2012



 next sibling parent  Jacob Carlborg <doob me.com>  writes:


On 2012-04-09 02:24, Alex Rønne Petersen wrote:


 Google likes to invent random useless languages. See: Dart. Both
 languages are solutions looking for problems. ;)


Actually I like the idea behind Dart, to replace JavaScript. But that's 
basically the only think I like about it.

-- 
/Jacob Carlborg



 Apr 09 2012



prev sibling  parent  deadalnix <deadalnix gmail.com>  writes:


Le 09/04/2012 02:24, Alex Rønne Petersen a écrit :

 On 09-04-2012 02:18, Manu wrote:

 On 9 April 2012 02:24, Walter Bright <newshound2 digitalmars.com
 <mailto:newshound2 digitalmars.com>> wrote:

 On 4/8/2012 3:57 PM, Manu wrote:

 What do you base that statistic on? I'm not arguing that fact,
 just that I
 haven't seen any evidence one way or the other. What causes Go
 to create
 significantly more garbage than D? Are there benchmarks or test
 cases I should
 be aware of on the topic?


 The first ycombinator reference is a person who didn't run out of
 memory using D. That implies far less pressure on the gc.

 My understanding of Go is that when it does structural conformance,
 it builds some of the necessary data at runtime on the gc heap.

 Anyhow, D has a lot of facilities for putting things on the stack
 rather than the heap, immutable data doesn't need to get copied, and
 slices allow lots of reuse of existing objects.


 "optimized D was slightly faster than Go at almost anything and consumed
 up to 70% less memory"
 Interesting... I don't know enough about Go to reason that finding, I
 guess I assumed it has most of the same possibilities available to D.
 (no immutable data? no stack structs? no references/pointers/slices?
 crazy...)

 The only D program I have significant experience with is VisualD, and it
 hogs 1-2gb of ram for me under general usage, and eventually crashes,
 after paging heavily and bringing my computer to a crawl. Not a good
 sign from the first and only productive D app I've run yet ;)
 This seems a lot like his experience with Go... but comparisons aside, D
 still clearly isn't there yet when it comes to the GC either, and I'm
 amazed Google thing Go is production ready if that guys findings are
 true!


 Google likes to invent random useless languages. See: Dart. Both
 languages are solutions looking for problems. ;)

 And yes, precise GC is more essential than most people think.


I posted some an article about that. 64bits pretty much solve the false 
positive problem.

Still, precise GC is nice, but alternative like precise on heap and 
imprecise on stack are very valid alternatives.



 Apr 09 2012



prev sibling  parent  "SomeDude" <lovelydear mailmetrash.com>  writes:


On Monday, 9 April 2012 at 00:18:22 UTC, Manu wrote:

 On 9 April 2012 02:24, Walter Bright 
 <newshound2 digitalmars.com> wrote:

 still clearly isn't there yet when it comes to the GC either, 
 and I'm
 amazed Google thing Go is production ready if that guys 
 findings are true!


They use it on 64 bits servers with tons of RAM. But they can't 
target Android powered mobile devices with that language, and D 
may have similar issues.
On what kind of server does the web forum run ?



 Apr 09 2012



prev sibling  parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/8/12 5:57 PM, Manu wrote:

 On 8 April 2012 23:44, Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org <mailto:SeeWebsiteForEmail erdani.org>>
 wrote:
     Anyhow, the recent discussions on Go clarify that we need to improve
     our collector's precision, and pronto. The only thing that didn't
     yet make the problem more painful in D is that D programs create a
     lot less garbage.


 What do you base that statistic on?


"A lot less" is hardly a statistic :o).


 I'm not arguing that fact, just that
 I haven't seen any evidence one way or the other. What causes Go to
 create significantly more garbage than D?


As Walter mentioned, Go uses indirection and implicit allocation for its 
fundamental abstraction mechanisms. D has elaborate value types, the 
scope statement, and pass-by-alias, all of which drastically reduce the 
amount of garbage created.


Andrei



 Apr 08 2012



 next sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com>  writes:


On 4/9/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 and pass-by-alias


Speaking of alias, one killer feature would be to enable using alias
for expressions. E.g.:

struct Window { struct Point { int x, y; } Point point; }
void test() {
    Window window;
    alias window.point.x x;
    // use 'x' here which is really window.point.x
}

It makes it simpler to manipulate nested structs and their fields by
reference without involving pointers or using with statements. AFAIK
C++ can use references for this purpose (ala &int x =
window.point.x;), but I guess this isn't very efficient unless the
compiler can optimize it.

Besides myself I've also seen other people request it (I think Nick S.
wanted the feature).



 Apr 08 2012



 next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>  writes:


On 4/8/12 7:21 PM, Andrej Mitrovic wrote:

 On 4/9/12, Andrei Alexandrescu<SeeWebsiteForEmail erdani.org>  wrote:

 and pass-by-alias


 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:

 struct Window { struct Point { int x, y; } Point point; }
 void test() {
      Window window;
      alias window.point.x x;
      // use 'x' here which is really window.point.x
 }


Yah, we should add that at some point. Walter and I discussed about it 
and it's virtually approved. But to be on the conservative side, it's 
not for expressions but for mere pointer-chasing chains.

Andrei



 Apr 08 2012



  parent reply Manu <turkeyman gmail.com>  writes:


On 9 April 2012 03:25, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org>wrote:


 On 4/8/12 7:21 PM, Andrej Mitrovic wrote:


 On 4/9/12, Andrei Alexandrescu<SeeWebsiteForEmai**l erdani.org<SeeWebsiteForEmail erdani.org>>
  wrote:


 and pass-by-alias


 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:

 struct Window { struct Point { int x, y; } Point point; }
 void test() {
     Window window;
     alias window.point.x x;
     // use 'x' here which is really window.point.x
 }


 Yah, we should add that at some point. Walter and I discussed about it and
 it's virtually approved. But to be on the conservative side, it's not for
 expressions but for mere pointer-chasing chains.



Why use alias, instead of allowing 'ref' on local declarations?
The alias approach suffers from complications when referencing a complex
expression. Particularly if that expression involves a non-pure function
call. A local ref would seem less problematic to me?



 Apr 08 2012



  parent  Marco Leise <Marco.Leise gmx.de>  writes:


Am Mon, 9 Apr 2012 03:38:15 +0300
schrieb Manu <turkeyman gmail.com>:


 On 9 April 2012 03:25, Andrei Alexandrescu
<SeeWebsiteForEmail erdani.org>wrote:
 

 On 4/8/12 7:21 PM, Andrej Mitrovic wrote:


 On 4/9/12, Andrei Alexandrescu<SeeWebsiteForEmai**l erdani.org<SeeWebsiteForEmail erdani.org>>
  wrote:


 and pass-by-alias


 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:

 struct Window { struct Point { int x, y; } Point point; }
 void test() {
     Window window;
     alias window.point.x x;
     // use 'x' here which is really window.point.x
 }


 Yah, we should add that at some point. Walter and I discussed about it and
 it's virtually approved. But to be on the conservative side, it's not for
 expressions but for mere pointer-chasing chains.


 
 
 Why use alias, instead of allowing 'ref' on local declarations?
 The alias approach suffers from complications when referencing a complex
 expression. Particularly if that expression involves a non-pure function
 call. A local ref would seem less problematic to me?


This is a valid concern. I remember the discussion from a while back. alias
makes it _look_ shorter and right, but you get what Manu said. Here is another
look on it:

MyClass a = foo();
MyClass x = bar();
alias a.c c1;
ref c2 = a.c;
a = x;
c1 = 5; // <- oops, using x instead of a
c2 = 5; // <- more what I expect

Not to say alias for 'moving' targets is a no-go, but I'd rather proper refs. :)

-- 
Marco



 Apr 08 2012



prev sibling  parent  Jacob Carlborg <doob me.com>  writes:


On 2012-04-09 02:21, Andrej Mitrovic wrote:

 On 4/9/12, Andrei Alexandrescu<SeeWebsiteForEmail erdani.org>  wrote:

 and pass-by-alias


 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:

 struct Window { struct Point { int x, y; } Point point; }
 void test() {
      Window window;
      alias window.point.x x;
      // use 'x' here which is really window.point.x
 }

 It makes it simpler to manipulate nested structs and their fields by
 reference without involving pointers or using with statements. AFAIK
 C++ can use references for this purpose (ala&int x =
 window.point.x;), but I guess this isn't very efficient unless the
 compiler can optimize it.

 Besides myself I've also seen other people request it (I think Nick S.
 wanted the feature).


I want this feature as well. I also want it to be possible to document.

-- 
/Jacob Carlborg



 Apr 09 2012



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On 9 April 2012 03:21, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:


 On 4/9/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 and pass-by-alias


 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:

 struct Window { struct Point { int x, y; } Point point; }
 void test() {
    Window window;
    alias window.point.x x;
    // use 'x' here which is really window.point.x
 }

 It makes it simpler to manipulate nested structs and their fields by
 reference without involving pointers or using with statements. AFAIK
 C++ can use references for this purpose (ala &int x =
 window.point.x;), but I guess this isn't very efficient unless the
 compiler can optimize it.


I can't think of many cases where the compiler can't optimise it when used
in the context you describe.


Besides myself I've also seen other people request it (I think Nick S.

 wanted the feature).


I probably wouldn't have thought to use alias in that way, I probably would
have asked to be able to use 'ref' on any declaration...
I do also get a little annoyed having to use pointers in this situation. At
least they don't suffer a different dereference syntax like C, but it does
seem a bit flimsy that they can be reassigned, can also be null, and I have
to involve the & operator, which can often lead to parentheses spam.

(...why can't you use 'ref' in regular declarations? I frequently find
myself wanting to use ref locally for this exact reason.)



 Apr 08 2012



prev sibling next sibling parent  Andrej Mitrovic <andrej.mitrovich gmail.com>  writes:


On 4/9/12, Manu <turkeyman gmail.com> wrote:

 (...why can't you use 'ref' in regular declarations? I frequently find
 myself wanting to use ref locally for this exact reason.)


I think the reason for this was because references are supposed to be
hidden from the user.

But if you look it from a safety angle you can escape a reference to a
local variable (ends up being garbage after function exit), but you
can't escape an alias declaration (I think so anywho).



 Apr 08 2012



prev sibling next sibling parent  Manu <turkeyman gmail.com>  writes:


On 9 April 2012 04:09, Andrej Mitrovic <andrej.mitrovich gmail.com> wrote:


 On 4/9/12, Manu <turkeyman gmail.com> wrote:

 I don't follow. Can you give an example that shows this insecurity?


 I mean escaping references to locals:

 ref int xref;
 void foo() {
   int x;
   xref = x;
 }

 or

 ref int foo() {
   int x;
   ref int xref = x;
   return xref;
 }

 I mean a ref would basically be a pointer with some syntax sugar, no?
 It would have the same drawbacks as a pointer.


Nobody returns a ref to a local from a function, and the compiler can
easily warn about that.
Sure, but that's all this was ever meant to be right? alias as a sugar to
simplify long expressions... except alias is unsafe too, but in a different
and more subtle way.



 Apr 09 2012



prev sibling  parent  Artur Skawina <art.08.09 gmail.com>  writes:


On 04/09/12 02:21, Andrej Mitrovic wrote:

 On 4/9/12, Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> wrote:

 and pass-by-alias


 
 Speaking of alias, one killer feature would be to enable using alias
 for expressions. E.g.:
 
 struct Window { struct Point { int x, y; } Point point; }
 void test() {
     Window window;
     alias window.point.x x;
     // use 'x' here which is really window.point.x
 }
 
 It makes it simpler to manipulate nested structs and their fields by
 reference without involving pointers or using with statements. AFAIK
 C++ can use references for this purpose (ala &int x =
 window.point.x;), but I guess this isn't very efficient unless the
 compiler can optimize it.


struct Window { struct Point { int x, y; } Point point; }
void test() {
    Window window;
     property ref x() { return window.point.x; }
    // use 'x' here which is really window.point.x
}

And, yes, the compiler can and does optimize it away.

artur



 Apr 09 2012