www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - LinkedIn Article to be: Why you need to start moving off C/C++ to D,

reply "John Carter" <john.carter taitradio.com> writes:
So LinkedIn invited me to write an article.... yeah, yeah, I 
know, they invite everybody and their brother.

But I figure this could be useful.

I want to be programming in D for my next project.

And that will only happen if corporate ecosystem around me starts 
adopting it.

So I plan to write an article for them, and put a bit of effort 
and polish in to it.

I have been digesting every dconf2014 video posted so far.

Really excellent points in many of them, especially in the 
compile time reflection and tiny ubiquitous machines talks.

So this is a call for input... there is a lot of good material on 
dlang.org, which I will use (steal) shamelessly.

Any other good blog posts / social media comments / pointers I 
can digest and use?

The hard fact is it wouldn't be so urgent to move to D if C/C++ 
didn't hurt so much.

Walter and Andrei are very nice peacable guys, they very sensibly 
concentrate on selling the good points of D, rather than slagging 
the weak points of C/C++.

I'm on the other side of the fence.

I live the pain of C/C++ and have no qualms about pointing them 
out.

So pointers to sound reasoned analysis of the problems of 
C/C++/Java are welcome too.

I might put a bit of an embedded device slant on the whole thing, 
since...

* That is part of my personal core competance.
* That is where the greatest growth in the computer industry will 
happen.
* That is where there is a very strong case for D replacing C/C++.

Once I have digested any links/ideas you guys have... I will post 
a draft for you guys to review / comment on.
Jul 14 2014
next sibling parent "Right" <Ohright ohplease.com> writes:
      D might someday be a replacement for those languages, but
right
now, with a bad GC as the default memory model? I don't think so.


meta-programming.
Jul 14 2014
prev sibling next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via Digitalmars-d wrote:
[...]

 Any other good blog posts / social media comments / pointers I can
 digest and use?


This one came to mind:

http://bartoszmilewski.com/2013/09/19/edward-chands/



 The hard fact is it wouldn't be so urgent to move to D if C/C++ didn't
 hurt so much.


I feel your pain. :) I was programming C/C++ for decades...  starting
with C, then C++, then back to C (after being horribly, horribly scarred
by C++), then settled somewhere in the middle (basically using C++ as "C
with classes", rather than the oft-touted advanced C++ style, which
seemed to me like investing your life in a tower of cards ready to
crumble at the slightest provocation). All the while, I was longing for
something better. Java didn't appeal to me for various reasons (its
verbosity, which is intolerable without an IDE -- and I hate IDEs; its
shoving OO idealogy down your throat even where OO clearly doesn't fit
well, such as classes with only static members, for example; the ugly
mess with boxed/unboxed types; difficulty of interfacing with other
languages like C/C++ libraries; etc.).  I was very happy to discover D.
:)


[...]

 I might put a bit of an embedded device slant on the whole thing,
 since...
 
 * That is part of my personal core competance.
 * That is where the greatest growth in the computer industry will happen.
 * That is where there is a very strong case for D replacing C/C++.


Before we start selling D on embedded devices, how sure are we that D is
ready to take on the task? I.e., how well does it handle low-memory
devices? ARM architectures? We better have (very good) answers for
these, otherwise we risk destroying D's cause by defending it poorly.


T

-- 
Trying to define yourself is like trying to bite your own teeth. -- Alan Watts
Jul 14 2014
next sibling parent reply "John Carter" <john.carter taitradio.com> writes:
On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Before we start selling D on embedded devices, how sure are we 
 that D is
 ready to take on the task? I.e., how well does it handle 
 low-memory
 devices? ARM architectures? We better have (very good) answers 
 for
 these, otherwise we risk destroying D's cause by defending it 
 poorly.



Good point, that's why I said "Start moving".

These issues never just "Solve Themselves".

Somebody somewhere must start moving, start derisking, preparing 
the toolchain.

Even a simple thing like moving from one version of gcc to 
another has non-zero risk, and some warm body has to do the work 
to get it ready for that particular project / target CPU / target 
OS / board / host CPU / host OS...

Yup, I've been that guy.

However, if nobody moves you don't get there. You sit on an ever 
crustier version of the compiler while your competitors stream 
past you.

It's a call to arms to guys like me, the guys who line up the 
toolchain, build the cross compilers for roll out to the rest of 
the devs, to start moving.

Part of what I want to say in that article is outline what needs 
to be done to get us from "Here" to "There".
Jul 14 2014
parent "Jesse Phillips" <Jesse.K.Phillips+D gmail.com> writes:
On Tuesday, 15 July 2014 at 00:21:28 UTC, John Carter wrote:

 Part of what I want to say in that article is outline what 
 needs to be done to get us from "Here" to "There".


I think making it a call to arms for the embedded sector would be 
good. To do this will probably require providing some evidence 
that early adoption will pay off.
Jul 14 2014
prev sibling next sibling parent reply "Chris" <wendlec tcd.ie> writes:
On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via 
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers I 
 can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


 From the link above:

"It’s a common but false belief that reference counting (using 
shared pointers in particular) is better than garbage collection. 
There is actual research* showing that the two approaches are 
just two sides of the same coin. You should realize that deleting 
a shared pointer may lead to an arbitrary long pause in program 
execution, with similar performance characteristics as a garbage 
sweep. It’s not only because every serious reference counting 
algorithm must be able to deal with cycles, but also because 
every time a reference count goes to zero on a piece of data a 
whole graph of pointers reachable from that object has to be 
traversed. A data structure built with shared pointers might take 
a long time to delete and, except for simple cases, you’ll never 
know which shared pointer will go out of scope last and trigger 
it."

* http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf
Jul 15 2014
next sibling parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 15.07.2014 22:03, schrieb Chris:

 On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via Digitalmars-d
 wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers I can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


  From the link above:

 "It’s a common but false belief that reference counting (using shared
 pointers in particular) is better than garbage collection. There is
 actual research* showing that the two approaches are just two sides of
 the same coin. You should realize that deleting a shared pointer may
 lead to an arbitrary long pause in program execution, with similar
 performance characteristics as a garbage sweep. It’s not only because
 every serious reference counting algorithm must be able to deal with
 cycles, but also because every time a reference count goes to zero on a
 piece of data a whole graph of pointers reachable from that object has
 to be traversed. A data structure built with shared pointers might take
 a long time to delete and, except for simple cases, you’ll never know
 which shared pointer will go out of scope last and trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


+1 :)
Jul 15 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Jul 15, 2014 at 10:16:33PM +0200, Paulo Pinto via Digitalmars-d wrote:

 Am 15.07.2014 22:03, schrieb Chris:

On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via Digitalmars-d
wrote:

On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via
Digitalmars-d wrote:
[...]

Any other good blog posts / social media comments / pointers I can
digest and use?


This one came to mind:

http://bartoszmilewski.com/2013/09/19/edward-chands/


 From the link above:

"It’s a common but false belief that reference counting (using shared
pointers in particular) is better than garbage collection. There is
actual research* showing that the two approaches are just two sides
of the same coin. You should realize that deleting a shared pointer
may lead to an arbitrary long pause in program execution, with
similar performance characteristics as a garbage sweep. It’s not only
because every serious reference counting algorithm must be able to
deal with cycles, but also because every time a reference count goes
to zero on a piece of data a whole graph of pointers reachable from
that object has to be traversed. A data structure built with shared
pointers might take a long time to delete and, except for simple
cases, you’ll never know which shared pointer will go out of scope
last and trigger it."

* http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 
 +1 :)


+100. :-)

The only way to *really* guarantee 100% predictable memory reclamation
is to write your own. Except that we all know how scalable and bug-free
that is. Not to mention, when you need to deallocate a large complex
data structure, *somebody* has to do the work -- either you do it
yourself, or the reference counting implementation, or the GC. No matter
how you cut it, it's work that has to be done, and you have to pay for
it somehow; the cost isn't going to magically disappear just because you
use reference counting (or whatever other scheme you dream up).


T

-- 
Klein bottle for rent ... inquire within. -- Stephen Mulraney
Jul 15 2014
next sibling parent reply "Araq" <rumpf_a web.de> writes:
 The only way to *really* guarantee 100% predictable memory 
 reclamation
 is to write your own. Except that we all know how scalable and 
 bug-free
 that is. Not to mention, when you need to deallocate a large 
 complex
 data structure, *somebody* has to do the work -- either you do 
 it
 yourself, or the reference counting implementation, or the GC. 
 No matter
 how you cut it, it's work that has to be done, and you have to 
 pay for
 it somehow; the cost isn't going to magically disappear just 
 because you
 use reference counting (or whatever other scheme you dream up).


Actually it completely disappears in a copying collector since 
only the live data is copied over ...
Jul 15 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Jul 15, 2014 at 09:03:36PM +0000, Araq via Digitalmars-d wrote:

The only way to *really* guarantee 100% predictable memory
reclamation is to write your own. Except that we all know how
scalable and bug-free that is. Not to mention, when you need to
deallocate a large complex data structure, *somebody* has to do the
work -- either you do it yourself, or the reference counting
implementation, or the GC. No matter how you cut it, it's work that
has to be done, and you have to pay for it somehow; the cost isn't
going to magically disappear just because you use reference counting
(or whatever other scheme you dream up).


 
 Actually it completely disappears in a copying collector since only
 the live data is copied over ...


Nope, you pay for it during the copy. Read the linked paper, it explains
the duality of tracing and reference-counting. Whether you trace the
references from live objects or from dead objects, the overall
computation is equivalent, and the cost is effectively the same. Once
you've applied the usual optimizations, it's just a matter of time/space
tradeoffs.


T

-- 
Creativity is not an excuse for sloppiness.
Jul 15 2014
parent reply "Araq" <rumpf_a web.de> writes:
On Tuesday, 15 July 2014 at 21:11:24 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Tue, Jul 15, 2014 at 09:03:36PM +0000, Araq via 
 Digitalmars-d wrote:

The only way to *really* guarantee 100% predictable memory
reclamation is to write your own. Except that we all know how
scalable and bug-free that is. Not to mention, when you need 
to
deallocate a large complex data structure, *somebody* has to 
do the
work -- either you do it yourself, or the reference counting
implementation, or the GC. No matter how you cut it, it's 
work that
has to be done, and you have to pay for it somehow; the cost 
isn't
going to magically disappear just because you use reference 
counting
(or whatever other scheme you dream up).


 
 Actually it completely disappears in a copying collector since 
 only
 the live data is copied over ...


 Nope, you pay for it during the copy. Read the linked paper, it 
 explains
 the duality of tracing and reference-counting. Whether you 
 trace the
 references from live objects or from dead objects, the overall
 computation is equivalent, and the cost is effectively the 
 same. Once
 you've applied the usual optimizations, it's just a matter of 
 time/space
 tradeoffs.


This is wrong on so many levels... Oh well, I don't care. Believe 
what you want.
Jul 15 2014
parent "Chris" <wendlec tcd.ie> writes:
On Tuesday, 15 July 2014 at 23:02:19 UTC, Araq wrote:

 On Tuesday, 15 July 2014 at 21:11:24 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 On Tue, Jul 15, 2014 at 09:03:36PM +0000, Araq via 
 Digitalmars-d wrote:

The only way to *really* guarantee 100% predictable memory
reclamation is to write your own. Except that we all know how
scalable and bug-free that is. Not to mention, when you need 
to
deallocate a large complex data structure, *somebody* has to 
do the
work -- either you do it yourself, or the reference counting
implementation, or the GC. No matter how you cut it, it's 
work that
has to be done, and you have to pay for it somehow; the cost 
isn't
going to magically disappear just because you use reference 
counting
(or whatever other scheme you dream up).


 
 Actually it completely disappears in a copying collector 
 since only
 the live data is copied over ...


 Nope, you pay for it during the copy. Read the linked paper, 
 it explains
 the duality of tracing and reference-counting. Whether you 
 trace the
 references from live objects or from dead objects, the overall
 computation is equivalent, and the cost is effectively the 
 same. Once
 you've applied the usual optimizations, it's just a matter of 
 time/space
 tradeoffs.


 This is wrong on so many levels... Oh well, I don't care. 
 Believe what you want.


Please enlighten us.
Jul 16 2014
prev sibling parent reply "Kagamin" <spam here.lot> writes:
On Tuesday, 15 July 2014 at 20:44:35 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 Not to mention, when you need to deallocate a large complex
 data structure, *somebody* has to do the work -- either you do 
 it
 yourself, or the reference counting implementation, or the GC.


The first and the last options are still prominent.
Jul 16 2014
parent reply "Araq" <rumpf_a web.de> writes:
On Wednesday, 16 July 2014 at 16:57:18 UTC, Kagamin wrote:

 On Tuesday, 15 July 2014 at 20:44:35 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 Not to mention, when you need to deallocate a large complex
 data structure, *somebody* has to do the work -- either you do 
 it
 yourself, or the reference counting implementation, or the GC.


 The first and the last options are still prominent.


A copying GC copies the live data, deallocation of a large 
complex data structure is free in this scenario. Same if you use 
a manually managed memory region for the data structure and then 
deallocate the region via some batch operation. But hey, this 
simple fact must be wrong because some guy read a single paper 
about GCs that doesn't even cover this point.
Jul 16 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 16, 2014 at 06:11:54PM +0000, Araq via Digitalmars-d wrote:

 On Wednesday, 16 July 2014 at 16:57:18 UTC, Kagamin wrote:

On Tuesday, 15 July 2014 at 20:44:35 UTC, H. S. Teoh via Digitalmars-d
wrote:

Not to mention, when you need to deallocate a large complex data
structure, *somebody* has to do the work -- either you do it
yourself, or the reference counting implementation, or the GC.


The first and the last options are still prominent.


 
 A copying GC copies the live data, deallocation of a large complex
 data structure is free in this scenario. Same if you use a manually
 managed memory region for the data structure and then deallocate the
 region via some batch operation. But hey, this simple fact must be
 wrong because some guy read a single paper about GCs that doesn't even
 cover this point.


Have you even read the paper? What you just said is exactly what the
paper is describing. There are two ends of the spectrum of memory
reclamation algorithms, at one end, you're tracing "matter" (live
objects), and the other end you're tracing "antimatter" (dead objects).
They are just duals of each other, and optimized GC/RC algorithms tend
to approach the middle ground, with time/memory tradeoffs as an
adjustable parameter.


T

-- 
People tell me that I'm paranoid, but they're just out to get me.
Jul 16 2014
next sibling parent reply "Araq" <rumpf_a web.de> writes:
On Wednesday, 16 July 2014 at 18:24:11 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Wed, Jul 16, 2014 at 06:11:54PM +0000, Araq via 
 Digitalmars-d wrote:

 On Wednesday, 16 July 2014 at 16:57:18 UTC, Kagamin wrote:

On Tuesday, 15 July 2014 at 20:44:35 UTC, H. S. Teoh via 
Digitalmars-d
wrote:

Not to mention, when you need to deallocate a large complex 
data
structure, *somebody* has to do the work -- either you do it
yourself, or the reference counting implementation, or the 
GC.


The first and the last options are still prominent.


 
 A copying GC copies the live data, deallocation of a large 
 complex
 data structure is free in this scenario. Same if you use a 
 manually
 managed memory region for the data structure and then 
 deallocate the
 region via some batch operation. But hey, this simple fact 
 must be
 wrong because some guy read a single paper about GCs that 
 doesn't even
 cover this point.


 Have you even read the paper? What you just said is exactly 
 what the
 paper is describing. There are two ends of the spectrum of 
 memory
 reclamation algorithms, at one end, you're tracing "matter" 
 (live
 objects), and the other end you're tracing "antimatter" (dead 
 objects).
 They are just duals of each other, and optimized GC/RC 
 algorithms tend
 to approach the middle ground, with time/memory tradeoffs as an
 adjustable parameter.


The paper focusses on RC vs "tracing". My point is "tracing" vs 
"copying" is another tradeoff. Here is a mark&sweep algorithm:

- Trace live objects.
- For each dead object: Deallocate.

Here is a copying GC:

- Trace and copy live objects.
- There is no deallocation step. The old region is free for 
further usage.
Jul 16 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 16, 2014 at 07:02:15PM +0000, Araq via Digitalmars-d wrote:

 On Wednesday, 16 July 2014 at 18:24:11 UTC, H. S. Teoh via Digitalmars-d
 wrote:

On Wed, Jul 16, 2014 at 06:11:54PM +0000, Araq via Digitalmars-d wrote:

On Wednesday, 16 July 2014 at 16:57:18 UTC, Kagamin wrote:

On Tuesday, 15 July 2014 at 20:44:35 UTC, H. S. Teoh via >Digitalmars-d
wrote:

Not to mention, when you need to deallocate a large complex >>data
structure, *somebody* has to do the work -- either you do it
yourself, or the reference counting implementation, or the >>GC.


The first and the last options are still prominent.


A copying GC copies the live data, deallocation of a large complex
data structure is free in this scenario. Same if you use a manually
managed memory region for the data structure and then deallocate the
region via some batch operation. But hey, this simple fact must be
wrong because some guy read a single paper about GCs that doesn't
even cover this point.


Have you even read the paper? What you just said is exactly what the
paper is describing. There are two ends of the spectrum of memory
reclamation algorithms, at one end, you're tracing "matter" (live
objects), and the other end you're tracing "antimatter" (dead
objects).  They are just duals of each other, and optimized GC/RC
algorithms tend to approach the middle ground, with time/memory
tradeoffs as an adjustable parameter.


 
 The paper focusses on RC vs "tracing". My point is "tracing" vs
 "copying" is another tradeoff. Here is a mark&sweep algorithm:
 
 - Trace live objects.
 - For each dead object: Deallocate.
 
 Here is a copying GC:
 
 - Trace and copy live objects.
 - There is no deallocation step. The old region is free for further usage.


Yes, and my point was that you pay for the (implicit) deallocation by
the need to copy all live objects. The cost of copying is not zero
compared to a non-copying GC. No matter how you do it, the cost has to
be paid *somewhere*. It's just a question of which method will be less
costly based on what your application does.


T

-- 
Too many people have open minds but closed eyes.
Jul 16 2014
parent "Araq" <rumpf_a web.de> writes:
 Yes, and my point was that you pay for the (implicit) 
 deallocation by
 the need to copy all live objects. The cost of copying is not 
 zero
 compared to a non-copying GC. No matter how you do it, the cost 
 has to
 be paid *somewhere*. It's just a question of which method will 
 be less
 costly based on what your application does.


The cost models are vastly different and as you finally note 
application dependent. *The* cost does not have to paid 
*somewhere* because there is not a single cost to begin with.
Jul 16 2014
prev sibling parent reply =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"Araq" <rumpf_a web.de> wrote:

 The paper focusses on RC vs "tracing". My point is "tracing" vs "copying"
 is another tradeoff. Here is a mark&sweep algorithm:
 
 - Trace live objects.
 - For each dead object: Deallocate.
 
 Here is a copying GC:
 
 - Trace and copy live objects.
 - There is no deallocation step. The old region is free for further usage.


It's more like:
- Allocate a new memory region as big as the current one.
- Trace live objects and copy them to the new region.
- Deallocate old region.

Not only are you deallocating in each step, you are also allocating.

This only pays off if you use very small and short-living objects such that
the deallocation of the whole region is much faster than the deallocation
of the (dead) individual objects and you only have to copy a small fraction
of the objects.

Also be aware that you can only use half of the available (virtual) memory.
And if physical memory is low, you have constant swapping due to the
copying.

Tobi
Jul 17 2014
parent reply "Araq" <rumpf_a web.de> writes:
On Thursday, 17 July 2014 at 15:38:44 UTC, Tobias Müller wrote:

 "Araq" <rumpf_a web.de> wrote:

 The paper focusses on RC vs "tracing". My point is "tracing" 
 vs "copying"
 is another tradeoff. Here is a mark&sweep algorithm:
 
 - Trace live objects.
 - For each dead object: Deallocate.
 
 Here is a copying GC:
 
 - Trace and copy live objects.
 - There is no deallocation step. The old region is free for 
 further usage.


 It's more like: ...


Your description is still naive and as such your post is more 
misleading than helpful. My description is naive to get the basic 
point across, I have no idea what the point of your post is.
Jul 17 2014
parent reply =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"Araq" <rumpf_a web.de> wrote:

 On Thursday, 17 July 2014 at 15:38:44 UTC, Tobias Müller wrote:

 "Araq" <rumpf_a web.de> wrote:

 The paper focusses on RC vs "tracing". My point is "tracing" >> vs "copying"
 is another tradeoff. Here is a mark&sweep algorithm:

 - Trace live objects.




 - For each dead object: Deallocate.

 Here is a copying GC:
 - Trace and copy live objects.




 - There is no deallocation step. The old region is free for >> further usage.


 
 It's more like: ...


 
 Your description is still naive and as such your post is more misleading
 than helpful. My description is naive to get the basic point across, I
 have no idea what the point of your post is.


Since you snipped everything, my points are:

1. "There is no deallocation" is just plain wrong.

2. The cost of complex object graphs vs simple objects lies in the mark,
not in the sweep. mark-copy has no advantage vs. mark sweep in that
respect. Same cost applies for RC.

3. The real difference between mark-sweep and mark-copy lies in number of
dead objects vs number of live objects.

4. Your comparison with a region allocator is moot, because _really_ just
bulk-deallocate, without a mark step.

Tobi
Jul 17 2014
parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 17 July 2014 at 19:19:40 UTC, Tobias Müller wrote:

 1. "There is no deallocation" is just plain wrong.


It is actually fairly common to not deallocate and reuse the pool 
for later allocations.

Also, it doesn't matter as the complexity of deallocation doesn't 
depend on the size of the block deallocated.

In the case of a copying collector, the deallocation won't even 
show up in benchmark as tracing and copying will dominate CPU 
time by a LOT !
Jul 17 2014
parent =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"deadalnix" <deadalnix gmail.com> wrote:

 On Thursday, 17 July 2014 at 19:19:40 UTC, Tobias Müller wrote:

 1. "There is no deallocation" is just plain wrong.
 


 
 It is actually fairly common to not deallocate and reuse the pool for later
allocations.


If by deallocation you mean returning the memory to the OS, then yes.
But the same holds for 'free' in C.

Marking a memory region as free for reuse _is_ deallocation IMO. I guess
it's a terminology thing.


 Also, it doesn't matter as the complexity of deallocation doesn't depend
 on the size of the block deallocated.

 In the case of a copying collector, the deallocation won't even show up
 in benchmark as tracing and copying will dominate CPU time by a LOT !


Yes that's what I said, bulk deallocation is cheap. Got snipped away
unfortunately.

Tobi
Jul 17 2014
prev sibling parent reply "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 16 July 2014 at 18:24:11 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 They are just duals of each other, and optimized GC/RC 
 algorithms tend
 to approach the middle ground, with time/memory tradeoffs as an
 adjustable parameter.


Not really. If you are performance conscious you structure your 
data and algorithms to match the tools you are using. Therefore 
performance oriented RC outperforms GC. You can have a lot more 
special-casing with RC in a realistic implementation. Few 
application authors consider rewriting the GC, but doing your own 
minimal overhead specialized RC is quite common…
Jul 16 2014
parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Wed, Jul 16, 2014 at 07:14:25PM +0000, via Digitalmars-d wrote:

 On Wednesday, 16 July 2014 at 18:24:11 UTC, H. S. Teoh via Digitalmars-d
 wrote:

They are just duals of each other, and optimized GC/RC algorithms
tend to approach the middle ground, with time/memory tradeoffs as an
adjustable parameter.


 
 Not really. If you are performance conscious you structure your data
 and algorithms to match the tools you are using.


[...]

Well, yes, so what we're saying here is that the set of general memory
management problems is hard to solve, but if we can rewrite our program
so that its particular instance of memory management lies in an "easy"
subset for which there are known superior solutions, then performance
will be better.  This doesn't invalidate the fact that the *general*
memory management problem is hard.  It's like saying that sorting, in
general, cannot perform better than O(n log n), but the subset of
sorting problems where the data is already almost sorted, there are
algorithms that will perform better than O(n log n). The only problem
is, they will suck when given data outside of that "easy" subset.

Is it possible to transform given any arbitrary program into an
equivalent program whose particular instance of memory management lies
in the "easy" subset? I don't know. But even if it's possible, you still
have to "pay" for the cost of memory management -- this time by paying
with program structure rather than runtime performance: you will have to
constrain the way you write the program, otherwise you will have to deal
with a memory management problem instance outside the "easy" set.

This leads back to my idea of some kind of structural matching between
memory lifetime and program structure. If you can somehow structure your
program such that it corresponds with the structure of its memory usage
patterns, then you're in an "easy" subset of the set of general memory
management problems. It doesn't change the fact, though, that when there
is a mismatch, you're back in the "hard" set, and the problem is costly
to solve no matter what algorithm you use.


T

-- 
MACINTOSH: Most Applications Crash, If Not, The Operating System Hangs
Jul 16 2014
parent "Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= writes:
On Wednesday, 16 July 2014 at 19:51:39 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 It's like saying that sorting, in
 general, cannot perform better than O(n log n), but the subset 
 of
 sorting problems where the data is already almost sorted, there 
 are
 algorithms that will perform better than O(n log n). The only 
 problem
 is, they will suck when given data outside of that "easy" 
 subset.


Nah, it's like picking keys that allows you to use a trie.


 management problems. It doesn't change the fact, though, that 
 when there
 is a mismatch, you're back in the "hard" set, and the problem 
 is costly
 to solve no matter what algorithm you use.


GC is always easier, sure.
Jul 16 2014
prev sibling next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

 From the link above:

 "It’s a common but false belief that reference counting (using 
 shared pointers in particular) is better than garbage 
 collection. There is actual research* showing that the two 
 approaches are just two sides of the same coin. You should 
 realize that deleting a shared pointer may lead to an arbitrary 
 long pause in program execution, with similar performance 
 characteristics as a garbage sweep. It’s not only because every 
 serious reference counting algorithm must be able to deal with 
 cycles, but also because every time a reference count goes to 
 zero on a piece of data a whole graph of pointers reachable 
 from that object has to be traversed. A data structure built 
 with shared pointers might take a long time to delete and, 
 except for simple cases, you’ll never know which shared pointer 
 will go out of scope last and trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


Yes but D's GC is crap. I actually wanted to a a quick hack in it
recently to avoid going through the lock every time for every
operation, but ended up giving up. Also the API between the GC
and the compiler need to be reworked.

Finally someone posted here a research paper from ibm if memory
serve. It show that ref counting is a win for single threaded
code, but is a net loose when sharing across thread, because of
the synchronization overhead (especially the hammering of the
cache line containing the reference count).
Jul 15 2014
parent reply Paulo Pinto <pjmlp progtools.org> writes:
Am 15.07.2014 22:58, schrieb deadalnix:

 On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

 From the link above:

 "It’s a common but false belief that reference counting (using shared
 pointers in particular) is better than garbage collection. There is
 actual research* showing that the two approaches are just two sides of
 the same coin. You should realize that deleting a shared pointer may
 lead to an arbitrary long pause in program execution, with similar
 performance characteristics as a garbage sweep. It’s not only because
 every serious reference counting algorithm must be able to deal with
 cycles, but also because every time a reference count goes to zero on
 a piece of data a whole graph of pointers reachable from that object
 has to be traversed. A data structure built with shared pointers might
 take a long time to delete and, except for simple cases, you’ll never
 know which shared pointer will go out of scope last and trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 Yes but D's GC is crap. I actually wanted to a a quick hack in it
 recently to avoid going through the lock every time for every
 operation, but ended up giving up. Also the API between the GC
 and the compiler need to be reworked.


Which boils down to having to improve the GC implementation, not that GC 
in general is bad.



 Finally someone posted here a research paper from ibm if memory
 serve. It show that ref counting is a win for single threaded
 code, but is a net loose when sharing across thread, because of
 the synchronization overhead (especially the hammering of the
 cache line containing the reference count).



Which given the fact that most computers nowadays, even embedded ones, 
are becoming at very least dual core, is quite important fact.

--
Paulo
Jul 15 2014
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 15 July 2014 at 21:13:22 UTC, Paulo Pinto wrote:

 Am 15.07.2014 22:58, schrieb deadalnix:

 On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

 From the link above:

 "It’s a common but false belief that reference counting 
 (using shared
 pointers in particular) is better than garbage collection. 
 There is
 actual research* showing that the two approaches are just two 
 sides of
 the same coin. You should realize that deleting a shared 
 pointer may
 lead to an arbitrary long pause in program execution, with 
 similar
 performance characteristics as a garbage sweep. It’s not only 
 because
 every serious reference counting algorithm must be able to 
 deal with
 cycles, but also because every time a reference count goes to 
 zero on
 a piece of data a whole graph of pointers reachable from that 
 object
 has to be traversed. A data structure built with shared 
 pointers might
 take a long time to delete and, except for simple cases, 
 you’ll never
 know which shared pointer will go out of scope last and 
 trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 Yes but D's GC is crap. I actually wanted to a a quick hack in 
 it
 recently to avoid going through the lock every time for every
 operation, but ended up giving up. Also the API between the GC
 and the compiler need to be reworked.


 Which boils down to having to improve the GC implementation, 
 not that GC in general is bad.


Perhaps we should stop calling it a "Garbage Collector" and call 
it "Fully Lazy Snapshot Reference Counting" instead :)
Jul 15 2014
parent "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Jul 15, 2014 at 09:19:27PM +0000, John Colvin via Digitalmars-d wrote:

 On Tuesday, 15 July 2014 at 21:13:22 UTC, Paulo Pinto wrote:

Am 15.07.2014 22:58, schrieb deadalnix:

On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

From the link above:

"It’s a common but false belief that reference counting (using
shared pointers in particular) is better than garbage collection.
There is actual research* showing that the two approaches are just
two sides of the same coin. You should realize that deleting a
shared pointer may lead to an arbitrary long pause in program
execution, with similar performance characteristics as a garbage
sweep. It’s not only because every serious reference counting
algorithm must be able to deal with cycles, but also because every
time a reference count goes to zero on a piece of data a whole
graph of pointers reachable from that object has to be traversed. A
data structure built with shared pointers might take a long time to
delete and, except for simple cases, you’ll never know which shared
pointer will go out of scope last and trigger it."

* http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


Yes but D's GC is crap. I actually wanted to a a quick hack in it
recently to avoid going through the lock every time for every
operation, but ended up giving up. Also the API between the GC and
the compiler need to be reworked.


Which boils down to having to improve the GC implementation, not that
GC in general is bad.


 
 Perhaps we should stop calling it a "Garbage Collector" and call it
 "Fully Lazy Snapshot Reference Counting" instead :)


+1. :D

It does make one wonder, though, and the paper does briefly discuss this
topic at the end, if there exists a "sweet spot" in the design space
where you can get the benefits of both approaches and minimize their
weakness of each. Or whether there exists a kind of "universal"
collection algorithm with knobs that, if tuned to one extreme, behaves
like a GC, and if tuned to the other extreme, behaves like an RC. In an
ideal world, if such a universal implementation exists, then you could
in theory fine-tune it precisely to your specific application's needs.


T

-- 
Being able to learn is a great learning; being able to unlearn is a greater
learning.
Jul 15 2014
prev sibling parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

 On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via 
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers 
 I can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


 From the link above:

 "It’s a common but false belief that reference counting (using 
 shared pointers in particular) is better than garbage 
 collection. There is actual research* showing that the two 
 approaches are just two sides of the same coin. You should 
 realize that deleting a shared pointer may lead to an arbitrary 
 long pause in program execution, with similar performance 
 characteristics as a garbage sweep. It’s not only because every 
 serious reference counting algorithm must be able to deal with 
 cycles, but also because every time a reference count goes to 
 zero on a piece of data a whole graph of pointers reachable 
 from that object has to be traversed. A data structure built 
 with shared pointers might take a long time to delete and, 
 except for simple cases, you’ll never know which shared pointer 
 will go out of scope last and trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


I've been wondering about this. Could the following argument be 
true?

Situations where automatic memory management are necessary are, 
by definition, the situations where one cannot easily reason 
about where memory freeing can occur. Therefore, no automatic 
memory management system can be considered practically 
predictable, unless you didn't* need it in the first place.

*strictly speaking
Jul 15 2014
next sibling parent reply "H. S. Teoh via Digitalmars-d" <digitalmars-d puremagic.com> writes:
On Tue, Jul 15, 2014 at 09:11:23PM +0000, John Colvin via Digitalmars-d wrote:

 On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via Digitalmars-d
wrote:




[...]

http://bartoszmilewski.com/2013/09/19/edward-chands/


From the link above:

"It’s a common but false belief that reference counting (using shared
pointers in particular) is better than garbage collection. There is
actual research* showing that the two approaches are just two sides
of the same coin. You should realize that deleting a shared pointer
may lead to an arbitrary long pause in program execution, with
similar performance characteristics as a garbage sweep. It’s not only
because every serious reference counting algorithm must be able to
deal with cycles, but also because every time a reference count goes
to zero on a piece of data a whole graph of pointers reachable from
that object has to be traversed. A data structure built with shared
pointers might take a long time to delete and, except for simple
cases, you’ll never know which shared pointer will go out of scope
last and trigger it."

* http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 
 I've been wondering about this. Could the following argument be true?
 
 Situations where automatic memory management are necessary are, by
 definition, the situations where one cannot easily reason about where
 memory freeing can occur. Therefore, no automatic memory management
 system can be considered practically predictable, unless you didn't*
 need it in the first place.
 
 *strictly speaking


Sounds about right. Which is why you have advice like preallocating
everything before a game level (i.e., eliminate the need for memory
management during that level), or minimizing GC load by avoiding
frequent allocations of small objects (i.e., reduce the amount work that
needs to be done by the memory management system), keeping things on
stack rather than heap if possible (maximize trivial instances of memory
management and minimize non-trivial instances -- can also be viewed as:
if you structure your program to correspond with the structure of your
memory usage, then memory management has a 1-to-1 correspondence with
the control structure of the program itself so it doesn't need to be
done as a separate task).

Hmm.

Now that I come to think of it, it seems that the complexity of memory
management arises due to structural mismatch between memory usage and
program structure, that is, memory acquisition and release do not
correspond 1-to-1 with the call structure of the program. If your
program can be structured such that it matches your memory usage pattern
exactly, then everything can be stack-allocated, and there is no need
for sophisticated memory management schemes. It's when the lifetimes of
your memory allocations differ from the lifetime of program units (i.e.,
functions), that extra work, in the form of memory management
algorithms, is needed to compensate for the "impedance mismatch", so to
speak.

Which suggests the rather interesting idea that perhaps some innovative
way of structuring the parts of your program such that each corresponds
with the structure (and lifetime) of a particular piece of data, then it
should minimize, or maybe completely eliminate, the need for complex
memory management! (I have no idea if such a thing exists, though. But
it's an interesting thought.)

For example, if your code creates a string in one section of a function,
which is subsequently consumed by the second section, then by the time
the function exits the string is not needed anymore, so you could
ostensibly just allocate it on the stack instead of the heap. It's when
you need to return the string that things become complicated: because
now there's a mismatch between the lifetime of your function and the
lifetime of the string. So now you need some way of keeping track of the
lifetime of the string, and dispose of it when it's no longer needed.
But if your program can be structured such that the allocators of memory
and the consumers of memory are always matched together 1-to-1, then you
can always allocate only on the stack and no memory will need to persist
past the function in which it is allocated.  Not sure when such a
structuring is possible, or whether it is at all possible to structure
arbitrary programs this way, though. Definitely not possible in a
stack-only paradigm, I think; but maybe possible (or we can go a little
farther) with concurrent paradigms?


T

-- 
"A one-question geek test. If you get the joke, you're a geek: Seen on a
California license plate on a VW Beetle: 'FEATURE'..." -- Joshua D. Wachs -
Natural Intelligence, Inc.
Jul 15 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Tuesday, 15 July 2014 at 21:57:30 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Tue, Jul 15, 2014 at 09:11:23PM +0000, John Colvin via 
 Digitalmars-d wrote:

 On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
Digitalmars-d
wrote:




 [...]

http://bartoszmilewski.com/2013/09/19/edward-chands/


From the link above:

"It’s a common but false belief that reference counting 
(using shared
pointers in particular) is better than garbage collection. 
There is
actual research* showing that the two approaches are just two 
sides
of the same coin. You should realize that deleting a shared 
pointer
may lead to an arbitrary long pause in program execution, with
similar performance characteristics as a garbage sweep. It’s 
not only
because every serious reference counting algorithm must be 
able to
deal with cycles, but also because every time a reference 
count goes
to zero on a piece of data a whole graph of pointers 
reachable from
that object has to be traversed. A data structure built with 
shared
pointers might take a long time to delete and, except for 
simple
cases, you’ll never know which shared pointer will go out of 
scope
last and trigger it."

* http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 
 I've been wondering about this. Could the following argument 
 be true?
 
 Situations where automatic memory management are necessary 
 are, by
 definition, the situations where one cannot easily reason 
 about where
 memory freeing can occur. Therefore, no automatic memory 
 management
 system can be considered practically predictable, unless you 
 didn't*
 need it in the first place.
 
 *strictly speaking


 Sounds about right. Which is why you have advice like 
 preallocating
 everything before a game level (i.e., eliminate the need for 
 memory
 management during that level), or minimizing GC load by avoiding
 frequent allocations of small objects (i.e., reduce the amount 
 work that
 needs to be done by the memory management system), keeping 
 things on
 stack rather than heap if possible (maximize trivial instances 
 of memory
 management and minimize non-trivial instances -- can also be 
 viewed as:
 if you structure your program to correspond with the structure 
 of your
 memory usage, then memory management has a 1-to-1 
 correspondence with
 the control structure of the program itself so it doesn't need 
 to be
 done as a separate task).

 Hmm.

 Now that I come to think of it, it seems that the complexity of 
 memory
 management arises due to structural mismatch between memory 
 usage and
 program structure, that is, memory acquisition and release do 
 not
 correspond 1-to-1 with the call structure of the program. If 
 your
 program can be structured such that it matches your memory 
 usage pattern
 exactly, then everything can be stack-allocated, and there is 
 no need
 for sophisticated memory management schemes. It's when the 
 lifetimes of
 your memory allocations differ from the lifetime of program 
 units (i.e.,
 functions), that extra work, in the form of memory management
 algorithms, is needed to compensate for the "impedance 
 mismatch", so to
 speak.

 Which suggests the rather interesting idea that perhaps some 
 innovative
 way of structuring the parts of your program such that each 
 corresponds
 with the structure (and lifetime) of a particular piece of 
 data, then it
 should minimize, or maybe completely eliminate, the need for 
 complex
 memory management! (I have no idea if such a thing exists, 
 though. But
 it's an interesting thought.)

 For example, if your code creates a string in one section of a 
 function,
 which is subsequently consumed by the second section, then by 
 the time
 the function exits the string is not needed anymore, so you 
 could
 ostensibly just allocate it on the stack instead of the heap. 
 It's when
 you need to return the string that things become complicated: 
 because
 now there's a mismatch between the lifetime of your function 
 and the
 lifetime of the string. So now you need some way of keeping 
 track of the
 lifetime of the string, and dispose of it when it's no longer 
 needed.
 But if your program can be structured such that the allocators 
 of memory
 and the consumers of memory are always matched together 1-to-1, 
 then you
 can always allocate only on the stack and no memory will need 
 to persist
 past the function in which it is allocated.  Not sure when such 
 a
 structuring is possible, or whether it is at all possible to 
 structure
 arbitrary programs this way, though. Definitely not possible in 
 a
 stack-only paradigm, I think; but maybe possible (or we can go 
 a little
 farther) with concurrent paradigms?


 T


This is a rather interesting thought and boils down to the usual 
conflict of machine logic not matching program / real world 
logic. You mentioned a while ago that one of your teachers said 
that programs were often overly complicated, bug and error prone, 
because there was a mismatch of data structure and program logic 
(which D's ranges try to fix). I find this way of thinking 
interesting, because it shows that we are often impeded by the 
attitude that we try to please the machine and adapt reality to 
how the machine works, rather than trying to adapt the machine to 
reality.*

The problem you described above is omnipresent. Objective-C has 
autorelease, in plain C programs one has to be careful where and 
when to free what, etc. I was wondering after reading your post, 
whether it would be possible to have an automatic memory 
management system that could help to create a 1-1-match, or if 
even the compiler + language features could help to _easily_ 
achieve that. Maybe the copy approach mentioned earlier points 
into that direction.

Anyway, I think you are on to something here. Maybe the solution 
is not a hyper-super-waterproof GC algorithm (that will never be 
achieved, I guess, in any language), but eliminating the need to 
collect the garbage as much as possible.


* I've found that D gives me a lot of freedom in modelling 
reality in a way the machine understands. But I've said that 
already.
Jul 16 2014
parent "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> writes:
On Wednesday, 16 July 2014 at 10:13:44 UTC, Chris wrote:

 The problem you described above is omnipresent. Objective-C has 
 autorelease, in plain C programs one has to be careful where 
 and when to free what, etc. I was wondering after reading your 
 post, whether it would be possible to have an automatic memory 
 management system that could help to create a 1-1-match, or if 
 even the compiler + language features could help to _easily_ 
 achieve that.


Of course. Allocating something on the stack and passing around 
pointers to it is safe iff you can guarantee that all the 
pointers to it are gone once you leave the stack frame (actually 
the variables scope). This is what is currently being discussed 
in the parallel thread(s) about borrowing with `scope`.


 Maybe the copy approach mentioned earlier points into that 
 direction.

 Anyway, I think you are on to something here. Maybe the 
 solution is not a hyper-super-waterproof GC algorithm (that 
 will never be achieved, I guess, in any language), but 
 eliminating the need to collect the garbage as much as possible.


+1

Borrowing makes it possible to do this safely, and in addition, 
enables crucial performance optimizations for reference counting, 
when it is needed.
Jul 16 2014
prev sibling parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
On Tuesday, 15 July 2014 at 21:11:24 UTC, John Colvin wrote:

 On Tuesday, 15 July 2014 at 20:03:15 UTC, Chris wrote:

 On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
 Digitalmars-d wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via 
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers 
 I can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


 From the link above:

 "It’s a common but false belief that reference counting (using 
 shared pointers in particular) is better than garbage 
 collection. There is actual research* showing that the two 
 approaches are just two sides of the same coin. You should 
 realize that deleting a shared pointer may lead to an 
 arbitrary long pause in program execution, with similar 
 performance characteristics as a garbage sweep. It’s not only 
 because every serious reference counting algorithm must be 
 able to deal with cycles, but also because every time a 
 reference count goes to zero on a piece of data a whole graph 
 of pointers reachable from that object has to be traversed. A 
 data structure built with shared pointers might take a long 
 time to delete and, except for simple cases, you’ll never know 
 which shared pointer will go out of scope last and trigger it."

 * http://www.cs.virginia.edu/~cs415/reading/bacon-garbage.pdf


 I've been wondering about this. Could the following argument be 
 true?

 Situations where automatic memory management are necessary are, 
 by definition, the situations where one cannot easily reason 
 about where memory freeing can occur. Therefore, no automatic 
 memory management system can be considered practically 
 predictable, unless you didn't* need it in the first place.

 *strictly speaking


Which happens all the time in any codebase written by more than 
one developer.

Developer attrition and third party libraries help to the entropy 
of memory leak bugs.

Personally, I don't believe anyone is able to reason properly 
about manual memory management, unless they wrote 100% of their 
code, and don't work in more than one codebase.

--
Paulo
Jul 15 2014
parent reply =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"Paulo Pinto" <pjmlp progtools.org> wrote:

 On Tuesday, 15 July 2014 at 21:11:24 UTC, John Colvin wrote:

 I've been wondering about this. Could the following argument be > true?
 
 Situations where automatic memory management are necessary are,
 by definition, the situations where one cannot easily reason
 about where memory freeing can occur.




My experience is completely different:

1. Shared ownership is rarely necessary.
2. Where is is necessary, deallocation often happens at the same (or a few
similar) _code_ location, just the actual _object_ that triggers
deallocation is different at runtime.


 Therefore, no automatic
 memory management system can be considered practically
 predictable, unless you didn't* need it in the first place.
 
 *strictly speaking


 
 Which happens all the time in any codebase written by more than one developer.
 
 Developer attrition and third party libraries help to the entropy of memory
leak bugs.
 
 Personally, I don't believe anyone is able to reason properly about
 manual memory management, unless they wrote 100% of their code, and don't
 work in more than one codebase.


IMO manual memory management is quite easy. In 99% of all cases ownership
is clear to trivial.

But maybe this is just because I usually write C++ and ownership is crucial
in C++. Maybe it's more difficult if your accustomed to a GC.

Tobi
Jul 17 2014
next sibling parent reply "deadalnix" <deadalnix gmail.com> writes:
On Thursday, 17 July 2014 at 19:58:32 UTC, Tobias Müller wrote:

 IMO manual memory management is quite easy. In 99% of all cases 
 ownership
 is clear to trivial.

 But maybe this is just because I usually write C++ and 
 ownership is crucial
 in C++. Maybe it's more difficult if your accustomed to a GC.

 Tobi


Except that you and I both do a shitton of stupid mistakes when
coding all the time. And if you think you don't, you probably do
a lot more than I (as being blind to one's own weakness usually
don't help improving).

As a result, even when hat needs to be done is obvious (which it
is quite often when it come to memory allocations) mistake will
creep in.

Memory allocation mistake are special. They lead to memory leak
(which won't show up in tests, but only after the program is long
running, which lead to long and annoying debuging experience
where you try to analyze the heap and find how the hell you ended
up allocating all of this) or worse, undefined behavior (which
typically will cause crashes in unrelated part of the code,
leading to even more confusing debuging sessions).

As such, automatic memory management has value in itself. It is a
safe default. I understand that manual memory management is often
needed, and that is why we need way to escape that and manually
manage.
Jul 17 2014
parent =?UTF-8?Q?Tobias=20M=C3=BCller?= <troplin bluewin.ch> writes:
"deadalnix" <deadalnix gmail.com> wrote:

 On Thursday, 17 July 2014 at 19:58:32 UTC, Tobias Müller wrote:

 IMO manual memory management is quite easy. In 99% of all cases > ownership
 is clear to trivial.
 
 But maybe this is just because I usually write C++ and > ownership is crucial
 in C++. Maybe it's more difficult if your accustomed to a GC.
 
 Tobi


 
 Except that you and I both do a shitton of stupid mistakes when
 coding all the time. And if you think you don't, you probably do
 a lot more than I (as being blind to one's own weakness usually
 don't help improving).


I'm well aware of that, otherwise I wouldn't be reading here (and on rust
mailing list).

I don't want to advertise completely manual memory management, don't get me
wrong. But most of the time GC and RC are just overkill because ownership
is clear.


 As a result, even when hat needs to be done is obvious (which it
 is quite often when it come to memory allocations) mistake will
 creep in.


My point was not that errors cannot happen but more that as you say it "hat
needs to be done is obvious" in most cases.


 Memory allocation mistake are special. They lead to memory leak
 (which won't show up in tests,


With deterministic memory management they do. Just scan the heap before and
after the tests (or a bunch of tests).


 but only after tue program is long
 running, which lead to long and annoying debuging experience
 where you try to analyze the heap and find how the hell you ended
 up allocating all of this)


In my experience memory leaks are seldom a problem. They are easily
detected, found and fixed.

More difficult to find are "false" memory leaks, memory that is not needed
but still referenced. But those also happen with automatic memory
management.


 or worse, undefined behavior (which
 typically will cause crashes in unrelated part of the code,
 leading to even more confusing debuging sessions).


Memory corruption is a different beast, those bugs are really hard.

But (again IME) memory management is most often not the cause of those.
Multithreading issues are much more prominent.


 As such, automatic memory management has value in itself. It is a safe default.


That's for sure. But GC/RC are overkill in most cases. Static verification
of ownership is often enough.


 I understand that manual memory management is often
 needed, and that is why we need way to escape that and manually
 manage.


And with a little bit of care/encapsulation also quite safe.

But again, I don't want to advertise manual memory management, but I just
don't agree with the premise that ownership relations are so complex that
deallocation becomes unpredictable.

Tobi
Jul 17 2014
prev sibling parent "Paulo Pinto" <pjmlp progtools.org> writes:
On Thursday, 17 July 2014 at 19:58:32 UTC, Tobias Müller wrote:

 "Paulo Pinto" <pjmlp progtools.org> wrote:

 ...
 Personally, I don't believe anyone is able to reason properly 
 about
 manual memory management, unless they wrote 100% of their 
 code, and don't
 work in more than one codebase.


 IMO manual memory management is quite easy. In 99% of all cases 
 ownership
 is clear to trivial.

 But maybe this is just because I usually write C++ and 
 ownership is crucial
 in C++. Maybe it's more difficult if your accustomed to a GC.

 Tobi


When I started coding BASIC and Z80 were at the rage at home 
computers, so even though I favour RC/GC over manual memory 
management, I do have lots of experience with manual memory 
management.

Specially in enterprise teams with high attrition having more 
than 30 developers, where no one can really tell who owns what. 
And to top that, in one project we even had funny macros 
converting between handles and pointers, our own MMU, so to speak.

So I have my doubts when many developers work in the same code 
base that ownership is trivial.

--
Paulo
Jul 18 2014
prev sibling parent reply "Remo" <remo4d gmail.com> writes:
On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
Digitalmars-d wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via 
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers I 
 can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


This is pretty bad promotion for Haskell.
The Author of this post apparently do not know C++ well enough or 
just ignore this knowledge.
Me as a C++ developer who likes Haskell (and D) after reading 
this I like C++ a bit more and Haskell a bit less.


 It’s a common but false belief that reference counting (using 
 shared pointers in particular) is better than garbage 
 collection.


And then he pointed to a "A Unified Theory of Garbage Collection" 
paper where "reference counting" as a strategy (method) for a 
Garbage Collection will be disused.
This is NOT the same as std::shared_ptr do !

So if some one really hates D then he can write post like this 
about C++ and D.
Jul 16 2014
next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 16 July 2014 at 11:22:45 UTC, Remo wrote:

 And then he pointed to a "A Unified Theory of Garbage 
 Collection" paper where "reference counting" as a strategy 
 (method) for a Garbage Collection will be disused.
 This is NOT the same as std::shared_ptr do !


How std::shared_ptr manages cycles?
Jul 16 2014
parent "Remo" <remo4d gmail.com> writes:
On Wednesday, 16 July 2014 at 17:18:56 UTC, Kagamin wrote:

 On Wednesday, 16 July 2014 at 11:22:45 UTC, Remo wrote:

 And then he pointed to a "A Unified Theory of Garbage 
 Collection" paper where "reference counting" as a strategy 
 (method) for a Garbage Collection will be disused.
 This is NOT the same as std::shared_ptr do !


 How std::shared_ptr manages cycles?


There is std::weak_ptr to handle cycles.
http://en.cppreference.com/w/cpp/memory/weak_ptr
The only question is how often one need to handle cycles ?


Using GC is probably easier as shared_ptr in some case.
But in many other cases only lightweight and deterministic 
unique_ptr is required.

Apparently in Rust one can decide what to use.
Box<T> the same as std::unique_ptr<T> in C++.
Rc<T> is something like std::shared_ptr<T> in C++ but not exactly 
the same.
But there are also Gc<T> (Immutable garbage-collected pointer) 
and Arc<T> without counterparts in C++ at this time.
Jul 16 2014
prev sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 07/16/2014 01:22 PM, Remo wrote:

 On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via Digitalmars-d
 wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers I can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


 This is pretty bad promotion for Haskell.
 The Author of this post apparently do not know C++ well enough
 or just ignore this knowledge.


That would be somewhat curious.
http://www.amazon.com/In-Action-Industrial-Programming-Techniques/dp/0201699486

In any case, personal attacks are irrelevant and do not belong here. 
There are less intrusive ways of formulating this thought.


 Me as a C++ developer who likes Haskell (and D) after reading this I
 like C++ a bit more and Haskell a bit less.
 ...


You are entitled to your opinion, but I'll still point out that taking 
disagreement with some piece of writing as evidence for the (assumed!) 
contrary is not a valid way of building a well-reasoned one.


 It’s a common but false belief that reference counting (using shared
 pointers in particular) is better than garbage collection.


 And then he pointed to a "A Unified Theory of Garbage Collection" paper
 where "reference counting" as a strategy (method) for a Garbage
 Collection will be disused.
 This is NOT the same as std::shared_ptr do !
 ...


This seems a little bit superficial; what would be a _rebuttal_ of his 
fundamental point?
Jul 16 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Wednesday, 16 July 2014 at 21:02:42 UTC, Timon Gehr wrote:

 On 07/16/2014 01:22 PM, Remo wrote:

 On Monday, 14 July 2014 at 23:43:57 UTC, H. S. Teoh via 
 Digitalmars-d
 wrote:

 On Mon, Jul 14, 2014 at 11:22:53PM +0000, John Carter via
 Digitalmars-d wrote:
 [...]

 Any other good blog posts / social media comments / pointers 
 I can
 digest and use?


 This one came to mind:

 http://bartoszmilewski.com/2013/09/19/edward-chands/


 This is pretty bad promotion for Haskell.
 The Author of this post apparently do not know C++ well enough
 or just ignore this knowledge.


 That would be somewhat curious.
 http://www.amazon.com/In-Action-Industrial-Programming-Techniques/dp/0201699486

 In any case, personal attacks are irrelevant and do not belong 
 here. There are less intrusive ways of formulating this thought.


 Me as a C++ developer who likes Haskell (and D) after reading 
 this I
 like C++ a bit more and Haskell a bit less.
 ...


 You are entitled to your opinion, but I'll still point out that 
 taking disagreement with some piece of writing as evidence for 
 the (assumed!) contrary is not a valid way of building a 
 well-reasoned one.


 It’s a common but false belief that reference counting (using 
 shared
 pointers in particular) is better than garbage collection.


 And then he pointed to a "A Unified Theory of Garbage 
 Collection" paper
 where "reference counting" as a strategy (method) for a Garbage
 Collection will be disused.
 This is NOT the same as std::shared_ptr do !
 ...


 This seems a little bit superficial; what would be a _rebuttal_ 
 of his fundamental point?


The funny thing about C++ is that there is a plethora of books 
that teach you how to do it right, which is a sign that there is 
something inherently wrong with the language*. I find that in D 
there aren't many ways to *really* do it wrong, but still you 
have the freedom to try different approaches. D is not overly 
prescriptive, but often keeps you from shooting yourself in the 
foot (or blow away your whole leg**) What can happen in D is that 
you don't use the most efficient way of doing something, it will 
make your program slower, but it won't blow away your legs, arms 
or head, and you can easily fix it later, if needs be.


* And also a sign that there is a huge industry behind it, and, 
of course, people who make a living being C++ gurus don't want 
the language to slowly disappear. C++ reminds me a little bit of 
religion: high priests, mysteries, dogmata ...

** "C makes it easy to shoot yourself in the foot. C++ makes it 
harder, but when you do, it blows away your whole leg."
– Bjarne Stroustrup
Jul 17 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:

 The funny thing about C++ is that there is a plethora of books 
 that teach you how to do it right, which is a sign that there 
 is something inherently wrong with the language*. I find that 
 in D there aren't many ways to *really* do it wrong, but still 
 you have the freedom to try different approaches. D is not 
 overly prescriptive, but often keeps you from shooting yourself 
 in the foot (or blow away your whole leg**) What can happen in 
 D is that you don't use the most efficient way of doing 
 something, it will make your program slower, but it won't blow 
 away your legs, arms or head, and you can easily fix it later, 
 if needs be.


 * And also a sign that there is a huge industry behind it, and, 
 of course, people who make a living being C++ gurus don't want 
 the language to slowly disappear. C++ reminds me a little bit 
 of religion: high priests, mysteries, dogmata ...

 ** "C makes it easy to shoot yourself in the foot. C++ makes it 
 harder, but when you do, it blows away your whole leg."
 – Bjarne Stroustrup


Also, if the trend in C++ is to go back to functional programming 
(don't use classes, inheritance etc.), then what's the point? Why 
not use C instead. It's kinda absurd.
Jul 17 2014
next sibling parent reply "eles" <eles eles.com> writes:
On Thursday, 17 July 2014 at 09:32:15 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:

 The funny thing about C++ is that there is a plethora of books 
 that teach you how to do it right, which is a sign that there 
 is something inherently wrong with the language*. I find that 
 in D there aren't many ways to *really* do it wrong, but still 
 you have the freedom to try different approaches. D is not 
 overly prescriptive, but often keeps you from shooting 
 yourself in the foot (or blow away your whole leg**) What can 
 happen in D is that you don't use the most efficient way of 
 doing something, it will make your program slower, but it 
 won't blow away your legs, arms or head, and you can easily 
 fix it later, if needs be.


 * And also a sign that there is a huge industry behind it, 
 and, of course, people who make a living being C++ gurus don't 
 want the language to slowly disappear. C++ reminds me a little 
 bit of religion: high priests, mysteries, dogmata ...

 ** "C makes it easy to shoot yourself in the foot. C++ makes 
 it harder, but when you do, it blows away your whole leg."
 – Bjarne Stroustrup


 Also, if the trend in C++ is to go back to functional 
 programming (don't use classes, inheritance etc.), then what's 
 the point? Why not use C instead. It's kinda absurd.


For templates, stronger type checking, larger standard library 
and, sometimes, the dreaded try{}.
Jul 17 2014
parent reply "Chris" <wendlec tcd.ie> writes:
On Thursday, 17 July 2014 at 09:52:45 UTC, eles wrote:

 On Thursday, 17 July 2014 at 09:32:15 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:

 The funny thing about C++ is that there is a plethora of 
 books that teach you how to do it right, which is a sign that 
 there is something inherently wrong with the language*. I 
 find that in D there aren't many ways to *really* do it 
 wrong, but still you have the freedom to try different 
 approaches. D is not overly prescriptive, but often keeps you 
 from shooting yourself in the foot (or blow away your whole 
 leg**) What can happen in D is that you don't use the most 
 efficient way of doing something, it will make your program 
 slower, but it won't blow away your legs, arms or head, and 
 you can easily fix it later, if needs be.


 * And also a sign that there is a huge industry behind it, 
 and, of course, people who make a living being C++ gurus 
 don't want the language to slowly disappear. C++ reminds me a 
 little bit of religion: high priests, mysteries, dogmata ...

 ** "C makes it easy to shoot yourself in the foot. C++ makes 
 it harder, but when you do, it blows away your whole leg."
 – Bjarne Stroustrup


 Also, if the trend in C++ is to go back to functional 
 programming (don't use classes, inheritance etc.), then what's 
 the point? Why not use C instead. It's kinda absurd.


 For templates, stronger type checking, larger standard library 
 and, sometimes, the dreaded try{}.


Then why not create C+++ that keeps these useful features and get 
rid of all the dangerous crap people have stopped using / are 
discouraged from using?
Jul 17 2014
parent reply "eles" <eles eles.com> writes:
On Thursday, 17 July 2014 at 11:20:30 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 09:52:45 UTC, eles wrote:

 On Thursday, 17 July 2014 at 09:32:15 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:






 Then why not create C+++ that keeps these useful features and 
 get rid of all the dangerous crap people have stopped using / 
 are discouraged from using?


But this is already working. You have plenty of places (MISRA C++ 
anyone) where such things can be enforced.

Generally, people are not afraid to write C++, they are afraid of 
reading C++.



 Jul 17 2014



  parent reply "Chris" <wendlec tcd.ie>  writes:


On Thursday, 17 July 2014 at 11:29:40 UTC, eles wrote:

 On Thursday, 17 July 2014 at 11:20:30 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 09:52:45 UTC, eles wrote:

 On Thursday, 17 July 2014 at 09:32:15 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:






 Then why not create C+++ that keeps these useful features and 
 get rid of all the dangerous crap people have stopped using / 
 are discouraged from using?


 But this is already working. You have plenty of places (MISRA 
 C++ anyone) where such things can be enforced.


But not in the language as such.


 Generally, people are not afraid to write C++, they are afraid 
 of reading C++.





 Jul 17 2014



  parent  "eles" <eles eles.com>  writes:


On Thursday, 17 July 2014 at 11:48:20 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 11:29:40 UTC, eles wrote:

 On Thursday, 17 July 2014 at 11:20:30 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 09:52:45 UTC, eles wrote:

 On Thursday, 17 July 2014 at 09:32:15 UTC, Chris wrote:

 On Thursday, 17 July 2014 at 08:56:40 UTC, Chris wrote:






 Then why not create C+++ that keeps these useful features and 
 get rid of all the dangerous crap people have stopped using / 
 are discouraged from using?


 But this is already working. You have plenty of places (MISRA 
 C++ anyone) where such things can be enforced.


 But not in the language as such.


You may say that there is no global standard for it. Coding 
styles and norms enforce this and are used where needed.

Simply put it, there is no such huge market for it to standardize 
everything. And that will last until Linus will step back from 
his rant over C++.



 Jul 17 2014



prev sibling  parent  Russel Winder via Digitalmars-d <digitalmars-d puremagic.com>  writes:


On Thu, 2014-07-17 at 09:32 +0000, Chris via Digitalmars-d wrote:
[…]

 Also, if the trend in C++ is to go back to functional programming 
 (don't use classes, inheritance etc.), then what's the point? Why 
 not use C instead. It's kinda absurd.


C cannot do RAII, nor can it do internal DSL (via symbol in expression
definitions). And neither C nor C++ can do functional programming due to
the lack of tail call optimization.

Anyone reverting to C for new applications is either working on an
embedded system with very, very limited resources, or they are
revisionists ;-)

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder



 Jul 17 2014



prev sibling next sibling parent  "Trass3r" <un known.com>  writes:


 Any other good blog posts / social media comments / pointers I 
 can digest and use?


http://planet.dsource.org



 Jul 14 2014



prev sibling next sibling parent reply "Meta" <jared771 gmail.com>  writes:


On Monday, 14 July 2014 at 23:22:54 UTC, John Carter wrote:

 Any other good blog posts / social media comments / pointers I 
 can digest and use?


There are two excellent articles:

http://blog.thecybershadow.net/2014/03/21/functional-image-processing-in-d/

http://wiki.dlang.org/Component_programming_with_ranges



 Jul 14 2014



  parent  "Puming" <zhaopuming gmail.com>  writes:


and the one from Walter:

http://www.drdobbs.com/architecture-and-design/component-programming-in-d/240008321

you can see more links that I collected here:

https://github.com/zhaopuming/awesome-d



On Tuesday, 15 July 2014 at 03:46:40 UTC, Meta wrote:

 On Monday, 14 July 2014 at 23:22:54 UTC, John Carter wrote:

 Any other good blog posts / social media comments / pointers I 
 can digest and use?


 There are two excellent articles:

 http://blog.thecybershadow.net/2014/03/21/functional-image-processing-in-d/

 http://wiki.dlang.org/Component_programming_with_ranges





 Jul 14 2014



prev sibling next sibling parent  "Gary Willoughby" <dev nomad.so>  writes:


On Monday, 14 July 2014 at 23:22:54 UTC, John Carter wrote:

 I have been digesting every dconf2014 video posted so far.


See the 2013 ones too:
http://dconf.org/2013/schedule/index.html



 Jul 15 2014



prev sibling  parent  "Wyatt" <wyatt.epp gmail.com>  writes:


On Monday, 14 July 2014 at 23:22:54 UTC, John Carter wrote:

 So pointers to sound reasoned analysis of the problems of 
 C/C++/Java are welcome too.


http://yosefk.com/c++fqa/
I don't know if FQA is really in the spirit of "sound reasoned 
analysis", but it's certainly thorough!

-Wyatt



 Jul 15 2014