www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - MIT Technology Review: An Interview With Bjarne Stroustrup

reply Pragma <ericanderton yahoo.removeme.com> writes:
Something I ran into that the group might enjoy:

From: http://www.techreview.com/InfoTech/17831/

"Bjarne Stroustrup, the inventor of the C++ programming language, 
defends his legacy and examines what's wrong with most software code."

As always, Slashdot has some colorful coverage on this:

http://it.slashdot.org/it/06/12/05/0045234.shtml

"MIT's Technology Review has a Q&A with C++ inventor Bjarne Stroustrup. 
Highlights include Bjarne's answers on the trade-offs involved in the 
design of C++, and how they apply today, and his thoughts on the 
solution to the problems. From the interview: 'Software developers have 
become adept at the difficult art of building reasonably reliable 
systems out of unreliable parts. The snag is that often we do not know 
exactly how we did it.'"

-- 
- EricAnderton at yahoo
Dec 05 2006
next sibling parent reply "John Reimer" <terminal.node gmail.com> writes:
On Tue, 05 Dec 2006 08:55:09 -0800, Pragma  
<ericanderton yahoo.removeme.com> wrote:


 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


Off-topic observation:

I love the way the threading works in the comments section of this  
article!  You just click on the comment and it immediatly shows the  
comment text.  Clicking it again collapses it. You can do that for each  
comment. Now that's a great system!  To bad the D newsgroup couldn't do  
something like that for its web interface.

-JJR
Dec 05 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 
 Off-topic observation:
 
 I love the way the threading works in the comments section of this 
 article!  You just click on the comment and it immediatly shows the 
 comment text.  Clicking it again collapses it. You can do that for each 
 comment. Now that's a great system!  To bad the D newsgroup couldn't do 
 something like that for its web interface.


The new Slashdot interface does the same thing.  I love it.  But I've 
yet to see a web-based newsreader that does this.  I suppose it would 
involve the use of all sorts of acronyms I know very little about.


Sean
Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 
 Off-topic observation:
 
 I love the way the threading works in the comments section of this 
 article!  You just click on the comment and it immediatly shows the 
 comment text.  Clicking it again collapses it. You can do that for each 
 comment. Now that's a great system!  To bad the D newsgroup couldn't do 
 something like that for its web interface.
 
 -JJR
 


It's a nice concept, but (IMO) it makes for slow reading if you want to 
digest an entire thread on-page.  I'd much rather see the posts in full, 
but have the tree kept for expanding and collapsing threads as I go.

Kind of like a compromise between a TreeView and Slashdot I suppose.

Now, if you want to see something *really slick* for commenting, take a 
look at JackSlocum's site:

http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/

It's pretty script-heavy, but it shows what can be done with AJAX and a 
solid widget set.  Be sure to check out the "Documentation" section too.

-- 
- EricAnderton at yahoo
Dec 05 2006
next sibling parent reply Brad Anderson <brad dsource.org> writes:
Pragma wrote:

 John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma
 <ericanderton yahoo.removeme.com> wrote:


 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 Off-topic observation:

 I love the way the threading works in the comments section of this
 article!  You just click on the comment and it immediatly shows the
 comment text.  Clicking it again collapses it. You can do that for
 each comment. Now that's a great system!  To bad the D newsgroup
 couldn't do something like that for its web interface.

 -JJR


 
 It's a nice concept, but (IMO) it makes for slow reading if you want to
 digest an entire thread on-page.  I'd much rather see the posts in full,
 but have the tree kept for expanding and collapsing threads as I go.
 
 Kind of like a compromise between a TreeView and Slashdot I suppose.
 
 Now, if you want to see something *really slick* for commenting, take a
 look at JackSlocum's site:
 
 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/
 
 
 It's pretty script-heavy, but it shows what can be done with AJAX and a
 solid widget set.  Be sure to check out the "Documentation" section too.
 


That site crushed my browser.  I also got the FF2 "runaway script, debug or
stop" dialog.

A second visit once everything would theoretically be cached was no better.
The google pageads were pretty slow, but things were crushed enough that even
Firebug didn't respond very well.  So I couldn't tell if it was a bunch of
Ajax calls or what was running so slowly.

BA
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Brad Anderson wrote:

 
 That site crushed my browser.  I also got the FF2 "runaway script, debug or
 stop" dialog.
 
 A second visit once everything would theoretically be cached was no better.
 The google pageads were pretty slow, but things were crushed enough that even
 Firebug didn't respond very well.  So I couldn't tell if it was a bunch of
 Ajax calls or what was running so slowly.


Sorry about that. I've never had that problem on that site. :(

But then again, the YUI plus Jack's extensions are nothing trivial to 
load and run.  It's a pretty big library.

-- 
- EricAnderton at yahoo
Dec 05 2006
prev sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Brad Anderson wrote:


 Pragma wrote:

 John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma
 <ericanderton yahoo.removeme.com> wrote:


 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 Off-topic observation:

 I love the way the threading works in the comments section of this
 article!  You just click on the comment and it immediatly shows the
 comment text.  Clicking it again collapses it. You can do that for
 each comment. Now that's a great system!  To bad the D newsgroup
 couldn't do something like that for its web interface.

 -JJR


 
 It's a nice concept, but (IMO) it makes for slow reading if you want to
 digest an entire thread on-page.  I'd much rather see the posts in full,
 but have the tree kept for expanding and collapsing threads as I go.
 
 Kind of like a compromise between a TreeView and Slashdot I suppose.
 
 Now, if you want to see something *really slick* for commenting, take a
 look at JackSlocum's site:
 




http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/

 
 
 It's pretty script-heavy, but it shows what can be done with AJAX and a
 solid widget set.  Be sure to check out the "Documentation" section too.
 


 
 That site crushed my browser.  I also got the FF2 "runaway script, debug
 or stop" dialog.
 
 A second visit once everything would theoretically be cached was no
 better. The google pageads were pretty slow, but things were crushed
 enough that even
 Firebug didn't respond very well.  So I couldn't tell if it was a bunch of
 Ajax calls or what was running so slowly.
 
 BA


It works in mine FF2, but Konqueror only shows the background color, and
says nothing about errors. Personally I would never bother to read that
page, it was way too busy, I almost thought it was badly rendered at first.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
Dec 05 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Pragma wrote:

 John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:


 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 Off-topic observation:

 I love the way the threading works in the comments section of this 
 article!  You just click on the comment and it immediatly shows the 
 comment text.  Clicking it again collapses it. You can do that for 
 each comment. Now that's a great system!  To bad the D newsgroup 
 couldn't do something like that for its web interface.

 -JJR


 
 It's a nice concept, but (IMO) it makes for slow reading if you want to 
 digest an entire thread on-page.  I'd much rather see the posts in full, 
 but have the tree kept for expanding and collapsing threads as I go.
 
 Kind of like a compromise between a TreeView and Slashdot I suppose.
 
 Now, if you want to see something *really slick* for commenting, take a 
 look at JackSlocum's site:
 
 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 
 
 
 It's pretty script-heavy, but it shows what can be done with AJAX and a 
 solid widget set.  Be sure to check out the "Documentation" section too.
 


That baby is too slow ... my FF2 had to suffer before opening it ..
Dec 05 2006
parent reply =?ISO-8859-15?Q?Julio_C=E9sar_Carrascal_Urquijo?= writes:
Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take 
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX and 
 a solid widget set.  Be sure to check out the "Documentation" section 
 too.


 
 That baby is too slow ... my FF2 had to suffer before opening it ..


The Django book has the same commenting system but it's a lot faster:

http://www.djangobook.com/en/beta/chapter14/
Dec 06 2006
next sibling parent Lars Ivar Igesund <larsivar igesund.net> writes:
Julio César Carrascal Urquijo wrote:


 Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:




  >>

 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:






http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/

 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.


 
 That baby is too slow ... my FF2 had to suffer before opening it ..


 
 The Django book has the same commenting system but it's a lot faster:
 
 http://www.djangobook.com/en/beta/chapter14/


This one also worked fine with Konqueror, that other link must do something
weird then.

-- 
Lars Ivar Igesund
blog at http://larsivi.net
DSource & #D: larsivi
Dec 06 2006
prev sibling next sibling parent reply Brad Anderson <brad dsource.org> writes:
Julio César Carrascal Urquijo wrote:

 Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.


 That baby is too slow ... my FF2 had to suffer before opening it ..


 
 The Django book has the same commenting system but it's a lot faster:
 
 http://www.djangobook.com/en/beta/chapter14/


Wow, that is *really slick*
Dec 06 2006
next sibling parent Bill Baxter <wbaxter gmail.com> writes:
Brad Anderson wrote:

 Julio César Carrascal Urquijo wrote:
 

Hasan Aljudy wrote:


Pragma wrote:


John Reimer wrote:

Now, if you want to see something *really slick* for commenting, take
a look at JackSlocum's site:

http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


It's pretty script-heavy, but it shows what can be done with AJAX and
a solid widget set.  Be sure to check out the "Documentation" section
too.


That baby is too slow ... my FF2 had to suffer before opening it ..


The Django book has the same commenting system but it's a lot faster:

http://www.djangobook.com/en/beta/chapter14/


 
 
 Wow, that is *really slick*


That's exactly what's needed for the D spec pages.  The "Comments" link 
is nice, but usually the comments one wants to make are like "the *foo 
in this code should be just foo".  And many people don't even realize 
the Comments page is there as it is.  Too easy to miss.

--bb
Dec 06 2006
prev sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Brad Anderson wrote:

 Julio César Carrascal Urquijo wrote:

 Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:

 Now, if you want to see something *really slick* for commenting, take
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-yahoo-ui-and-yahooext/


 It's pretty script-heavy, but it shows what can be done with AJAX and
 a solid widget set.  Be sure to check out the "Documentation" section
 too.


 That baby is too slow ... my FF2 had to suffer before opening it ..


 The Django book has the same commenting system but it's a lot faster:

 http://www.djangobook.com/en/beta/chapter14/


 
 Wow, that is *really slick*


Amazingly, it's using the same codebase: YUI-ext!

- Eric

-- 
- EricAnderton at yahoo
Dec 07 2006
prev sibling parent reply Hasan Aljudy <hasan.aljudy gmail.com> writes:
Julio César Carrascal Urquijo wrote:

 Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:




  >>

 Now, if you want to see something *really slick* for commenting, take 
 a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX and 
 a solid widget set.  Be sure to check out the "Documentation" section 
 too.


 That baby is too slow ... my FF2 had to suffer before opening it ..


 
 The Django book has the same commenting system but it's a lot faster:
 
 http://www.djangobook.com/en/beta/chapter14/


Wow! Amazing indeed!
Commenting on a specific block! Such a simple idea yet very original and 
I can assume it can prove to be very useful and convenient.
Dec 06 2006
parent Bill Baxter <dnewsgroup billbaxter.com> writes:
Hasan Aljudy wrote:

 
 
 Julio César Carrascal Urquijo wrote:

 Hasan Aljudy wrote:

 Pragma wrote:

 John Reimer wrote:




  >>

 Now, if you want to see something *really slick* for commenting, 
 take a look at JackSlocum's site:

 http://www.jackslocum.com/blog/2006/10/09/my-wordpress-comments-system-built-with-ya
oo-ui-and-yahooext/ 


 It's pretty script-heavy, but it shows what can be done with AJAX 
 and a solid widget set.  Be sure to check out the "Documentation" 
 section too.


 That baby is too slow ... my FF2 had to suffer before opening it ..


 The Django book has the same commenting system but it's a lot faster:

 http://www.djangobook.com/en/beta/chapter14/


 
 Wow! Amazing indeed!
 Commenting on a specific block! Such a simple idea yet very original and 
 I can assume it can prove to be very useful and convenient.
 


But I wonder what happens when the maintainers of the documentation want 
to make changes that move, split, merge, delete, and reorganize paragraphs.

--bb
Dec 06 2006
prev sibling parent Hasan Aljudy <hasan.aljudy gmail.com> writes:
John Reimer wrote:

 On Tue, 05 Dec 2006 08:55:09 -0800, Pragma 
 <ericanderton yahoo.removeme.com> wrote:
 

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/


 
 Off-topic observation:
 
 I love the way the threading works in the comments section of this 
 article!  You just click on the comment and it immediatly shows the 
 comment text.  Clicking it again collapses it. You can do that for each 
 comment. Now that's a great system!  To bad the D newsgroup couldn't do 
 something like that for its web interface.
 
 -JJR
 


AJAX ruulz baby!!
Dec 05 2006
prev sibling next sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Pragma" <ericanderton yahoo.removeme.com> wrote in message 
news:el489h$1f7a$1 digitaldaemon.com...

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."


Bjarne: "There are just two kinds of languages: the ones everybody complains 
about and the ones nobody uses."

And for C++ programmers, those are the same languages.  ;)

When will Bjarne see that C++ is a festering homonculus of a language from 
which pours forth a black bile of rot and putrescence?  Okay, maybe I'm 
being a bit hyperbolistic, but..
Dec 05 2006
next sibling parent reply clayasaurus <clayasaurus gmail.com> writes:
Jarrett Billingsley wrote:

 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el489h$1f7a$1 digitaldaemon.com...

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."


 
 Bjarne: "There are just two kinds of languages: the ones everybody complains 
 about and the ones nobody uses."
 
 And for C++ programmers, those are the same languages.  ;)
 
 When will Bjarne see that C++ is a festering homonculus of a language from 
 which pours forth a black bile of rot and putrescence?  Okay, maybe I'm 
 being a bit hyperbolistic, but.. 
 
 


I like how he talks about C++ being 'expert' friendly and having 
'expert' level features. Is this just a nice way of saying that the 
language is difficult to use? :-P

~ Clay
Dec 05 2006
parent Pragma <ericanderton yahoo.removeme.com> writes:
clayasaurus wrote:

 Jarrett Billingsley wrote:
 
 I like how he talks about C++ being 'expert' friendly and having 
 'expert' level features. Is this just a nice way of saying that the 
 language is difficult to use? :-P


That's pretty much how I read that too.

-- 
- EricAnderton at yahoo
Dec 05 2006
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Jarrett Billingsley wrote:

 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el489h$1f7a$1 digitaldaemon.com...

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, defends 
 his legacy and examines what's wrong with most software code."


 
 Bjarne: "There are just two kinds of languages: the ones everybody complains 
 about and the ones nobody uses."
 
 And for C++ programmers, those are the same languages.  ;)
 
 When will Bjarne see that C++ is a festering homonculus of a language from 
 which pours forth a black bile of rot and putrescence?  Okay, maybe I'm 
 being a bit hyperbolistic, but.. 


In Bjarne's defense, I think it's worth noting two things:

* The success of C++ was very much dependent on its acceptance by the C 
community, and C compatibility was crucial.  Many of the language and 
syntax compromises were for very legitimate reasons at the time the 
language was designed.

* Bjarne quite unapologetically designed C++ for expert programmers.  It 
isn't his fault that the language has become so popular that it is now 
being used by people who can't tell the dull side of a pointer from, 
well, the pointy side.

There is no debating the fact that C++ has been an incredible success, 
and it still has very little competition in many core markets. 
Personally, my only real problems with the language are that its 
popularity has driven it to be used in projects and by people where 
another language would be more suitable, and the language supports such 
a wide array of programming styles that it is extremely difficult to 
maintain any kind of design coherence in large team projects.  Also, its 
age is such that many projects contain code written before templates 
even existed, and so a substantial codebase follows "old style C++" 
which is heavy with raw pointers, casts, etc.  In fact, I'm continually 
amazed at how little new code I see even today that is written using STL 
components.  IMO this is a strong argument for D's built-in dynamic 
array support and other features, as there is no doubt in my mind that 
one of the major problems with C++ is that the language was standardized 
and in use before the library was up to snuff.  By integrating these 
features into the core language, Walter has neatly sidestepped this 
problem and provided a nice, clean syntax for some of the most commonly 
used programming constructs.


Sean
Dec 05 2006
parent reply zz <zz zz.com> writes:
Sean Kelly wrote:


 There is no debating the fact that C++ has been an incredible success, 
 and it still has very little competition in many core markets. 
 Personally, my only real problems with the language are that its 
 popularity has driven it to be used in projects and by people where 
 another language would be more suitable, and the language supports such 
 a wide array of programming styles that it is extremely difficult to 
 maintain any kind of design coherence in large team projects.  Also, its 
 age is such that many projects contain code written before templates 
 even existed, and so a substantial codebase follows "old style C++" 
 which is heavy with raw pointers, casts, etc.  In fact, I'm continually 
 amazed at how little new code I see even today that is written using STL 
 components.  IMO this is a strong argument for D's built-in dynamic 
 array support and other features, as there is no doubt in my mind that 
 one of the major problems with C++ is that the language was standardized 
 and in use before the library was up to snuff.  By integrating these 
 features into the core language, Walter has neatly sidestepped this 
 problem and provided a nice, clean syntax for some of the most commonly 
 used programming constructs.


I agree with everything you mention above but there are cases where C++ 
will just be a better due to performance and nothing else.

I did some work in D recently that is being used at a client's place and 
it took about 2 hours to design and write (it's was not something that 
was performance critical), one of my work mates got interested in D and 
for the sake of curiosity we ran some test comparing D's built in arrays 
with boost::ptr_vector which we use quite a lot and the results were as 
follows:
VS2003 = ~3.75 secs
VS2003 with NedMalloc = ~1.35 secs
DMD = ~ 7 secs
We could not get DMC to compile Nedmalloc so we dropped testing DMC with 
c++.

While he liked that language and said the he might actually use it to 
prototype idea's, he will not use it in production code due to the 
performance.

Conclusions:
D is great, but DMD will have to do something about it's performance for 
some applications.

Zz

 
 Sean
Dec 05 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
zz wrote:

 
 I did some work in D recently that is being used at a client's place and 
 it took about 2 hours to design and write (it's was not something that 
 was performance critical), one of my work mates got interested in D and 
 for the sake of curiosity we ran some test comparing D's built in arrays 
 with boost::ptr_vector which we use quite a lot and the results were as 
 follows:
 VS2003 = ~3.75 secs
 VS2003 with NedMalloc = ~1.35 secs
 DMD = ~ 7 secs
 We could not get DMC to compile Nedmalloc so we dropped testing DMC with 
 c++.
 
 While he liked that language and said the he might actually use it to 
 prototype idea's, he will not use it in production code due to the 
 performance.


How long are these tests?  Could they be put online somewhere?  I'd be 
interested in trying these out for myself and see if I can close the gap 
a bit.


Sean
Dec 05 2006
parent zz <zz zz.com> writes:
Sean Kelly wrote:

 zz wrote:
 
 How long are these tests?  Could they be put online somewhere?  I'd be 
 interested in trying these out for myself and see if I can close the gap 
 a bit.


The test were run about 20 times with the average taken.

The test was very simplistic and I'll send them to you tomorrow (I'm out 
of the office until tomorrow).

Generally it's a structure with three strings, and it's newed filled in 
and pushed back into the vector 1,000,000 times (we have higher).

I also did the same test in C using talloc which is a hierarchical pool 
based memory allocator using VS2003 and DMC and if I recall correctly 
these are the numbers:
VS2003 = 4.0 - 4.5
VS2003 with nedmalloc = 2.0 - 2.5
DMC = 6.5 - 7.5

Zz

 
 Sean
Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:

 
 Conclusions:
 D is great, but DMD will have to do something about it's performance for 
 some applications.


I tend to agree.  D's array allocation algorithm, as well as it's GC 
behavior, are great for most cases.  But they can fall flat in certain 
cases - of course that's true of any algorithm really. :)

While I'm no STL guru, I have to ask: which allocator were you using 
with ptr_vector?

I know that with D, pre-allocating data for arrays can make a big 
difference if you expect to perform a lot of concatenations - especially 
with atomic elements like pointers or references.

-- 
- EricAnderton at yahoo
Dec 05 2006
next sibling parent reply Sean Kelly <sean f4.ca> writes:
Pragma wrote:

 zz wrote:

 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.


 
 I tend to agree.  D's array allocation algorithm, as well as it's GC 
 behavior, are great for most cases.  But they can fall flat in certain 
 cases - of course that's true of any algorithm really. :)
 
 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?
 
 I know that with D, pre-allocating data for arrays can make a big 
 difference if you expect to perform a lot of concatenations - especially 
 with atomic elements like pointers or references.


I still find this confusing.  The GC uses "power of two" sized blocks up 
to page size where the grow strategy kicks in, so it should already be 
using "double my current size" allocation behind the scenes.  But I 
suppose I should really spend some time testing this to see if it can be 
improved.  One slightly crazy idea that's been kicked around is to allow 
the user to override the default grow strategy, but that only seems 
useful if the default one really isn't appropriate for most situations, 
and I'm hoping that testing plus perhaps some tweaks to the algorithm 
will obviate the need for this.


Sean
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Sean Kelly wrote:

 Pragma wrote:

 zz wrote:

 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.


 I tend to agree.  D's array allocation algorithm, as well as it's GC 
 behavior, are great for most cases.  But they can fall flat in certain 
 cases - of course that's true of any algorithm really. :)

 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

 I know that with D, pre-allocating data for arrays can make a big 
 difference if you expect to perform a lot of concatenations - 
 especially with atomic elements like pointers or references.


 
 I still find this confusing.  The GC uses "power of two" sized blocks up 
 to page size where the grow strategy kicks in, so it should already be 
 using "double my current size" allocation behind the scenes.  But I 
 suppose I should really spend some time testing this to see if it can be 
 improved.  One slightly crazy idea that's been kicked around is to allow 
 the user to override the default grow strategy, but that only seems 
 useful if the default one really isn't appropriate for most situations, 
 and I'm hoping that testing plus perhaps some tweaks to the algorithm 
 will obviate the need for this.


Sean, I'm pretty sure you've seen this before.  But just in case:

http://svn.dsource.org/projects/ddl/trunk/ddl/ExpContainer.d

It might give you (or others) a head-start along that path.


-- 
- EricAnderton at yahoo
Dec 05 2006
prev sibling parent Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:

 Pragma wrote:

 zz wrote:

 Conclusions:
 D is great, but DMD will have to do something about it's performance 
 for some applications.


 I tend to agree.  D's array allocation algorithm, as well as it's GC 
 behavior, are great for most cases.  But they can fall flat in certain 
 cases - of course that's true of any algorithm really. :)

 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?

 I know that with D, pre-allocating data for arrays can make a big 
 difference if you expect to perform a lot of concatenations - 
 especially with atomic elements like pointers or references.


 
 I still find this confusing.  The GC uses "power of two" sized blocks up 
 to page size where the grow strategy kicks in, so it should already be 
 using "double my current size" allocation behind the scenes.  But I 
 suppose I should really spend some time testing this to see if it can be 
 improved.  One slightly crazy idea that's been kicked around is to allow 
 the user to override the default grow strategy, but that only seems 
 useful if the default one really isn't appropriate for most situations, 
 and I'm hoping that testing plus perhaps some tweaks to the algorithm 
 will obviate the need for this.
 


If concatenating to the vector is the bottleneck, they may have been
concatenating to the D 'vector' 
with arr = arr ~ elem; instead of arr ~= elem;? In that case it wouldn't be
preallocating, but 
creating a bunch of temporaries.

Just a guess.

If that's the case, wouldn't if be pretty 'easy' and safe to optimize the
built-in?


 
 Sean
Dec 05 2006
prev sibling parent reply zz <zz zz.com> writes:
Pragma wrote:

 zz wrote:
 
 
 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?
 


For the first number we used VS2003 SMT CRT (Default allocator)
With the other test we used nedmalloc as the allocator (just over load 
new and delete in C++ (it's a c library).

http://www.nedprod.com/programs/portable/nedmalloc/

Zz
Dec 05 2006
next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).
 
 http://www.nedprod.com/programs/portable/nedmalloc/


Wow.  Well that explains it then.

Sean, you should seriously check this thing out.  Maybe this could be 
adapted to be used in Ares?

-- 
- EricAnderton at yahoo
Dec 05 2006
prev sibling next sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).
 
 http://www.nedprod.com/programs/portable/nedmalloc/


Wow.  Well that explains it then.

Sean, you should seriously check this thing out.  Maybe this could be
adapted to be used in Ares?

[Edit:] The freshmeat article I found for nedmalloc calls it an 
"alternative malloc implementation for multiple threads without lock 
contention based on dlmalloc 2.8.3."  Suddenly, the speed increase is 
makes a lot more sense.

-- 
- EricAnderton at yahoo
Dec 05 2006
parent reply Sean Kelly <sean f4.ca> writes:
Pragma wrote:

 zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).

 http://www.nedprod.com/programs/portable/nedmalloc/


 
 Wow.  Well that explains it then.
 
 Sean, you should seriously check this thing out.  Maybe this could be
 adapted to be used in Ares?


It shouldn't be too hard to create a D wrapper for it, but converting it 
to a garbage collector would be non-trivial.  It may be best to simple 
offer it as an add-on package for folks who really need that extra bit 
of performance.


Sean
Dec 10 2006
parent reply zz <zz zz.com> writes:
Sean Kelly wrote:

 It shouldn't be too hard to create a D wrapper for it, but converting it 
 to a garbage collector would be non-trivial.  It may be best to simple 
 offer it as an add-on package for folks who really need that extra bit 
 of performance.
 


I couldn't get NedMalloc to compile under DMC but dlmalloc which 
nedmalloc uses compiled and the results I got from some c code were like 
1:5.
I wonder what would happen if dlmalloc was used as the base allocator in 
Ares or Phobos, would there be any improvement? (I just messed thing up 
when playing with phobos but will look into it again as for Ares the 
dsource site seems to be having problems).

Zz
Dec 10 2006
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
zz wrote:

 I wonder what would happen if dlmalloc was used as the base allocator in 
 Ares or Phobos, would there be any improvement? (I just messed thing up 
 when playing with phobos but will look into it again as for Ares the 
 dsource site seems to be having problems).


For Phobos, I don't think it would make much of a difference. AFAICT it 
doesn't use malloc at all, except for COM objects. It requests pages 
directly from the OS instead.
So unless you use a lot of COM objects the difference will be negligible.
IIRC Ares uses essentially the same GC so the same would apply.
Dec 10 2006
parent zz <zz zz.com> writes:
Frits van Bommel wrote:

 zz wrote:
 

 I wonder what would happen if dlmalloc was used as the base allocator 
 in Ares or Phobos, would there be any improvement? (I just messed 
 thing up when playing with phobos but will look into it again as for 
 Ares the dsource site seems to be having problems).


 
 
 For Phobos, I don't think it would make much of a difference. AFAICT it 
 doesn't use malloc at all, except for COM objects. It requests pages 
 directly from the OS instead.
 So unless you use a lot of COM objects the difference will be negligible.
 IIRC Ares uses essentially the same GC so the same would apply.


Thanks for the clarification, it's a gray area for me.

I noticed a few calls to std.c.stdlib.malloc and family then assumed 
that something done at that level may make some difference.

Zz
Dec 10 2006
prev sibling parent reply Bill Baxter <dnewsgroup billbaxter.com> writes:
zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).
 
 http://www.nedprod.com/programs/portable/nedmalloc/
 
 Zz


I was going to ask why he didn't compare his algorithm with dlmalloc, 
but I see it actually is based on dlmalloc.  So I wonder what exactly 
the differences is between this and dlmalloc.

Incidentally I know this guy from his posts on the FOX mailing lists. 
He's been working on this thing called Tn[1] for about 10 years.  He's 
either insane or brilliant I can't really tell.  :-) Read the 
description of "What is Tn" to see what I mean.

[1] http://www.nedprod.com/Tn/index.html

--bb
Dec 05 2006
next sibling parent zz <zz zz.com> writes:
Bill Baxter wrote:

 I was going to ask why he didn't compare his algorithm with dlmalloc, 
 but I see it actually is based on dlmalloc.  So I wonder what exactly 
 the differences is between this and dlmalloc.


nedmalloc actually uses dlmalloc underneath.


 Incidentally I know this guy from his posts on the FOX mailing lists. 
 He's been working on this thing called Tn[1] for about 10 years.  He's 
 either insane or brilliant I can't really tell.  :-) Read the 
 description of "What is Tn" to see what I mean.


It's a very thin line between the two, but we use nedmalloc internally 
and at least there I would have to claim that he is brilliant.

Zz


 [1] http://www.nedprod.com/Tn/index.html
 
 --bb
Dec 05 2006
prev sibling next sibling parent Pragma <ericanderton yahoo.removeme.com> writes:
Bill Baxter wrote:

 zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).

 http://www.nedprod.com/programs/portable/nedmalloc/

 Zz


 
 I was going to ask why he didn't compare his algorithm with dlmalloc, 
 but I see it actually is based on dlmalloc.  So I wonder what exactly 
 the differences is between this and dlmalloc.
 
 Incidentally I know this guy from his posts on the FOX mailing lists. 
 He's been working on this thing called Tn[1] for about 10 years.  He's 
 either insane or brilliant I can't really tell.  :-) Read the 
 description of "What is Tn" to see what I mean.
 
 [1] http://www.nedprod.com/Tn/index.html
 
 --bb


Wow, that's pretty neat.  I'm leaning more towards brilliance than 
insanity on this one.

For a solid baseline of insanity, please visit the Time Cube:

http://www.timecube.com/

(Warning: you'll go schizophrenic just *trying* to make sense that page.)

-- 
- EricAnderton at yahoo
Dec 07 2006
prev sibling parent reply Pragma <ericanderton yahoo.removeme.com> writes:
Bill Baxter wrote:

 zz wrote:

 Pragma wrote:

 zz wrote:


 While I'm no STL guru, I have to ask: which allocator were you using 
 with ptr_vector?


 For the first number we used VS2003 SMT CRT (Default allocator)
 With the other test we used nedmalloc as the allocator (just over load 
 new and delete in C++ (it's a c library).

 http://www.nedprod.com/programs/portable/nedmalloc/

 Zz


 
 I was going to ask why he didn't compare his algorithm with dlmalloc, 
 but I see it actually is based on dlmalloc.  So I wonder what exactly 
 the differences is between this and dlmalloc.
 
 Incidentally I know this guy from his posts on the FOX mailing lists. 
 He's been working on this thing called Tn[1] for about 10 years.  He's 
 either insane or brilliant I can't really tell.  :-) Read the 
 description of "What is Tn" to see what I mean.
 
 [1] http://www.nedprod.com/Tn/index.html
 
 --bb


Wow, that's pretty neat.  I'm leaning more towards brilliance than
insanity on this one.

For a solid baseline of insanity, please visit the Time Cube:

http://www.timecube.com/

(Warning: you'll go schizophrenic just trying to make sense of that page.)

-- 
- EricAnderton at yahoo
Dec 07 2006
next sibling parent =?ISO-8859-1?Q?Julio_C=E9sar_Carrascal_Urquijo?= writes:
Pragma wrote:

 For a solid baseline of insanity, please visit the Time Cube:
 
 http://www.timecube.com/
 
 (Warning: you'll go schizophrenic just trying to make sense of that page.)
 


Mmmmm. It looks like the output of a Markov chain generator.
Dec 07 2006
prev sibling parent reply "Jarrett Billingsley" <kb3ctd2 yahoo.com> writes:
"Pragma" <ericanderton yahoo.removeme.com> wrote in message 
news:el9b6l$khl$3 digitaldaemon.com...


 For a solid baseline of insanity, please visit the Time Cube:

 http://www.timecube.com/

 (Warning: you'll go schizophrenic just trying to make sense of that page.)


Wow...

This _has_ to have been the influence for this joke page: 
http://www.somethingawful.com/learning_triangle/index.htm

Unless, of course, there are more pages like it, which certainly wouldn't 
surprise me.
Dec 07 2006
parent Sean Kelly <sean f4.ca> writes:
Jarrett Billingsley wrote:

 "Pragma" <ericanderton yahoo.removeme.com> wrote in message 
 news:el9b6l$khl$3 digitaldaemon.com...
 

 For a solid baseline of insanity, please visit the Time Cube:

 http://www.timecube.com/

 (Warning: you'll go schizophrenic just trying to make sense of that page.)


 
 Wow...
 
 This _has_ to have been the influence for this joke page: 
 http://www.somethingawful.com/learning_triangle/index.htm
 
 Unless, of course, there are more pages like it, which certainly wouldn't 
 surprise me. 


More than you can count.  Here is a page with links to some of the 
classics:

http://www.f4.ca/crackpot/


Sean
Dec 07 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
zz wrote:

 I agree with everything you mention above but there are cases where C++ 
 will just be a better due to performance and nothing else.
 
 I did some work in D recently that is being used at a client's place and 
 it took about 2 hours to design and write (it's was not something that 
 was performance critical), one of my work mates got interested in D and 
 for the sake of curiosity we ran some test comparing D's built in arrays 
 with boost::ptr_vector which we use quite a lot and the results were as 
 follows:
 VS2003 = ~3.75 secs
 VS2003 with NedMalloc = ~1.35 secs
 DMD = ~ 7 secs
 We could not get DMC to compile Nedmalloc so we dropped testing DMC with 
 c++.
 
 While he liked that language and said the he might actually use it to 
 prototype idea's, he will not use it in production code due to the 
 performance.
 
 Conclusions:
 D is great, but DMD will have to do something about it's performance for 
 some applications.


There are many possible explanations for this, and not all of them are 
because C++ has better performance. We need to see the code and the 
compiler switches used.

Here's a case where D is substantially faster than C++: 
http://www.digitalmars.com/d/cppbench.html
Dec 05 2006
next sibling parent reply BCS <BCS pathilink.com> writes:
 
 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html


bad link
Dec 05 2006
parent zz <zz zz.com> writes:
BCS wrote:

 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html


 
 
 bad link


http://www.digitalmars.com//d/cppstrings.html

Zz
Dec 05 2006
prev sibling parent reply zz <zz zz.com> writes:
Walter Bright wrote:

 
 There are many possible explanations for this, and not all of them are 
 because C++ has better performance. We need to see the code and the 
 compiler switches used.


Walter, it's not a C++ vs D issue (I preffer D) it's a compiler/library 
issue.

I don't claim C++ has better performance, but that VS2003 does a better 
job than DMC/DMD when there lot of memory allocations and even better 
when combined with nedmalloc.


 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html


I'll have to run it under VS2003 or VS2005.

I'll post the trivial test we did tommorow, but in another test in C 
using VS2003 default allocator, the same code was much faster in VS2003 
under release then in DMC -o.

In the test we are doing new is called 1,000,000 times and the result 
put into a boost::ptr_vector in c++.

Zz
Dec 05 2006
parent reply "Craig Black" <cblack ara.com> writes:
I don't see why D doesn't use nedmalloc for its non-GC malloc 
implementation.  This would allow expert developers that avoid GC in certain 
situations glean the best performance possible.

-Craig

"zz" <zz zz.com> wrote in message news:el51rm$2cth$1 digitaldaemon.com...

 Walter Bright wrote:

 There are many possible explanations for this, and not all of them are 
 because C++ has better performance. We need to see the code and the 
 compiler switches used.


 Walter, it's not a C++ vs D issue (I preffer D) it's a compiler/library 
 issue.

 I don't claim C++ has better performance, but that VS2003 does a better 
 job than DMC/DMD when there lot of memory allocations and even better when 
 combined with nedmalloc.


 Here's a case where D is substantially faster than C++: 
 http://www.digitalmars.com/d/cppbench.html


 I'll have to run it under VS2003 or VS2005.

 I'll post the trivial test we did tommorow, but in another test in C using 
 VS2003 default allocator, the same code was much faster in VS2003 under 
 release then in DMC -o.

 In the test we are doing new is called 1,000,000 times and the result put 
 into a boost::ptr_vector in c++.

 Zz
Dec 06 2006
parent reply Sean Kelly <sean f4.ca> writes:
Craig Black wrote:

 I don't see why D doesn't use nedmalloc for its non-GC malloc 
 implementation.  This would allow expert developers that avoid GC in certain 
 situations glean the best performance possible.


The user could do this quite easily himself.  Just create a D header 
module for the malloc calls and use them instead of std.c.stdlib.malloc, 
etc.


Sean
Dec 06 2006
parent reply "Craig Black" <cblack ara.com> writes:
Perhaps.  But why not do this by default and spare everyone the headache?

-Craig

"Sean Kelly" <sean f4.ca> wrote in message 
news:el75b4$1ho3$2 digitaldaemon.com...

 Craig Black wrote:

 I don't see why D doesn't use nedmalloc for its non-GC malloc 
 implementation.  This would allow expert developers that avoid GC in 
 certain situations glean the best performance possible.


 The user could do this quite easily himself.  Just create a D header 
 module for the malloc calls and use them instead of std.c.stdlib.malloc, 
 etc.


 Sean
Dec 06 2006
parent reply zz <zz zz.com> writes:
I modified the C++ that was written to reflect the D code more closely:
And now DMD is slightly faster than VS2003 without nedmalloc.

Has anyone managed to compile nedmalloc under DMC?

I don't believe this is a good test, but anyway here it is.

And as I said before D is great and I'll continue using it everyday.

Zz

Results:
----------------------------------------------
VS2003:
Element count: 1000000

ContextSwitches - 122127
First level fills = 0
Second level fills = 0

ETime(   0:00:05.921 ) UTime(   0:00:05.687 ) KTime(   0:00:00.171 )
ITime(   0:00:05.296 )

----------------------------------------------
VS2003 With nedmalloc:
Element count: 1000000

ContextSwitches - 52979
First level fills = 0
Second level fills = 0

ETime(   0:00:02.593 ) UTime(   0:00:02.562 ) KTime(   0:00:00.062 )
ITime(   0:00:02.328 )

----------------------------------------------
DMD:
Element count: 1000000

ContextSwitches - 118191
First level fills = 0
Second level fills = 0

ETime(   0:00:05.281 ) UTime(   0:00:05.203 ) KTime(   0:00:00.031 )
ITime(   0:00:04.875 )

----------------------------------------------
C++ Code:
#include <iostream>
#include <string>
#include <memory>
#include <boost/ptr_container/ptr_vector.hpp>
#include "nedmalloc.h"

void* operator new(size_t sz)
{
	void* m = nedmalloc(sz);
	if(!m) puts("out of memory");
	return m;
}

void operator delete(void* m)
{
	nedfree(m);
}

class cpp_element
{
public:
	std::string Creator;
	std::string CreationDate;
	std::string Label;
};

class cpp_root
{
public:
	boost::ptr_vector<cpp_element> elements;
};


int main(int argc, char* argv[])
{
	cpp_root* _root = new cpp_root();
	std::auto_ptr<cpp_root> root(_root);

	std::string cd1 = "AAAAAAAAAAAAAAAAAAAAAAAAA";
	std::string cd2 = "BBBBBBBBBBBBBBBBBBBBBBBBB";
	std::string cd3 = "CCCCCCCCCCCCCCCCCCCCCCCCC";

	for(int i = 0; i < 1000000; i++)
	{
		cpp_element* element = new cpp_element();

		element->CreationDate = cd1 + "cd1";
		element->Creator      = cd2 + "cd2";
		element->Label        = cd3 + "cd3";

		root->elements.push_back(element);
	}

	printf("Element count: %d\n", root->elements.size());

	return 0;
}


----------------------------------------------
D Code:
module test;
import std.stdio;

class d_element
{
public:
	char[] Creator;
	char[] CreationDate;
	char[] Label;
}

class d_root
{
public:
	d_element[] elements;
}

int main()
{
	char[] cd1 = "AAAAAAAAAAAAAAAAAAAAAAAAA";
	char[] cd2 = "BBBBBBBBBBBBBBBBBBBBBBBBB";
	char[] cd3 = "CCCCCCCCCCCCCCCCCCCCCCCCC";
	
	d_root root = new d_root();
	
	for(int i = 0; i < 1000000; i++)
	{
		d_element element = new d_element();
		element.CreationDate = cd1 ~= "cd1";
		element.Creator      = cd2 ~= "cd2";
		element.Label        = cd3 ~= "cd3";
		root.elements ~= element;
	}

	printf("Element count: %d\n", root.elements.length);	
	return 0;
}
Dec 06 2006
parent reply Frits van Bommel <fvbommel REMwOVExCAPSs.nl> writes:
zz wrote:

         cpp_element* element = new cpp_element();
 
         element->CreationDate = cd1 + "cd1";
         element->Creator      = cd2 + "cd2";
         element->Label        = cd3 + "cd3";


<snip>

         d_element element = new d_element();
         element.CreationDate = cd1 ~= "cd1";
         element.Creator      = cd2 ~= "cd2";
         element.Label        = cd3 ~= "cd3";


I don't think those lines are really equivalent. Hint: write the last 
element in the list to output, in both versions. You might notice a 
difference :).
Dec 06 2006
parent reply zz <zz zz.com> writes:
Frits van Bommel wrote:

 zz wrote:
 

         cpp_element* element = new cpp_element();

         element->CreationDate = cd1 + "cd1";
         element->Creator      = cd2 + "cd2";
         element->Label        = cd3 + "cd3";


 
 <snip>
 

         d_element element = new d_element();
         element.CreationDate = cd1 ~= "cd1";
         element.Creator      = cd2 ~= "cd2";
         element.Label        = cd3 ~= "cd3";


 
 
 I don't think those lines are really equivalent. Hint: write the last 
 element in the list to output, in both versions. You might notice a 
 difference :).


Ouch. That was really silly of me (Didn't notice it).
It should have been element.CreationDate = cd ~ "cd1" and yes it would 
make a difference.

What I had yesterday was using cd1.dup, didn't notice the silly mistake 
when making the change.

After fixing this problem here are my results on a different machine.

 From my point of view these tests are not really nessesary on my side 
since I still continue using D and I belive that someday the memory 
stuff will be optimized.

Zz

-------------------------------------------------------
VS2003
Element count: 1000000

ContextSwitches - 24787
First level fills = 0
Second level fills = 0

ETime(   0:00:05.328 ) UTime(   0:00:05.046 ) KTime(   0:00:00.187 )
ITime(   0:00:04.828 )

-------------------------------------------------------
VS2003 with nedmalloc
Element count: 1000000

ContextSwitches - 8854
First level fills = 0
Second level fills = 0

ETime(   0:00:02.390 ) UTime(   0:00:02.218 ) KTime(   0:00:00.078 )
ITime(   0:00:02.015 )

-------------------------------------------------------
DMC
Element count: 1000000

ContextSwitches - 35008
First level fills = 0
Second level fills = 0

ETime(   0:00:07.312 ) UTime(   0:00:07.000 ) KTime(   0:00:00.031 )
ITime(   0:00:06.656 )

D change:
	for(int i = 0; i < 1000000; i++)
	{
		d_element element = new d_element();
		element.CreationDate = cd1 ~ "cd1";
		element.Creator      = cd2 ~ "cd2";
		element.Label        = cd3 ~ "cd3";
		root.elements ~= element;
	}
Dec 06 2006
next sibling parent Sean Kelly <sean f4.ca> writes:
zz wrote:

 
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.


It will, but the changes will happen sooner if particular use cases are 
brought up as problematic.  Particularly if tweaks can be made here and 
there to improve performance without writing an entirely new garbage 
collector.  That said, I have considered writing a GC more targeted at 
concurrent programming than the current implementation (basically an 
improved version of Hoard with mark/sweep collection added), but doing 
so would take time I don't have at the moment.  Maybe sometime after 1.0.


Sean
Dec 06 2006
prev sibling next sibling parent Dave <Dave_member pathlink.com> writes:
zz wrote:

 
 After fixing this problem here are my results on a different machine.
 
  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.
 
 Zz
 
 -------------------------------------------------------
 VS2003
 Element count: 1000000
 
 ContextSwitches - 24787
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:05.328 ) UTime(   0:00:05.046 ) KTime(   0:00:00.187 )
 ITime(   0:00:04.828 )
 
 -------------------------------------------------------
 VS2003 with nedmalloc
 Element count: 1000000
 
 ContextSwitches - 8854
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:02.390 ) UTime(   0:00:02.218 ) KTime(   0:00:00.078 )
 ITime(   0:00:02.015 )
 
 -------------------------------------------------------
 DMC
 Element count: 1000000
 
 ContextSwitches - 35008
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:07.312 ) UTime(   0:00:07.000 ) KTime(   0:00:00.031 )
 ITime(   0:00:06.656 )
 
 D change:
     for(int i = 0; i < 1000000; i++)
     {
         d_element element = new d_element();
         element.CreationDate = cd1 ~ "cd1";
         element.Creator      = cd2 ~ "cd2";
         element.Label        = cd3 ~ "cd3";
         root.elements ~= element;
     }
 
 


If the code is changed for both D and C++ to not cat the small strings:

     for(int i = 0; i < 1000000; i++)
     {
         d_element element = new d_element();
         element.CreationDate = cd1;// ~ "cd1";
         element.Creator      = cd2;// ~ "cd2";
         element.Label        = cd3;// ~ "cd3";
         root.elements ~= element;
     }

VC++: 2.609 (using NedAlloc)
DMD:  0.937

Pretty interesting. The opCatAssign seems very fast. opCat not.

Zz, I don't know how closely your test mimics the bottlenecks in your
production code, but maybe if 
you could move things like that opCat out of loops in some cases, you'd get
faster code than with 
ptr_vector<>?
Dec 06 2006
prev sibling parent reply Walter Bright <newshound digitalmars.com> writes:
zz wrote:

  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.


One thing that is happening is that the C++ code allocates memory, it 
never frees it. The same with the D code.

What happens with a garbage collector is it gets new chunks of memory 
from the operating system as it needs it. But before it does, it runs a 
collection cycle.

So in the D version, it will be running probably several collection 
cycles (accomplishing nothing), while the C++ version does not. This 
will make the gc version slower.

To get better numbers, one can add calls to gc.disable() and gc.enable() 
to tell the gc that there is no point to running a collection cycle for 
this section of code.
Dec 07 2006
next sibling parent reply zz <zz zz.com> writes:
Walter Bright wrote:

 zz wrote:
 

  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.


 
 
 One thing that is happening is that the C++ code allocates memory, it 
 never frees it. The same with the D code.


Not until it's out of scope (in this case main).


 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs a 
 collection cycle.


Would something like dlmalloc by Doug Lea make a difference in D, 
someone suggested trying it out so I tested it using DMC on some C code 
and there was a big difference the two compiled versions (Numbers are 
below and I was supprised).


 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.
 
 To get better numbers, one can add calls to gc.disable() and gc.enable() 
 to tell the gc that there is no point to running a collection cycle for 
 this section of code.


The above is something that, i'll not do (In DMC i can get better 
numbers by just changing my allocator how can I go about it in D).

Zz

-----------------------------------------------
- Default memory allocator
dmc -o+all test.c
Capacity C: 1048576  Count: 1000000

ContextSwitches - 19954
First level fills = 0
Second level fills = 0

ETime(   0:00:06.562 ) UTime(   0:00:06.015 ) KTime(   0:00:00.375 )
ITime(   0:00:03.828 )


-----------------------------------------------
dmc -DREPLACE_SYSTEM_ALLOCATOR -o+all test.c
Capacity C: 1048576  Count: 1000000

ContextSwitches - 3092
First level fills = 0
Second level fills = 0

ETime(   0:00:01.250 ) UTime(   0:00:00.968 ) KTime(   0:00:00.250 )
ITime(   0:00:00.796 )

Note: I used dlmalloc Version pre-2.8.4 Wed Aug  2 14:13:56 2006 which 
comes with nedmalloc.
The current version is 2.8.3 and ptmalloc3 comes with an earlier version 
of the one used above.
Dec 07 2006
next sibling parent reply Dave <Dave_member pathlink.com> writes:
zz wrote:

 Walter Bright wrote:

 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.



 One thing that is happening is that the C++ code allocates memory, it 
 never frees it. The same with the D code.


 
 Not until it's out of scope (in this case main).
 


With that in mind, I wrapped your original code in a loop for 10 iterations and
decreased the inner 
loop to 100000.

C++ (NedAlloc):
Total Element count: 1000000
5.438000

D:
Total Element count: 1000000
1.360000

Because it's not doing all of it's allocation in one tight loop, maybe this
better 'emulates' a 
'typical' service or client application? Point is, have you and your coworker
from the OP actually 
tried D in a couple of real-world application examples?

Quote: "While he liked that language and said the he might actually use it to
prototype idea's, he 
will not use it in production code due to the performance."

That sounds like me two years ago (and D hasn't gotten all that much faster
since then) <g>

Don't take this as some sort of attack -- you bring up some very good points.
And I agree that the 
GC performance should be looked at (and that great performance is critical),
but I'm wondering if D 
is getting a fair shake in your shop?

Even if you had to spend 10% development time to hoist some allocations out of
loops with D, if you 
save 20% developing the first-cut you'll end up better off ;)


 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs 
 a collection cycle.


 
 Would something like dlmalloc by Doug Lea make a difference in D, 
 someone suggested trying it out so I tested it using DMC on some C code 
 and there was a big difference the two compiled versions (Numbers are 
 below and I was supprised).
 

 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.

 To get better numbers, one can add calls to gc.disable() and 
 gc.enable() to tell the gc that there is no point to running a 
 collection cycle for this section of code.


 
 The above is something that, i'll not do (In DMC i can get better 
 numbers by just changing my allocator how can I go about it in D).
 
 Zz
 
 -----------------------------------------------
 - Default memory allocator
 dmc -o+all test.c
 Capacity C: 1048576  Count: 1000000
 
 ContextSwitches - 19954
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:06.562 ) UTime(   0:00:06.015 ) KTime(   0:00:00.375 )
 ITime(   0:00:03.828 )
 
 
 -----------------------------------------------
 dmc -DREPLACE_SYSTEM_ALLOCATOR -o+all test.c
 Capacity C: 1048576  Count: 1000000
 
 ContextSwitches - 3092
 First level fills = 0
 Second level fills = 0
 
 ETime(   0:00:01.250 ) UTime(   0:00:00.968 ) KTime(   0:00:00.250 )
 ITime(   0:00:00.796 )
 
 Note: I used dlmalloc Version pre-2.8.4 Wed Aug  2 14:13:56 2006 which 
 comes with nedmalloc.
 The current version is 2.8.3 and ptmalloc3 comes with an earlier version 
 of the one used above.
Dec 07 2006
parent reply zz <zz zz.com> writes:
Dave wrote:

 zz wrote:
 

 Walter Bright wrote:


 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.




 One thing that is happening is that the C++ code allocates memory, it 
 never frees it. The same with the D code.



 Not until it's out of scope (in this case main).


 
 With that in mind, I wrapped your original code in a loop for 10 
 iterations and decreased the inner loop to 100000.
 
 C++ (NedAlloc):
 Total Element count: 1000000
 5.438000
 
 D:
 Total Element count: 1000000
 1.360000
 


Cant' argue with that:
In this case D is faster I get:
C++ NedAlloc = 2.859
D = 1.421

But if you leave the original count and put in the outer loop for 10, 
D's performance becomes really bad.

c++ (NedMalloc) = 00:26.796
D = 02:23.375


 Quote: "While he liked that language and said the he might actually use 
 it to prototype idea's, he will not use it in production code due to the 
 performance."
 
 That sounds like me two years ago (and D hasn't gotten all that much 
 faster since then) <g>


Sill uses D to prototype idea's, but this was someone else who is new.


 Don't take this as some sort of attack -- you bring up some very good 
 points. And I agree that the GC performance should be looked at (and 
 that great performance is critical), but I'm wondering if D is getting a 
 fair shake in your shop?


On my side I've used it in small stuff that is running at clients sites, 
We handle a lot of data that comes from mainframes and AS/400 and it you 
need to process the text reports from those platforms a lot of string 
processing needs to be done, in one case I recall we needed to convert 
the data (over 1 gig of raw data) into XML and I was asked by my boss to 
see how we can get it done and who should write it, I one sitting I did 
a prototype in D and that was what was sent to the client (I don't think 
it could have been done faster), for this case and some others we never 
looked at performance since:
a) the client was happy just to get something.
b) most but not all large jobs involving processing are run overnight.

While it's only 2 of us who use D from time to time, there is a lot of 
respect for D at work but sometimes people put you to the test with 1 to 
1 examples and from there they might make their desicions on whether to 
use D or not.


 Even if you had to spend 10% development time to hoist some allocations 
 out of loops with D, if you save 20% developing the first-cut you'll end 
 up better off ;)


Yes, but this was a 1 to 1 test more on the line my gun is bigger than 
yours.

Zz
Dec 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
zz wrote:

 Dave wrote:

 zz wrote:


 Walter Bright wrote:


 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.




 One thing that is happening is that the C++ code allocates memory, 
 it never frees it. The same with the D code.



 Not until it's out of scope (in this case main).


 With that in mind, I wrapped your original code in a loop for 10 
 iterations and decreased the inner loop to 100000.

 C++ (NedAlloc):
 Total Element count: 1000000
 5.438000

 D:
 Total Element count: 1000000
 1.360000


 Cant' argue with that:
 In this case D is faster I get:
 C++ NedAlloc = 2.859
 D = 1.421
 
 But if you leave the original count and put in the outer loop for 10, 
 D's performance becomes really bad.
 
 c++ (NedMalloc) = 00:26.796
 D = 02:23.375


Try explicitly calling _gc.fullCollect() between iterations of the inner 
loop, and disabling the GC explicitly there.  Manipulation of the GC for 
performance-critical areas is an important and intentional feature.


Sean
Dec 08 2006
parent reply Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:

 zz wrote:

 Dave wrote:

 zz wrote:


 Walter Bright wrote:


 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.




 One thing that is happening is that the C++ code allocates memory, 
 it never frees it. The same with the D code.



 Not until it's out of scope (in this case main).


 With that in mind, I wrapped your original code in a loop for 10 
 iterations and decreased the inner loop to 100000.

 C++ (NedAlloc):
 Total Element count: 1000000
 5.438000

 D:
 Total Element count: 1000000
 1.360000


 Cant' argue with that:
 In this case D is faster I get:
 C++ NedAlloc = 2.859
 D = 1.421

 But if you leave the original count and put in the outer loop for 10, 
 D's performance becomes really bad.

 c++ (NedMalloc) = 00:26.796
 D = 02:23.375


 
 Try explicitly calling _gc.fullCollect() between iterations of the inner 
 loop, and disabling the GC explicitly there.  Manipulation of the GC for 
 performance-critical areas is an important and intentional feature.
 


With that in mind... What if some of the GC API were built-in to the language?
This could be one of 
those areas that would set D apart as a lower-level, performance orientated
language. If a GC 
implementation didn't support something, it would be stubbed for portability.
Likewise anything not 
covered explicitly by a built-in could be covered by an import.

Specifically:

gcFullCollect()
gcGenCollect()
gcDisable()
gcEnable()

or some such.

?


 
 Sean
Dec 08 2006
parent reply Sean Kelly <sean f4.ca> writes:
Dave wrote:

 Sean Kelly wrote:

 zz wrote:

 Dave wrote:

 zz wrote:


 Walter Bright wrote:


 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.




 One thing that is happening is that the C++ code allocates memory, 
 it never frees it. The same with the D code.



 Not until it's out of scope (in this case main).


 With that in mind, I wrapped your original code in a loop for 10 
 iterations and decreased the inner loop to 100000.

 C++ (NedAlloc):
 Total Element count: 1000000
 5.438000

 D:
 Total Element count: 1000000
 1.360000


 Cant' argue with that:
 In this case D is faster I get:
 C++ NedAlloc = 2.859
 D = 1.421

 But if you leave the original count and put in the outer loop for 10, 
 D's performance becomes really bad.

 c++ (NedMalloc) = 00:26.796
 D = 02:23.375


 Try explicitly calling _gc.fullCollect() between iterations of the 
 inner loop, and disabling the GC explicitly there.  Manipulation of 
 the GC for performance-critical areas is an important and intentional 
 feature.


 
 With that in mind... What if some of the GC API were built-in to the 
 language? This could be one of those areas that would set D apart as a 
 lower-level, performance orientated language. If a GC implementation 
 didn't support something, it would be stubbed for portability. Likewise 
 anything not covered explicitly by a built-in could be covered by an 
 import.
 
 Specifically:
 
 gcFullCollect()
 gcGenCollect()
 gcDisable()
 gcEnable()
 
 or some such.


Phobos already has std.gc for exactly this purpose.  Or are you 
suggesting some of these features should be added as keywords, or 
perhaps automatically available, similar to the declaration of Object 
and Exception?


Sean
Dec 08 2006
parent Dave <Dave_member pathlink.com> writes:
Sean Kelly wrote:

 Dave wrote:

 Sean Kelly wrote:

 Try explicitly calling _gc.fullCollect() between iterations of the 
 inner loop, and disabling the GC explicitly there.  Manipulation of 
 the GC for performance-critical areas is an important and intentional 
 feature.


 With that in mind... What if some of the GC API were built-in to the 
 language? This could be one of those areas that would set D apart as a 
 lower-level, performance orientated language. If a GC implementation 
 didn't support something, it would be stubbed for portability. 
 Likewise anything not covered explicitly by a built-in could be 
 covered by an import.

 Specifically:

 gcFullCollect()
 gcGenCollect()
 gcDisable()
 gcEnable()

 or some such.


 
 Phobos already has std.gc for exactly this purpose.  Or are you 
 suggesting some of these features should be added as keywords, or 
 perhaps automatically available, similar to the declaration of Object 
 and Exception?
 


Yes, that way those optimization 'hints' would be codified into the language
itself (like register 
in C or inline in C++).

The goal would be to lower the barrier as much as possible ('encourage the use
of') and give D 
something codified that most other languages don't have.

You know, taking this one step further for convenience and safety could give us:

Replace:

 gcDisable()
 gcEnable()




with a built-in for:
     std.gc.disable;
     scope(exit) std.gc.enable;

usage:

void foo()
{
     scopeGcDisable;
     for(...){ ... }
}


 
 Sean
Dec 08 2006
prev sibling parent Walter Bright <newshound digitalmars.com> writes:
zz wrote:

 Walter Bright wrote:

 zz wrote:


  From my point of view these tests are not really nessesary on my 
 side since I still continue using D and I belive that someday the 
 memory stuff will be optimized.



 One thing that is happening is that the C++ code allocates memory, it 
 never frees it. The same with the D code.


 
 Not until it's out of scope (in this case main).


True, but it doesn't free any memory inside the loop.


 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs 
 a collection cycle.


 
 Would something like dlmalloc by Doug Lea make a difference in D, 
 someone suggested trying it out so I tested it using DMC on some C code 
 and there was a big difference the two compiled versions (Numbers are 
 below and I was supprised).


You can call any C function from D, so if you want to explicitly manage 
memory using dlmalloc, that is certainly possible. D allows overriding 
new/delete on a per class/per struct basis, but such won't be garbage 
collected if you do so.


 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.

 To get better numbers, one can add calls to gc.disable() and 
 gc.enable() to tell the gc that there is no point to running a 
 collection cycle for this section of code.


 
 The above is something that, i'll not do (In DMC i can get better 
 numbers by just changing my allocator how can I go about it in D).


I'm not sure why you wouldn't want to do it, it is much more localized 
in effect than swapping out global operators new/delete.

The gc in D is pluggable, but nobody has written a different one to plug 
  in yet.
Dec 07 2006
prev sibling parent Reiner Pope <xxxxxx xxx.xx> writes:
Walter Bright wrote:

 zz wrote:

  From my point of view these tests are not really nessesary on my side 
 since I still continue using D and I belive that someday the memory 
 stuff will be optimized.


 
 One thing that is happening is that the C++ code allocates memory, it 
 never frees it. The same with the D code.
 
 What happens with a garbage collector is it gets new chunks of memory 
 from the operating system as it needs it. But before it does, it runs a 
 collection cycle.
 
 So in the D version, it will be running probably several collection 
 cycles (accomplishing nothing), while the C++ version does not. This 
 will make the gc version slower.
 
 To get better numbers, one can add calls to gc.disable() and gc.enable() 
 to tell the gc that there is no point to running a collection cycle for 
 this section of code.


Has anyone looked into analysis of D code to automatically insert 
frees/deletes in pertinent places, so that GC is required less often? 
The good thing about allowing this sort of analysis is that no harm is 
done if you don't find anything (since GC will still guarantee it is 
eventually collected), but it could yield benefits in certain places 
where deterministic deletion is possible.

Cheers,

Reiner
Dec 07 2006
prev sibling parent reply clayasaurus <clayasaurus gmail.com> writes:
Pragma wrote:

 Something I ran into that the group might enjoy:
 
 From: http://www.techreview.com/InfoTech/17831/
 
 "Bjarne Stroustrup, the inventor of the C++ programming language, 
 defends his legacy and examines what's wrong with most software code."
 
 As always, Slashdot has some colorful coverage on this:
 
 http://it.slashdot.org/it/06/12/05/0045234.shtml
 
 "MIT's Technology Review has a Q&A with C++ inventor Bjarne Stroustrup. 
 Highlights include Bjarne's answers on the trade-offs involved in the 
 design of C++, and how they apply today, and his thoughts on the 
 solution to the problems. From the interview: 'Software developers have 
 become adept at the difficult art of building reasonably reliable 
 systems out of unreliable parts. The snag is that often we do not know 
 exactly how we did it.'"
 


Here is the follow up interview

http://www.techreview.com/InfoTech/17868/page1/

http://developers.slashdot.org/article.pl?sid=06/12/09/220218
Dec 09 2006
parent Bruno Medeiros <brunodomedeiros+spam com.gmail> writes:
clayasaurus wrote:

 Pragma wrote:

 Something I ran into that the group might enjoy:

 From: http://www.techreview.com/InfoTech/17831/

 "Bjarne Stroustrup, the inventor of the C++ programming language, 
 defends his legacy and examines what's wrong with most software code."

 As always, Slashdot has some colorful coverage on this:

 http://it.slashdot.org/it/06/12/05/0045234.shtml

 "MIT's Technology Review has a Q&A with C++ inventor Bjarne 
 Stroustrup. Highlights include Bjarne's answers on the trade-offs 
 involved in the design of C++, and how they apply today, and his 
 thoughts on the solution to the problems. From the interview: 
 'Software developers have become adept at the difficult art of 
 building reasonably reliable systems out of unreliable parts. The snag 
 is that often we do not know exactly how we did it.'"


 
 Here is the follow up interview
 
 http://www.techreview.com/InfoTech/17868/page1/
 
 http://developers.slashdot.org/article.pl?sid=06/12/09/220218
 


Reading that, it has just sprung in my mind that Bjarne's initials spell 
BS... ;P

-- 
Bruno Medeiros - MSc in CS/E student
http://www.prowiki.org/wiki4d/wiki.cgi?BrunoMedeiros#D
Dec 11 2006