www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - D in the TIOBE top

reply "Paulo Pinto" <pjmlp progtools.org> writes:
D just entered into the top 20!

http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

I know that TIOBE's value is relative, but this is the type of charts
that managers usually understand.

--
Paulo
Sep 09 2011
next sibling parent d coder <dlang.coder gmail.com> writes:
From the Article ....





 Wikipedia gives a hint about the comeback of D: "The release of Andrei
 Alexandrescu's book 'The D Programming Language' on June 12, 2010 marked the
 stabilization of D 2.0." In the charts we can see that the D language is
 gradually rising again since that date.



Should we look forward to more books on D now?

Regards
- Puneet
Sep 09 2011
prev sibling next sibling parent Caligo <iteronvexor gmail.com> writes:
D will probably get even more attention once GCC 4.7 Stage 3 begins in a few
months (assuming D is going to be part of GCC).
Sep 09 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/10/11 12:50 AM, Paulo Pinto wrote:

 D just entered into the top 20!

 http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

 I know that TIOBE's value is relative, but this is the type of charts
 that managers usually understand.


Now on reddit:

http://www.reddit.com/r/reddit.com/comments/kaxgq/the_d_programming_language_is_back_in_tiobes_top/


Andrei
Sep 09 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/10/11 1:44 AM, Andrei Alexandrescu wrote:

 On 9/10/11 12:50 AM, Paulo Pinto wrote:

 D just entered into the top 20!

 http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

 I know that TIOBE's value is relative, but this is the type of charts
 that managers usually understand.


 Now on reddit:

 http://www.reddit.com/r/reddit.com/comments/kaxgq/the_d_programming_language_is_back_in_tiobes_top/


Posted in the wrong place, deleted and reposted here:

http://www.reddit.com/r/programming/comments/kaxjq/the_d_programming_language_is_back_in_tiobes_top/


Andrei
Sep 09 2011
parent Caligo <iteronvexor gmail.com> writes:
Weird, the september headline is gone now, and it's showing august headline
again.
Sep 10 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Paulo Pinto:


 I know that TIOBE's value is relative,


Well, its value is minimal.



 but this is the type of charts
 that managers usually understand.


I think most managers only see the top 11 of that list I am seeing now (some
managers probably see some language below too, like Delphi, Ada, Scala, etc,
but they are uncommon).

Bye,
bearophile
Sep 10 2011
next sibling parent reply Caligo <iteronvexor gmail.com> writes:
I don't understand why some would down vote such a news. a lot of haters out
there I suppose.
Sep 11 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/11/2011 10:28 PM, Caligo wrote:

 I don't understand why some would down vote such a news. a lot of haters out
 there I suppose.


Probably because Tiobe is a piece of shit when it comes to accuracy, so 
crowing about your "top 20" Tiobe number brings out the hate for lame 
marketing.
Sep 12 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/12/11 10:19 AM, Jeff Nowakowski wrote:

 On 09/11/2011 10:28 PM, Caligo wrote:

 I don't understand why some would down vote such a news. a lot of
 haters out
 there I suppose.


 Probably because Tiobe is a piece of shit when it comes to accuracy, so
 crowing about your "top 20" Tiobe number brings out the hate for lame
 marketing.


Accuracy (i.e. figuring the "right" ranking) is probably indeed low, but 
Tiobe's methods are a reasonable proxy for a language's presence. I see 
the increase of D's Tiobe rank not as a surprise or some random 
artifact, but a reflection of the strong progress in the past months.

Andrei
Sep 12 2011
next sibling parent Don <nospam nospam.com> writes:
On 12.09.2011 17:43, Andrei Alexandrescu wrote:

 On 9/12/11 10:19 AM, Jeff Nowakowski wrote:

 On 09/11/2011 10:28 PM, Caligo wrote:

 I don't understand why some would down vote such a news. a lot of
 haters out
 there I suppose.


 Probably because Tiobe is a piece of shit when it comes to accuracy, so
 crowing about your "top 20" Tiobe number brings out the hate for lame
 marketing.


 Accuracy (i.e. figuring the "right" ranking) is probably indeed low, but
 Tiobe's methods are a reasonable proxy for a language's presence. I see
 the increase of D's Tiobe rank not as a surprise or some random
 artifact, but a reflection of the strong progress in the past months.

 Andrei


If you look at the graph, it's clear that about half of the increase 
relative to last month is just a noise spike. D will drop out of the top 
20 next month. Nonetheless there does seem to be a small yet continuous 
upwards trend; it should be reliably in the top 20 within six months time.
Sep 12 2011
prev sibling next sibling parent Steve Teale <steve.teale britseyeview.com> writes:
Agreed Andrei,

As you may have noticed, I've been visiting D at maybe 6 month intervals for
some
time.

Suddenly it seems real.  There are alternative implementations, the library
looks
like something quite comprehensive, and we're more or less back to there being
one
version. Like I said - "I just wanted to say..."

Walter,

Thanks for the retrospective compliment. Coming from you, that means a lot ;=)

Steve
Sep 12 2011
prev sibling parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/12/2011 11:43 AM, Andrei Alexandrescu wrote:

 Accuracy (i.e. figuring the "right" ranking) is probably indeed low, but
 Tiobe's methods are a reasonable proxy for a language's presence. I see
 the increase of D's Tiobe rank not as a surprise or some random
 artifact, but a reflection of the strong progress in the past months.


D is prone to overcounting because it's common name matches lots of 
unrelated stuff, including things like 3-D programming. The Tiobe folks 
can't be counted on to come up with an accurate measure to filter it 
out. The latest spike could be for any number of reasons. You might as 
well be reading tea leaves.


just as mainstream than D is, and very likely much more. Want some 

There are over a dozen Scala books on the market. Stack overflow shows 
5,606 tagged with "scala", and 430 questions tagged with "d".

Lame marketing, Andrei. But hey, it works, right?
Sep 12 2011
next sibling parent Caligo <iteronvexor gmail.com> writes:
On Mon, Sep 12, 2011 at 1:38 PM, Jeff Nowakowski <jeff dilacero.org> wrote:


 On 09/12/2011 11:43 AM, Andrei Alexandrescu wrote:


 Accuracy (i.e. figuring the "right" ranking) is probably indeed low, but
 Tiobe's methods are a reasonable proxy for a language's presence. I see
 the increase of D's Tiobe rank not as a surprise or some random
 artifact, but a reflection of the strong progress in the past months.


 D is prone to overcounting because it's common name matches lots of
 unrelated stuff, including things like 3-D programming. The Tiobe folks
 can't be counted on to come up with an accurate measure to filter it out.
 The latest spike could be for any number of reasons. You might as well be
 reading tea leaves.


 as mainstream than D is, and very likely much more. Want some stats?

 over a dozen Scala books on the market. Stack overflow shows 5,606 tagged
 with "scala", and 430 questions tagged with "d".

 Lame marketing, Andrei. But hey, it works, right?


The lame marketing is when programming languages need big corporate money
and power to get them to any level of popularity.  Did you forget what
happened with Google Go?  Not a language that's any better than D, but because
of Google now everyone knows about it.  No different with Scala, Java, etc.

When was the last time we had something like Scalathon for D with corporate
sponsors such as Google?
Sep 12 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/12/2011 01:38 PM, Jeff Nowakowski wrote:

 On 09/12/2011 11:43 AM, Andrei Alexandrescu wrote:

 Accuracy (i.e. figuring the "right" ranking) is probably indeed low, but
 Tiobe's methods are a reasonable proxy for a language's presence. I see
 the increase of D's Tiobe rank not as a surprise or some random
 artifact, but a reflection of the strong progress in the past months.


 D is prone to overcounting because it's common name matches lots of
 unrelated stuff, including things like 3-D programming. The Tiobe folks
 can't be counted on to come up with an accurate measure to filter it
 out. The latest spike could be for any number of reasons. You might as
 well be reading tea leaves.


 just as mainstream than D is, and very likely much more. Want some

 There are over a dozen Scala books on the market. Stack overflow shows
 5,606 tagged with "scala", and 430 questions tagged with "d".

 Lame marketing, Andrei. But hey, it works, right?


I have no connection with Tiobe and the name "D" wasn't chosen to 
manipulate its ranking method. Are you disagreeing with my posting of a 
piece of information on a community-voted forum? I wouldn't quite 
understand what the matter is here, if it weren't for your posting history.


Thanks,

Andrei
Sep 12 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/12/2011 04:25 PM, Andrei Alexandrescu wrote:

 I have no connection with Tiobe and the name "D" wasn't chosen to
 manipulate its ranking method. Are you disagreeing with my posting of a
 piece of information on a community-voted forum? I wouldn't quite
 understand what the matter is here, if it weren't for your posting history.


Nice strawman. I never said you chose the D name. I made it clear why I 
think using Tiobe to crow about D is just lame marketing. What's silly 
is framing marketing as "posting of a piece of information on a 
community-voted forum". That's taking generics to a whole new level!
Sep 12 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/12/2011 05:48 PM, Jeff Nowakowski wrote:

 On 09/12/2011 04:25 PM, Andrei Alexandrescu wrote:

 I have no connection with Tiobe and the name "D" wasn't chosen to
 manipulate its ranking method. Are you disagreeing with my posting of a
 piece of information on a community-voted forum? I wouldn't quite
 understand what the matter is here, if it weren't for your posting
 history.


 Nice strawman. I never said you chose the D name. I made it clear why I
 think using Tiobe to crow about D is just lame marketing.


It's not all that clear to me. I see that post as an attempt at inflaming.


 What's silly
 is framing marketing as "posting of a piece of information on a
 community-voted forum". That's taking generics to a whole new level!


That's pretty much what I did. It's very simple, really - the people who 
maintain the Tiobe index wrote a few good words about D in their monthly 
installment for September. I found fit to share that on reddit. I fail 
to see where I've done something lame or of questionable intent or 
whatnot. If you were in my position, wouldn't you have posted that 
article? I don't understand what you're trying to prove.

BTW, apparently others comment on the news, too. 
http://www.infoworld.com/d/application-development/objective-c-c-d-language-winners-in-programmin
-popularity-172587. 
I find it odd that you hang out on this newsgroup although every 
indication of D's success infuriates you.


Andrei
Sep 12 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

 If you were in my position, wouldn't you have posted that
 article?


I hope I wouldn't have published it, since I don't like misleading 
marketing.


 BTW, apparently others comment on the news, too.
 http://www.infoworld.com/d/application-development/objective-c-c-d-language-winners-in-programming-popularity-172587.


Then he doesn't know better about the problems with Tiobe. You do, or at 
least you should. D originally fell out of the top 20 because it 
plummeted after Tiobe actually started applying a tiny bit of care to 
their stats. You also acknowledged in the reddit thread that you thought 
Tiobe was meaningless -- that is, until D hit the front page. Smiley or 
not, I'll take that at face value given the context.


 I find it odd that you hang out on this newsgroup although every
 indication of D's success infuriates you.


Wrong. I follow D because I find it an interesting language. Rather than 
being upset at a successful D, I've been hoping it would turn out great 
after all these years. Why else would I make a comment supporting 
bearophile about error-prone behavior like array initialization?

I criticize when I find things to critique, usually when they hit my pet 
peeve issues. Bullshit marketing happens to be one of them, and Tiobe is 
right up there.
Sep 12 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/12/2011 07:11 PM, Jeff Nowakowski wrote:

 On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

 If you were in my position, wouldn't you have posted that
 article?


 I hope I wouldn't have published it, since I don't like misleading
 marketing.


 BTW, apparently others comment on the news, too.
 http://www.infoworld.com/d/application-development/objective-c-c-d-language-winners-in-programming-popularity-172587.


 Then he doesn't know better about the problems with Tiobe. You do, or at
 least you should. D originally fell out of the top 20 because it
 plummeted after Tiobe actually started applying a tiny bit of care to
 their stats. You also acknowledged in the reddit thread that you thought
 Tiobe was meaningless -- that is, until D hit the front page. Smiley or
 not, I'll take that at face value given the context.


Too bad, you missed out on a good joke. Feigning lack of a sense of 
humor for the sake of appearing self-righteous is a losing gambit as a 
sense of humor indicates intelligence.

Andrei
Sep 12 2011
prev sibling parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Mon, 12 Sep 2011 20:11:57 -0400, Jeff Nowakowski <jeff dilacero.org>  
wrote:


 On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

 If you were in my position, wouldn't you have posted that
 article?


 I hope I wouldn't have published it, since I don't like misleading  
 marketing.


Since when is marketing *ever* not misleading?  I've yet to see the  
marketing campaign where Coke or Pepsi touts how their products make you  
fat.

Watch the movie Crazy People.  It's a good movie about being truthful in  
advertising :)

-Steve
Sep 13 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/13/2011 8:07 AM, Steven Schveighoffer wrote:

 Since when is marketing *ever* not misleading?


I don't think that's quite a fair statement. Good marketing informs a customer 
about a product that may be just the thing for his needs. With any product, 
there are different ways to view it, positive and negative, and they aren't 
wrong or misleading, just different.

As for popularity of a programming language, no two people will even agree on a 
definition of it, let alone an accurate means of measuring it. Tiobe explains 
their methodology, and readers are free to interpret it as they see fit.

I don't see that as misleading.
Sep 13 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Sep 2011 15:47:39 -0400, Walter Bright  
<newshound2 digitalmars.com> wrote:


 On 9/13/2011 8:07 AM, Steven Schveighoffer wrote:

 Since when is marketing *ever* not misleading?


 I don't think that's quite a fair statement. Good marketing informs a  
 customer about a product that may be just the thing for his needs. With  
 any product, there are different ways to view it, positive and negative,  
 and they aren't wrong or misleading, just different.


Oh, come on :)  There's positive and negative traits to everything.   
Marketing touts the positive and ignores the negatives.  How is that not  
misleading?

How many beer commercials show people getting into drunk driving  
accidents?  How many soda commercials show a 300 lb. hacker at his laptop  
with 15 empty soda cans around him?  Almost every miracle diet cure shows  
testimonials with the disclaimer "results not typical".  Marketing is  
misleading *on purpose*, it is your job as the consumer to not believe  
anything at its word, and to try and find the other side of the story.

If something is not misleading, it's generally published by an objective  
third party.  If anything, I'd say tiobe is the *least* misleading  
marketing that D could have.

Most marketing is *truthful*, meaning the statements are true, or at least  
could be true.  But it's typically partial truths, or skewed truth.


 As for popularity of a programming language, no two people will even  
 agree on a definition of it, let alone an accurate means of measuring  
 it. Tiobe explains their methodology, and readers are free to interpret  
 it as they see fit.

 I don't see that as misleading.


As interpreted by Mr. Nowakowski, it's misleading to say that D is good  
because some objective language popularity measurement shows D is gaining  
ground.  I say, that's just normal marketing.  Even if it's not a true  
accurate measurement, it is a report somebody published, and why not take  
advantage of it?

-Steve
Sep 14 2011
next sibling parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 07:23 AM, Steven Schveighoffer wrote:

 As interpreted by Mr. Nowakowski, it's misleading to say that D is good
 because some objective language popularity measurement shows D is
 gaining ground.


Tiobe is not objective. There is a "manually determined confidence 
factor", and also some arbitrary numbers like "The first 100 pages per 
search engine are checked for possible false positives and this is used 
to define the confidence factor."

This explains why D is ranked so much higher than Scala, despite other 
statistics that show the reverse. A lot of the early hits about D and 
programming may actually be about D, but that doesn't mean all the rest 
will be. Because D is so common, you're going to start getting junk 
results the further out you go.


 I say, that's just normal marketing. Even if it's not a
 true accurate measurement, it is a report somebody published, and why
 not take advantage of it?


Because you have some integrity?
Sep 14 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Wed, 14 Sep 2011 11:20:03 -0400, Jeff Nowakowski <jeff dilacero.org>  
wrote:


 On 09/14/2011 07:23 AM, Steven Schveighoffer wrote:

 As interpreted by Mr. Nowakowski, it's misleading to say that D is good
 because some objective language popularity measurement shows D is
 gaining ground.


 Tiobe is not objective. There is a "manually determined confidence  
 factor", and also some arbitrary numbers like "The first 100 pages per  
 search engine are checked for possible false positives and this is used  
 to define the confidence factor."

 This explains why D is ranked so much higher than Scala, despite other  
 statistics that show the reverse. A lot of the early hits about D and  
 programming may actually be about D, but that doesn't mean all the rest  
 will be. Because D is so common, you're going to start getting junk  
 results the further out you go.


It's objective in that the results/methodology are not picked to favor D.   
Whether it's *accurate* or not is certainly up for debate.

But what makes your statistics more "valid" than Tiobe's?  As Walter said,  
there are multiple ways to measure popularity, and every one is guaranteed  
to dissatisfy some people.


 I say, that's just normal marketing. Even if it's not a
 true accurate measurement, it is a report somebody published, and why
 not take advantage of it?


 Because you have some integrity?


I fail to see why quoting a published article that says some measured  
statistics show that D is gaining popularity lacks integrity.  People *do*  
take Tiobe seriously, for whatever reason.  Why should we not use that in  
our favor?

For instance, let's say someone starts to use D at a university somewhere,  
maybe even at a university that isn't a very highly ranked technical  
university.  Does it lack integrity to say "hey look, D is being used in  
universities!"?  I mean, marketing is promotion, it's not criticism.

I don't know, I think this discussion isn't going anywhere.  Signing off.

-Steve
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 12:48 PM, Steven Schveighoffer wrote:

 But what makes your statistics more "valid" than Tiobe's? As Walter
 said, there are multiple ways to measure popularity, and every one is
 guaranteed to dissatisfy some people.


I demonstrated the fundamental problem with their methodology. They use 
only the first 100 results to get statistics for a distribution that 
will have the vast majority of the false positives past the initial 
results. It's one thing to say "number of web pages about X" is an 
arguable measure of popularity, it's quite another to use a profoundly 
flawed way to get that number.

The statistics I quoted were varied, simple, and easy to verify. Come up 
with alternative evidence instead of just using a broad brush to make 
all measures equal. Clearly some languages are more popular than others.
Sep 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/14/2011 02:21 PM, Jeff Nowakowski wrote:

 On 09/14/2011 12:48 PM, Steven Schveighoffer wrote:

 But what makes your statistics more "valid" than Tiobe's? As Walter
 said, there are multiple ways to measure popularity, and every one is
 guaranteed to dissatisfy some people.


 I demonstrated the fundamental problem with their methodology.


Words like "demonstrated" or "clarified" are a bit exaggerated within 
this discussion, as is "flawed" when describing what Tiobe does. Not to 
mention the implication that I lack integrity, which is quite a bit out 
of character even by the standard set by the corrosiveness of your past 
posts. Is defending your view of Tiobe worth all this?


 The statistics I quoted were varied, simple, and easy to verify.


They're not ground truth; for example, it's known there is a lot of 
noise in Google's estimate number of pages. For example, searching for 
the exact phrase "scala programming" yields 149,000 estimated results, 
whereas searching for the exact phrase "scala programming language" 
yields 418,000 results. This is incorrect because the latter query 
includes the former. To some consolation, the relationship between the 
estimated result set sizes for e.g. "d programming" and "d programming 
language" looks more plausible (343,000 vs. 199,000).


Andrei
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 04:27 PM, Andrei Alexandrescu wrote:

 Words like "demonstrated" or "clarified" are a bit exaggerated within
 this discussion, as is "flawed" when describing what Tiobe does.


Only if you are delusional and can't recognize the fundamental problem 
with search results getting worse and worse the farther out you go (who 
hasn't experience this with their everyday searches?), and then only 
looking at the first 100 results to get a simple percentage for the 
whole distribution. On top of this, D matches all kinds of stuff, 
whereas a name like Scala does not.

I also gave some simple and hard stats that agreed with each other about 
just how lopsided the results for Scala were for D, yet D is at Tiobe 



 Not to mention the implication that I lack integrity, which is quite
 a bit out of character even by the standard set by the corrosiveness
 of your past posts. Is defending your view of Tiobe worth all this?


The way you keep on defending Tiobe isn't helping. There have also been 
other instances in the past where I've found less than savory 
statements, and I've already mentioned these before.


 The statistics I quoted were varied, simple, and easy to verify.


 They're not ground truth; for example, it's known there is a lot of
 noise in Google's estimate number of pages.


And the Amazon sales rank? And the number of books available? And the 
number of questions on Stack Overflow? Should I start looking for number 
of jobs next? I'm willing to be you lose there, too.


 For example, searching for the exact phrase "scala programming"
 yields 149,000 estimated results, whereas searching for the exact
 phrase "scala programming language" yields 418,000 results. This is
 incorrect because the latter query includes the former. To some
 consolation, the relationship between the estimated result set sizes
 for e.g. "d programming" and "d programming language" looks more
 plausible (343,000 vs. 199,000).



What you've done is demonstrated that Tiobe's method is even more 
flawed. You need to sample the distribution to get valid results, not 
just the first 100 pages.
Sep 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/14/11 3:52 PM, Jeff Nowakowski wrote:

 On 09/14/2011 04:27 PM, Andrei Alexandrescu wrote:

 Words like "demonstrated" or "clarified" are a bit exaggerated within
 this discussion, as is "flawed" when describing what Tiobe does.


 Only if you are delusional


(guess that's strike three)


 and can't recognize the fundamental problem
 with search results getting worse and worse the farther out you go (who
 hasn't experience this with their everyday searches?), and then only
 looking at the first 100 results to get a simple percentage for the
 whole distribution. On top of this, D matches all kinds of stuff,
 whereas a name like Scala does not.

 I also gave some simple and hard stats that agreed with each other about
 just how lopsided the results for Scala were for D, yet D is at Tiobe



 Not to mention the implication that I lack integrity, which is quite
 a bit out of character even by the standard set by the corrosiveness
 of your past posts. Is defending your view of Tiobe worth all this?


 The way you keep on defending Tiobe isn't helping.


This is not about defending Tiobe. All in all, for a given language 
ascending on Tiobe is a good sign. This has nothing to do with the 
relative rank of languages.


 There have also been
 other instances in the past where I've found less than savory
 statements, and I've already mentioned these before.


 The statistics I quoted were varied, simple, and easy to verify.


 They're not ground truth; for example, it's known there is a lot of
 noise in Google's estimate number of pages.


 And the Amazon sales rank? And the number of books available? And the
 number of questions on Stack Overflow? Should I start looking for number
 of jobs next? I'm willing to be you lose there, too.


One issue here is that the relative rank of Scala and D on Tiobe has 
become a straw man. Let's not forget how it started: I posted Tiobe's 
September newsletter to reddit and you considered it lame marketing. 
Well I disagree.

It seems this discussion has well outlived its usefulness, as 
unfortunately have ours in the past.


Andrei
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 05:41 PM, Andrei Alexandrescu wrote:

 One issue here is that the relative rank of Scala and D on Tiobe has
 become a straw man. Let's not forget how it started: I posted Tiobe's
 September newsletter to reddit and you considered it lame marketing.
 Well I disagree.


It's not a straw man. The only reason the D was even mentioned on Tiobe 

which by any measure that aren't bullshit Tiobe statistics it should be, 
then you would have nothing to post.

The Tiobe thing irked me, but I wasn't even going to say anything until 

posting here about all the "haters" who downvoted this "news". It's 
misinformation, not news.
Sep 14 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/15/2011 01:48 AM, Jeff Nowakowski wrote:

 On 09/14/2011 05:41 PM, Andrei Alexandrescu wrote:

 One issue here is that the relative rank of Scala and D on Tiobe has
 become a straw man. Let's not forget how it started: I posted Tiobe's
 September newsletter to reddit and you considered it lame marketing.
 Well I disagree.


 It's not a straw man. The only reason the D was even mentioned on Tiobe

 which by any measure that aren't bullshit Tiobe statistics it should be,
 then you would have nothing to post.


Scala's ranking is nothing but irrelevant for this discussion. The fact 
Andrei was sharing is that the D programming language is in a 
significantly better position than the D programming language one year ago.


 The Tiobe thing irked me, but I wasn't even going to say anything until

 posting here about all the "haters" who downvoted this "news". It's
 misinformation, not news.


What about the fact that D entered the top 20 at TIOBE is misinformation?
Sep 14 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/14/11 6:48 PM, Jeff Nowakowski wrote:

 On 09/14/2011 05:41 PM, Andrei Alexandrescu wrote:

 One issue here is that the relative rank of Scala and D on Tiobe
 has become a straw man. Let's not forget how it started: I posted
 Tiobe's September newsletter to reddit and you considered it lame
 marketing. Well I disagree.


 It's not a straw man. The only reason the D was even mentioned on



It did so by ascending 8 positions without Tiobe having changed its
algorithm. Scala's position being computed wrongly or correctly well
below all 8 positions jumped has at best very little to do with that.



 that aren't bullshit Tiobe statistics it should be, then you would
 have nothing to post.


Well, the problem here is that you mix an argument and its negation
in an illogical way. You simultaneously assume that Tiobe got D's



 The Tiobe thing irked me, but I wasn't even going to say anything

 Caligo posting here about all the "haters" who downvoted this
 "news".


Wonder why you took that out on me and others. One thing I try to do is 
not get in the position of saying things that I'm sorry about later on. 
You can be sure that I didn't quite warm up to your misplaced 
self-righteousness and inappropriate words, but I did my best to not let 
that make me respond in kind. You should know you'd do good to apologize.


 It's misinformation, not news.


D is up 8 positions on Tiobe. That's news to me, and good news at that.


Andrei
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 08:36 PM, Andrei Alexandrescu wrote:

 Well, the problem here is that you mix an argument and its negation
 in an illogical way. You simultaneously assume that Tiobe got D's



Yet you keep on ignoring the obvious reason why D's stats are so wrong. 
I didn't "assume" it, I gave obvious reasons for it.


 Wonder why you took that out on me and others. One thing I try to do is
 not get in the position of saying things that I'm sorry about later on.


Considering that Tiobe has a bad reputation which has been mentioned 
both on this board and on reddit multiple times, and both bearophile and 
Don had reservations about the Tiobe "news", I think I have some 
justification in saying should have shown some restraint in touting Tiobe.
Sep 14 2011
next sibling parent Josh Simmons <simmons.44 gmail.com> writes:
On Thu, Sep 15, 2011 at 11:29 AM, Jeff Nowakowski <jeff dilacero.org> wrote:

 On 09/14/2011 08:36 PM, Andrei Alexandrescu wrote:

 Well, the problem here is that you mix an argument and its negation
 in an illogical way. You simultaneously assume that Tiobe got D's



 Yet you keep on ignoring the obvious reason why D's stats are so wrong. I
 didn't "assume" it, I gave obvious reasons for it.


 Wonder why you took that out on me and others. One thing I try to do is
 not get in the position of saying things that I'm sorry about later on.


 Considering that Tiobe has a bad reputation which has been mentioned both on
 this board and on reddit multiple times, and both bearophile and Don had
 reservations about the Tiobe "news", I think I have some justification in
 saying should have shown some restraint in touting Tiobe.


I demand a refund.
Sep 14 2011
prev sibling parent reply Don <nospam nospam.com> writes:
On 15.09.2011 03:29, Jeff Nowakowski wrote:

 On 09/14/2011 08:36 PM, Andrei Alexandrescu wrote:

 Well, the problem here is that you mix an argument and its negation
 in an illogical way. You simultaneously assume that Tiobe got D's



 Yet you keep on ignoring the obvious reason why D's stats are so wrong.
 I didn't "assume" it, I gave obvious reasons for it.


 Wonder why you took that out on me and others. One thing I try to do is
 not get in the position of saying things that I'm sorry about later on.


 Considering that Tiobe has a bad reputation which has been mentioned
 both on this board and on reddit multiple times, and both bearophile and
 Don had reservations about the Tiobe "news", I think I have some
 justification in saying should have shown some restraint in touting Tiobe.


My take on this: Tiobe stats are of doubtful merit. They have a very 
large month-to-month noise component, and it's not even clear what 
they're measuring. But their headline for the month was about D. This is 
good publicity for us.
At the same time, D has achieved a much greater level of maturity.
I suspect the correlation between the two is very weak. But it doesn't 
really matter. We should regard the Tiobe thing as free marketing, 
rather than an indicator of how well we're doing.

The best indicator of how we're doing is how many active contributors we 
have. A high web profile only matters insofar as it attracts 
contributors. And I think that's true for any language which is not 
mainstream. Only once you have a very mature product, and you're 
competing for market share, do Tiobe-style metrics matter.

BTW "Go" must also have terrible quality statistics on Tiobe.
Sep 14 2011
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/15/2011 01:21 AM, Don wrote:

 On 15.09.2011 03:29, Jeff Nowakowski wrote:

 On 09/14/2011 08:36 PM, Andrei Alexandrescu wrote:

 Well, the problem here is that you mix an argument and its negation
 in an illogical way. You simultaneously assume that Tiobe got D's



 Yet you keep on ignoring the obvious reason why D's stats are so wrong.
 I didn't "assume" it, I gave obvious reasons for it.


 Wonder why you took that out on me and others. One thing I try to do is
 not get in the position of saying things that I'm sorry about later on.


 Considering that Tiobe has a bad reputation which has been mentioned
 both on this board and on reddit multiple times, and both bearophile and
 Don had reservations about the Tiobe "news", I think I have some
 justification in saying should have shown some restraint in touting
 Tiobe.


 My take on this: Tiobe stats are of doubtful merit. They have a very
 large month-to-month noise component, and it's not even clear what
 they're measuring. But their headline for the month was about D. This is
 good publicity for us.
 At the same time, D has achieved a much greater level of maturity.
 I suspect the correlation between the two is very weak. But it doesn't
 really matter. We should regard the Tiobe thing as free marketing,
 rather than an indicator of how well we're doing.

 The best indicator of how we're doing is how many active contributors we
 have. A high web profile only matters insofar as it attracts
 contributors. And I think that's true for any language which is not
 mainstream. Only once you have a very mature product, and you're
 competing for market share, do Tiobe-style metrics matter.

 BTW "Go" must also have terrible quality statistics on Tiobe.


Shut up and go programming! :o)

Andrei
Sep 15 2011
prev sibling next sibling parent "Jonathan M Davis" <jmdavisProg gmx.com> writes:
On Wednesday, September 14, 2011 16:48 Jeff Nowakowski wrote:

 On 09/14/2011 05:41 PM, Andrei Alexandrescu wrote:

 One issue here is that the relative rank of Scala and D on Tiobe has
 become a straw man. Let's not forget how it started: I posted Tiobe's
 September newsletter to reddit and you considered it lame marketing.
 Well I disagree.


 
 It's not a straw man. The only reason the D was even mentioned on Tiobe

 which by any measure that aren't bullshit Tiobe statistics it should be,
 then you would have nothing to post.
 
 The Tiobe thing irked me, but I wasn't even going to say anything until

 posting here about all the "haters" who downvoted this "news". It's
 misinformation, not news.


Tiobe attempts to measure the popularity of a language. It's an indicator of 
language popularity which is, on some level, paid attention to. So, the fact 
that D has moved up in the rankings is news. And as supporters of D, we 
consider it to be good news. So, reporting it makes sense.

Now, while Tiobe may be an indicator of a language's popularity, and it may be 
paid attention to, it could very well be that it has issues and is ultimately 
a bad indicator of language popularity. Measuring something like language 
popularity is difficult to do accurately. So, the fact that Scala is above or 
below D in the rankings may or may not ultimately really have anything to do 
with their actual relative popularity (personally, I would have thought that 
Scala was more popular). That isn't really relevant though.

The fact that D moved up in Tiobe's rankings _is_ news. If it were reported as 
proof that D is more popular than the languages below it on Tiobe's list, 
_that_ would be misinformation. But all that was reported was that D had gone 
up in Tiobe's rankings. That is completely true and isn't misinformation at 
all. And as some people pay attention to Tiobe (for better or worse), the fact 
that D is higher in Tiobe's rankings matters to some folks and does have an 
impact. The issue of whether Tiobe is actually measuring what it's trying to 
measure is a separate matter entirely. And if someone on reddit happens to 
have taken it as proof that D is more popular than Scala and said something 
snide about Scala, that's their problem. There's no reason to take it out on 
Andrei for reporting that D moved up in Tiobe's rankings, which is completely 
factual.

- Jonathan M Davis
Sep 14 2011
prev sibling parent Caligo <iteronvexor gmail.com> writes:
On Wed, Sep 14, 2011 at 6:48 PM, Jeff Nowakowski <jeff dilacero.org> wrote:


 It's not a straw man. The only reason the D was even mentioned on Tiobe was

 any measure that aren't bullshit Tiobe statistics it should be, then you
 would have nothing to post.

 The Tiobe thing irked me, but I wasn't even going to say anything until I

 here about all the "haters" who downvoted this "news". It's misinformation,
 not news.


Well, apologizes.  I didn't mean to set you off.

I have to say though, I still don't fully understand why so many would down
vote such a news (or whatever you wish to call it).  Sure, there may be some
who down voted because of TIOBE, but I wonder how many down voted because
they don't wish to see D gain any level of popularity or simply because they
don't like D?
Sep 14 2011
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2011 4:23 AM, Steven Schveighoffer wrote:

 Oh, come on :) There's positive and negative traits to everything. Marketing
 touts the positive and ignores the negatives. How is that not misleading?


Volkswagon was famous in the 1960's for advertising that its car was ugly and 
small. Curtis Mathis advertised that its TV sets were the most expensive on the 
market. It's all in how you think about it.



 How many beer commercials show people getting into drunk driving accidents? How
 many soda commercials show a 300 lb. hacker at his laptop with 15 empty soda
 cans around him? Almost every miracle diet cure shows testimonials with the
 disclaimer "results not typical". Marketing is misleading *on purpose*, it is
 your job as the consumer to not believe anything at its word, and to try and
 find the other side of the story.


There is simply no way to present all sides of any product, especially not in a 
sound bite. Nor would anyone even want to read such or sit through it. And it 
would *still* be open to charges of bias.

It would be misleading if Coke was advertised as healthy.

Most diet plans are presented in a misleading manner. But the most successful 
one, P90X, is not misleading. It is pretty bald about saying you gotta work
your 
ass off to succeed at it.



 As interpreted by Mr. Nowakowski, it's misleading to say that D is good because
 some objective language popularity measurement shows D is gaining ground.


I think Tiobe is a reasonable indicator of "buzz" on a language. Whether buzz
== 
popularity or not, it is certainly at least one definition of popularity.
Sep 14 2011
prev sibling next sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Sep 13, 2011, at 8:07 AM, Steven Schveighoffer wrote:


 On Mon, 12 Sep 2011 20:11:57 -0400, Jeff Nowakowski =


<jeff dilacero.org> wrote:

=20

 On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

=20
 If you were in my position, wouldn't you have posted that
 article?


=20
 I hope I wouldn't have published it, since I don't like misleading =




marketing.

=20
 Since when is marketing *ever* not misleading?  I've yet to see the =


marketing campaign where Coke or Pepsi touts how their products make you =
fat.

Marketing and advertising are not the same thing.=
Sep 13 2011
parent reply "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 13 Sep 2011 16:07:31 -0400, Sean Kelly <sean invisibleduck.org>  
wrote:


 On Sep 13, 2011, at 8:07 AM, Steven Schveighoffer wrote:


 On Mon, 12 Sep 2011 20:11:57 -0400, Jeff Nowakowski <jeff dilacero.org>  
 wrote:


 On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

 If you were in my position, wouldn't you have posted that
 article?


 I hope I wouldn't have published it, since I don't like misleading  
 marketing.


 Since when is marketing *ever* not misleading?  I've yet to see the  
 marketing campaign where Coke or Pepsi touts how their products make  
 you fat.


 Marketing and advertising are not the same thing.


Advertising is marketing.  Not all marketing is advertising, but certainly  
marketing your weaknesses is not a usual practice.

The goal of marketing is to get customers to buy your product.  For an  
freely-available open-source language, the situation is no different, the  
competition is just as fierce, the only difference is the lack of price.

But the whole idea that posting a positive view of a language/product is  
"misleading" strikes me as hilarious.  That's all I was saying.

-Steve
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 07:13 AM, Steven Schveighoffer wrote:

 But the whole idea that posting a positive view of a language/product is
 "misleading" strikes me as hilarious. That's all I was saying.


Actually, Andrei has in the past made some noise in regards to other 
languages and presentations about cherry picking good results. If you 
know something is flawed and yet use it as evidence in your favor just 
because it benefits you, then that is misleading. There are ways to 
market that aren't misleading. I reject your statement that all 
marketing is misleading.
Sep 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/14/11 6:32 AM, Jeff Nowakowski wrote:

 On 09/14/2011 07:13 AM, Steven Schveighoffer wrote:

 But the whole idea that posting a positive view of a language/product is
 "misleading" strikes me as hilarious. That's all I was saying.


 Actually, Andrei has in the past made some noise in regards to other
 languages and presentations about cherry picking good results.


To which you protested just as vigorously. Wait, I think I see a pattern 
:o).

I mentioned that about Pike's presentation of Go because it showed only 
the positive consequences of some obvious tradeoffs, as if they did some 
magic somewhere nobody thought of and it was all roses.


 If you
 know something is flawed and yet use it as evidence in your favor just
 because it benefits you, then that is misleading.


I'd say we all lighten up a little. Tiobe is not unequivocally 
meaningless and D registered a good jump lately, which I believe is 
rooted in reality. I was glad about that and posted it on reddit. Tiobe 
is open about its methods and people can draw their conclusions.


Thanks,

Andrei
Sep 14 2011
parent Jesse Phillips <jessekphillips+D gmail.com> writes:
Andrei Alexandrescu Wrote:


 I'd say we all lighten up a little. Tiobe is not unequivocally 
 meaningless and D registered a good jump lately, which I believe is 
 rooted in reality. I was glad about that and posted it on reddit. Tiobe 
 is open about its methods and people can draw their conclusions.


That is what I'm interested in about it. Tiobe's methods may not be best to
decide which language is used, should be used, or is interesting, but it is
consistent.

D moved into the top 20 (10 positions) in a year. And I'm pretty sure those
that have been following D, that in the last year, there has been an increased
interest/activity in the language. So while Tiobe may not be accurate, it is
showing what most here probably already feel.
Sep 14 2011
prev sibling parent reply Jimmy Cao <jcao219 gmail.com> writes:
I trust Tiobe.

popularity increase.  Why would it not be correct about D'?

On Tue, Sep 13, 2011 at 3:07 PM, Sean Kelly <sean invisibleduck.org> wrote:


 On Sep 13, 2011, at 8:07 AM, Steven Schveighoffer wrote:


 On Mon, 12 Sep 2011 20:11:57 -0400, Jeff Nowakowski <jeff dilacero.org>


 wrote:

 On 09/12/2011 07:06 PM, Andrei Alexandrescu wrote:

 If you were in my position, wouldn't you have posted that
 article?


 I hope I wouldn't have published it, since I don't like misleading




 marketing.

 Since when is marketing *ever* not misleading?  I've yet to see the


 marketing campaign where Coke or Pepsi touts how their products make you
 fat.

 Marketing and advertising are not the same thing.
Sep 13 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/13/2011 05:18 PM, Jimmy Cao wrote:

 I trust Tiobe.

 popularity increase.  Why would it not be correct about D'?


Because Tiobe doesn't do a good job across the board. I posted some hard 
stats on D, ranked 20, vs. Scala, ranked 50. The results were totally 
lopsided in Scala's favor. Tiobe is not reliable, especially when it 
comes to trying to weed out false positives to the letter "D".
Sep 14 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2011 4:29 AM, Jeff Nowakowski wrote:

 especially when it comes to trying to weed out false positives to the letter
"D".


What would you say about its "C" results?

Also, Tiobe searches for the phrase "D programming", not just the letter "D".
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 02:35 PM, Walter Bright wrote:

 What would you say about its "C" results?


I'm sure it suffers from the same problem, just based on their methodology.


 Also, Tiobe searches for the phrase "D programming", not just the letter
 "D".


As I mentioned, there's overlap with graphics programming, like 2d and 
3d. After that I've seen random things like "Ph.D". If they wanted 
accurate results they'd have to sample the distribution, not just the 
first 100 pages.

I'm willing to bet that there are hardly any pages about D past a few 
thousand pages, yet "D programming" returns like twice as many pages as 
"scala programming", where the latter is much less likely to turn up as 
false positives.
Sep 14 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2011 12:16 PM, Jeff Nowakowski wrote:

 I'm willing to bet that there are hardly any pages about D past a few thousand
 pages


No need to bet, why not look & see?
Sep 14 2011
parent reply Jeff Nowakowski <jeff dilacero.org> writes:
On 09/14/2011 03:27 PM, Walter Bright wrote:

 No need to bet, why not look & see?


Because Google these days cuts off your search well before then. When I 
did this experiment years ago when D was in the top 20, it went into the 
weeds after a few dozen pages. So I suppose D has grown in that it no 
longer drops off dramatically within the first 100 pages, but that Scala
Sep 14 2011
parent reply Adam D. Ruppe <destructionator gmail.com> writes:
 Stack overflow shows 5,606 tagged with "scala", and 430 questions
 tagged with "d".


Could that be simply because D is an easier language to use?
Sep 14 2011
next sibling parent Caligo <iteronvexor gmail.com> writes:
On Wed, Sep 14, 2011 at 3:58 PM, Adam D. Ruppe <destructionator gmail.com>wrote:


 Stack overflow shows 5,606 tagged with "scala", and 430 questions
 tagged with "d".


 Could that be simply because D is an easier language to use?


I remember when I started to use D1 for the first time about two years ago.
For most cases it was just a matter of copy/pasting my C code and compiling
with DMD.  If you know C/C++, D can be very easy.  I've never tried Scala,
but I'm learning Haskell and sometimes it's a challenge.
Sep 14 2011
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/14/2011 1:58 PM, Adam D. Ruppe wrote:

 Stack overflow shows 5,606 tagged with "scala", and 430 questions
 tagged with "d".


 Could that be simply because D is an easier language to use?


Or perhaps they prefer to ask about D here in these n.g. There are hundreds of 
thousands of postings here.
Sep 14 2011
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 9/12/2011 11:38 AM, Jeff Nowakowski wrote:

 Lame marketing, Andrei. But hey, it works, right?


I think D is doing spectacularly well considering that there is $0 spent on 
marketing it.
Sep 13 2011
prev sibling parent Kagamin <spam here.lot> writes:
bearophile Wrote:


 I think most managers only see the top 11 of that list I am seeing now (some
managers probably see some language below too, like Delphi, Ada, Scala, etc,
but they are uncommon).


DevExpress enterprise-quality visual controls library started as a Delphi
project and they still support the delphi version.
Sep 12 2011
prev sibling next sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
It seems people (Andrei too) are slowly adding more comments in that Reddit
thread.
A sub-thread:
http://www.reddit.com/r/programming/comments/kaxjq/the_d_programming_language_is_back_in_tiobes_top/c2iycwb

It starts with this question:


 How would you write this kind of Python in D?
 
 from struct import Struct
 def decode(buf):
     while buf:
         s = Struct(buf[0])
         val, = s.unpack(buf[1 : 1 + s.size])
         yield val
         buf = buf[1 + s.size:]
 
 Or this:
 
 def powerset(items):
     if not items:
         yield []
         return
     item, rest = items[0], items[1:]
     for opt in [[item], []]:
         for r in powerset(rest):
             yield opt + r
 
 btw, I like Haskell even more, and in it, the latter example is:
 
 powerset [] = return []
 powerset (x:xs) = do
   o <- [[x], []]
   r <- powerset xs
   return $ o ++ r
 
 Or, you can re-use the filterM function, and get:
 
 powerset = filterM (\x -> [True,False])



This is the lazy Haskell program, it's not so basic as it looks:


powerset [] = return []
powerset (x:xs) = do
  o <- [[x], []]
  r <- powerset xs
  return $ o ++ r

main = print $ powerset [1, 2, 3]

It outputs:
[[1,2,3],[1,2],[1,3],[1],[2,3],[2],[3],[]]


The second Haskell version of the same program:

import Control.Monad (filterM)

powerset = filterM (\x -> [True,False])

main = print $ powerset [1, 2, 3]



An eager D version, with arrays:

import std.stdio: writeln;

auto powerSet(T)(T[] items) {
    auto r = new T[][](1, 0);
    foreach (e; items) {
        T[][] rs;
        foreach (x; r)
            rs ~= x ~ [e];
        r ~= rs;
    }
    return r;
}

void main() {
    writeln(powerSet([1, 2, 3]));
}


That outputs:
[[], [1], [2], [1, 2], [3], [1, 3], [2, 3], [1, 2, 3]]


The lazyness has some advantages. If I call the D version with:

writeln(powerSet(array(iota(22))).length);

And the Haskell version with:

main = print $ length $ powerset [0..21]


The D code (GHC -O3) takes 9.7 seconds and 430 MB RAM

The Haskell code takes about 1.8 seconds and less than 200 MB of RAM.

The second Haskell version (with filterM) takes about 2.4 seconds and about 240
MB RAM.


A lazy D version:


import std.stdio, std.array, std.range;

struct PowerSet(T) {
    T[] items;

    int opApply(int delegate(ref T[]) dg) {
        int result;
        T[] res;

        if (!items.length) {
            result = dg(res);
        } else {
            outer:
            foreach (opt; [items[0..1], []])
                foreach (r; PowerSet(items[1..$])) {
                    res = opt ~ r;
                    result = dg(res);
                    if (result) break outer;
                }
        }

        return result;
    }
}

void main() {
    // can't use this because the silly walkLength()
    // doesn't recognize opApply yet:
    // http://d.puremagic.com/issues/show_bug.cgi?id=5691
    //writeln(walkLength(PowerSet!int(array(iota(22)))));

    size_t count;
    foreach (x; PowerSet!int(array(iota(22))))
        count++;
    writeln(count);
}


The lazy D version runs in about 22 seconds, using less than 10 MB. DMD
performs no advanced lazy optimizations here.



A more complex version that does't yield distinct sub-arrays to allocate less
memory:


import std.stdio, std.array, std.range;

struct PowerSet(T) {
    T[] items;

    int opApply(int delegate(ref T[]) dg) {
        int result;
        T[] res;
        T[30] buf;

        if (!items.length) {
            result = dg(res);
        } else {
            outer:
            foreach (opt; [items[0..1], []]) {
                buf[0 .. opt.length] = opt[];
                foreach (r; PowerSet(items[1..$])) {
                    buf[opt.length .. opt.length + r.length] = r[];
                    res = buf[0 .. (opt.length + r.length)];
                    result = dg(res);
                    if (result) break outer;
                }
            }
        }

        return result;
    }
}

void main() {
    size_t count;
    foreach (x; PowerSet!int(array(iota(22))))
        count++;
    writeln(count);
}


This runs in about 4.5 seconds, and uses less than 10 MB RAM.


With the syntax sugar I have suggested here:
http://d.puremagic.com/issues/show_bug.cgi?id=5660

the recursive lazy D version becomes something like this, that is not too much
bad looking (untested):


yield(T[]) powerSet(T)(T[] items) {
    if (!items.length)
        yield [];
    else    
        foreach (opt; [items[0..1], []])
            foreach (r; powerSet(items[1..$]))
                yield opt ~ r;
}

Bye,
bearophile
Sep 12 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/12/2011 01:36 PM, bearophile wrote:

 It seems people (Andrei too) are slowly adding more comments in that Reddit
thread.
 A sub-thread:
 http://www.reddit.com/r/programming/comments/kaxjq/the_d_programming_language_is_back_in_tiobes_top/c2iycwb

 It starts with this question:


 How would you write this kind of Python in D?

 from struct import Struct
 def decode(buf):
      while buf:
          s = Struct(buf[0])
          val, = s.unpack(buf[1 : 1 + s.size])
          yield val
          buf = buf[1 + s.size:]

 Or this:

 def powerset(items):
      if not items:
          yield []
          return
      item, rest = items[0], items[1:]
      for opt in [[item], []]:
          for r in powerset(rest):
              yield opt + r

 btw, I like Haskell even more, and in it, the latter example is:

 powerset [] = return []
 powerset (x:xs) = do
    o<- [[x], []]
    r<- powerset xs
    return $ o ++ r

 Or, you can re-use the filterM function, and get:

 powerset = filterM (\x ->  [True,False])



 This is the lazy Haskell program, it's not so basic as it looks:


 powerset [] = return []
 powerset (x:xs) = do
    o<- [[x], []]
    r<- powerset xs
    return $ o ++ r

 main = print $ powerset [1, 2, 3]

 It outputs:
 [[1,2,3],[1,2],[1,3],[1],[2,3],[2],[3],[]]


 The second Haskell version of the same program:

 import Control.Monad (filterM)

 powerset = filterM (\x ->  [True,False])

 main = print $ powerset [1, 2, 3]



 An eager D version, with arrays:

 import std.stdio: writeln;

 auto powerSet(T)(T[] items) {
      auto r = new T[][](1, 0);
      foreach (e; items) {
          T[][] rs;
          foreach (x; r)
              rs ~= x ~ [e];
          r ~= rs;
      }
      return r;
 }

 void main() {
      writeln(powerSet([1, 2, 3]));
 }


 That outputs:
 [[], [1], [2], [1, 2], [3], [1, 3], [2, 3], [1, 2, 3]]


 The lazyness has some advantages. If I call the D version with:

 writeln(powerSet(array(iota(22))).length);

 And the Haskell version with:

 main = print $ length $ powerset [0..21]


 The D code (GHC -O3) takes 9.7 seconds and 430 MB RAM

 The Haskell code takes about 1.8 seconds and less than 200 MB of RAM.

 The second Haskell version (with filterM) takes about 2.4 seconds and about
240 MB RAM.


 A lazy D version:


 import std.stdio, std.array, std.range;

 struct PowerSet(T) {
      T[] items;

      int opApply(int delegate(ref T[]) dg) {
          int result;
          T[] res;

          if (!items.length) {
              result = dg(res);
          } else {
              outer:
              foreach (opt; [items[0..1], []])
                  foreach (r; PowerSet(items[1..$])) {
                      res = opt ~ r;
                      result = dg(res);
                      if (result) break outer;
                  }
          }

          return result;
      }
 }

 void main() {
      // can't use this because the silly walkLength()
      // doesn't recognize opApply yet:
      // http://d.puremagic.com/issues/show_bug.cgi?id=5691
      //writeln(walkLength(PowerSet!int(array(iota(22)))));

      size_t count;
      foreach (x; PowerSet!int(array(iota(22))))
          count++;
      writeln(count);
 }


 The lazy D version runs in about 22 seconds, using less than 10 MB. DMD
performs no advanced lazy optimizations here.



 A more complex version that does't yield distinct sub-arrays to allocate less
memory:


 import std.stdio, std.array, std.range;

 struct PowerSet(T) {
      T[] items;

      int opApply(int delegate(ref T[]) dg) {
          int result;
          T[] res;
          T[30] buf;

          if (!items.length) {
              result = dg(res);
          } else {
              outer:
              foreach (opt; [items[0..1], []]) {
                  buf[0 .. opt.length] = opt[];
                  foreach (r; PowerSet(items[1..$])) {
                      buf[opt.length .. opt.length + r.length] = r[];
                      res = buf[0 .. (opt.length + r.length)];
                      result = dg(res);
                      if (result) break outer;
                  }
              }
          }

          return result;
      }
 }

 void main() {
      size_t count;
      foreach (x; PowerSet!int(array(iota(22))))
          count++;
      writeln(count);
 }


 This runs in about 4.5 seconds, and uses less than 10 MB RAM.


 With the syntax sugar I have suggested here:
 http://d.puremagic.com/issues/show_bug.cgi?id=5660

 the recursive lazy D version becomes something like this, that is not too much
bad looking (untested):


 yield(T[]) powerSet(T)(T[] items) {
      if (!items.length)
          yield [];
      else
          foreach (opt; [items[0..1], []])
              foreach (r; powerSet(items[1..$]))
                  yield opt ~ r;
 }

 Bye,
 bearophile


This eager D version uncovered a DMD bug:

import std.stdio, std.algorithm, std.range;
T[][] powerset(T)(T[] xs){
     T[][] r=new T[][](1<<xs.length);
     foreach(i,ref x;r) x=array(indexed(xs,filter!((a){return 
1<<a&i;})(iota(0,xs.length))));
     return r;
}

void main(){
     writeln(powerset([1,2,3]));
}

This does not compile iff the -inline flag is passed to DMD 2.055.
Sep 12 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/12/11 7:28 AM, Timon Gehr wrote:

 This eager D version uncovered a DMD bug:

 import std.stdio, std.algorithm, std.range;
 T[][] powerset(T)(T[] xs){
 T[][] r=new T[][](1<<xs.length);
 foreach(i,ref x;r) x=array(indexed(xs,filter!((a){return
 1<<a&i;})(iota(0,xs.length))));
 return r;
 }

 void main(){
 writeln(powerset([1,2,3]));
 }

 This does not compile iff the -inline flag is passed to DMD 2.055.


goto bugzilla;


Andrei
Sep 12 2011
parent reply dsimcha <dsimcha yahoo.com> writes:
== Quote from Andrei Alexandrescu (SeeWebsiteForEmail erdani.org)'s article

 On 9/12/11 7:28 AM, Timon Gehr wrote:

 This eager D version uncovered a DMD bug:

 import std.stdio, std.algorithm, std.range;
 T[][] powerset(T)(T[] xs){
 T[][] r=new T[][](1<<xs.length);
 foreach(i,ref x;r) x=array(indexed(xs,filter!((a){return
 1<<a&i;})(iota(0,xs.length))));
 return r;
 }

 void main(){
 writeln(powerset([1,2,3]));
 }

 This does not compile iff the -inline flag is passed to DMD 2.055.


 goto bugzilla;
 Andrei


Looks like this old bug report:

http://d.puremagic.com/issues/show_bug.cgi?id=4724

I'd love to see fixing all the lambda function issues be the next big push for
improving DMD.  Right now, they're so buggy it's not even funny.  Besides 4724,
the other one that really irks me and severely limits the usability of
std.parallelism is http://d.puremagic.com/issues/show_bug.cgi?id=5710 .
Sep 12 2011
parent David Nadlinger <see klickverbot.at> writes:
On 9/12/11 5:51 PM, dsimcha wrote:

 Besides 4724,
 the other one that really irks me and severely limits the usability of
 std.parallelism is http://d.puremagic.com/issues/show_bug.cgi?id=5710 .


Basically, it is a huge PITA for all designs that resemble std.algorithm 
(e.g. also my Thrift client pool code[1]), and thus, I think we should 
try to fix this as soon as possible – imho we should promote the design 
of the standard library and encourage people to follow it, not the other 
way round.

David


[1] 
http://klickverbot.at/code/gsoc/thrift/docs/thrift.codegen.async_client_pool.html
Sep 14 2011
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 09/12/2011 01:36 PM, bearophile wrote:

 A more complex version that does't yield distinct sub-arrays to allocate less
memory:


 import std.stdio, std.array, std.range;

 struct PowerSet(T) {
      T[] items;

      int opApply(int delegate(ref T[]) dg) {
          int result;
          T[] res;
          T[30] buf;

          if (!items.length) {
              result = dg(res);
          } else {
              outer:
              foreach (opt; [items[0..1], []]) {
                  buf[0 .. opt.length] = opt[];
                  foreach (r; PowerSet(items[1..$])) {
                      buf[opt.length .. opt.length + r.length] = r[];
                      res = buf[0 .. (opt.length + r.length)];
                      result = dg(res);
                      if (result) break outer;
                  }
              }
          }

          return result;
      }
 }

 void main() {
      size_t count;
      foreach (x; PowerSet!int(array(iota(22))))
          count++;
      writeln(count);
 }


 This runs in about 4.5 seconds, and uses less than 10 MB RAM.


You lose vs haskell because haskell does not recompute the same PowerSet 
multiple times.
Sep 12 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Timon Gehr:


 You lose vs haskell because haskell does not recompute the same PowerSet 
 multiple times.


Are you sure? I have tried with memoization, but now it's slower (about 5.8
seconds).

Note: to use memoize I have had to add the opCall. Is this really necessary for
memoization of a struct?


import std.stdio, std.array, std.range, std.functional;

struct PowerSet(T) {
    T[] items;

    static PowerSet opCall(T[] it) {
        PowerSet p;
        p.items = it;
        return p;
    }

    int opApply(int delegate(ref T[]) dg) {
        int result;
        T[] res;
        T[30] buf;

        if (!items.length) {
            result = dg(res);
        } else {
            outer:
            foreach (opt; [items[0..1], []]) {
                buf[0 .. opt.length] = opt[];
                alias memoize!PowerSet mPowerSet;
                foreach (r; mPowerSet(items[1..$])) {
                    buf[opt.length .. opt.length + r.length] = r[];
                    res = buf[0 .. (opt.length + r.length)];
                    result = dg(res);
                    if (result) break outer;
                }
            }
        }

        return result;
    }
}

void main() {
    size_t count;
    foreach (x; PowerSet!int(array(iota(22))))
        count++;
    writeln(count);
}


Bye,
bearophile
Sep 12 2011
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/12/2011 06:57 PM, bearophile wrote:

 Timon Gehr:


 You lose vs haskell because haskell does not recompute the same PowerSet
 multiple times.


 Are you sure? I have tried with memoization, but now it's slower (about 5.8
seconds).

 Note: to use memoize I have had to add the opCall. Is this really necessary
for memoization of a struct?


 import std.stdio, std.array, std.range, std.functional;

 struct PowerSet(T) {
      T[] items;

      static PowerSet opCall(T[] it) {
          PowerSet p;
          p.items = it;
          return p;
      }

      int opApply(int delegate(ref T[]) dg) {
          int result;
          T[] res;
          T[30] buf;

          if (!items.length) {
              result = dg(res);
          } else {
              outer:
              foreach (opt; [items[0..1], []]) {
                  buf[0 .. opt.length] = opt[];
                  alias memoize!PowerSet mPowerSet;
                  foreach (r; mPowerSet(items[1..$])) {
                      buf[opt.length .. opt.length + r.length] = r[];
                      res = buf[0 .. (opt.length + r.length)];
                      result = dg(res);
                      if (result) break outer;
                  }
              }
          }

          return result;
      }
 }

 void main() {
      size_t count;
      foreach (x; PowerSet!int(array(iota(22))))
          count++;
      writeln(count);
 }


 Bye,
 bearophile


std.functional.memoize uses AAs internally, so that is very slow.

Haskell builds a linked data structure, if you have

result: [...,[2,3],[1,2,3]]

then the [2,3] part is exactly the same for both. Also, if you only look 
at the length, the haskell code probably does not fully evaluate the 
power set.
Sep 12 2011
prev sibling parent "Martin Nowak" <dawg dawgfoto.de> writes:
On Mon, 12 Sep 2011 18:57:16 +0200, bearophile <bearophileHUGS lycos.com>  
wrote:


 Timon Gehr:


 You lose vs haskell because haskell does not recompute the same PowerSet
 multiple times.


 Are you sure? I have tried with memoization, but now it's slower (about  
 5.8 seconds).

 Note: to use memoize I have had to add the opCall. Is this really  
 necessary for memoization of a struct?


 import std.stdio, std.array, std.range, std.functional;

 struct PowerSet(T) {
     T[] items;

     static PowerSet opCall(T[] it) {
         PowerSet p;
         p.items = it;
         return p;
     }

     int opApply(int delegate(ref T[]) dg) {
         int result;
         T[] res;
         T[30] buf;

         if (!items.length) {
             result = dg(res);
         } else {
             outer:
             foreach (opt; [items[0..1], []]) {
                 buf[0 .. opt.length] = opt[];
                 alias memoize!PowerSet mPowerSet;
                 foreach (r; mPowerSet(items[1..$])) {
                     buf[opt.length .. opt.length + r.length] = r[];


You are copying recursively here. You should pass a slice to the recursive  
instance.

                     res = buf[0 .. (opt.length + r.length)];
                     result = dg(res);
                     if (result) break outer;
                 }
             }
         }

         return result;
     }
 }

 void main() {
     size_t count;
     foreach (x; PowerSet!int(array(iota(22))))
         count++;
     writeln(count);
 }


 Bye,
 bearophile


Besides this can be wrapped in a random access range,
which takes predictable 65ms * 2 ^ (setSize - 22) for me.
https://gist.github.com/1211961
Sep 12 2011
prev sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 09/12/2011 01:36 PM, bearophile wrote:

 It seems people (Andrei too) are slowly adding more comments in that Reddit
thread.
 A sub-thread:
 http://www.reddit.com/r/programming/comments/kaxjq/the_d_programming_language_is_back_in_tiobes_top/c2iycwb

 It starts with this question:


 How would you write this kind of Python in D?

 from struct import Struct
 def decode(buf):
      while buf:
          s = Struct(buf[0])
          val, = s.unpack(buf[1 : 1 + s.size])
          yield val
          buf = buf[1 + s.size:]

 Or this:

 def powerset(items):
      if not items:
          yield []
          return
      item, rest = items[0], items[1:]
      for opt in [[item], []]:
          for r in powerset(rest):
              yield opt + r

 btw, I like Haskell even more, and in it, the latter example is:

 powerset [] = return []
 powerset (x:xs) = do
    o<- [[x], []]
    r<- powerset xs
    return $ o ++ r

 Or, you can re-use the filterM function, and get:

 powerset = filterM (\x ->  [True,False])




Equivalent D version. Runs in ca 22 min on my machine (for computing the 
size of P(Z_21))

LList!(List!T) powerset(T)(Lazy!(List!T) xs){
     return lz({
             if(xs().empty) return list(Empty!T)();
             auto r=lz({return powerset(xs().tail)();});
             return chain(r,map((LList!T a){return 
cons(xs().head,a);},r))();
         });
}

writeln(powerset(take(21,naturals(0))).length);


2097152

real	21m58.438s
user	21m32.320s
sys	0m4.710s
Sep 12 2011
prev sibling next sibling parent reply bioinfornatics <bioinfornatics fedoraproject.orh> writes:
It is a good news. This news appear in same time where fedora add D2 in Fedora
16:
https://fedoraproject.org/wiki/Features/D2_programming

:-)

next step create powerfull free and open source library, more documentation and
tutorial on the web
Sep 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 9/14/11 6:22 AM, bioinfornatics wrote:

 It is a good news. This news appear in same time where fedora add D2 in Fedora
16:
 https://fedoraproject.org/wiki/Features/D2_programming

 :-)

 next step create powerfull free and open source library, more documentation and
 tutorial on the web


I wonder whether it's okay to post this on reddit.

Andrei
Sep 14 2011
parent reply "Marco Leise" <Marco.Leise gmx.de> writes:
Am 14.09.2011, 15:02 Uhr, schrieb Andrei Alexandrescu  
<SeeWebsiteForEmail erdani.org>:


 On 9/14/11 6:22 AM, bioinfornatics wrote:

 It is a good news. This news appear in same time where fedora add D2 in  
 Fedora 16:
 https://fedoraproject.org/wiki/Features/D2_programming

 :-)

 next step create powerfull free and open source library, more  
 documentation and
 tutorial on the web


 I wonder whether it's okay to post this on reddit.

 Andrei


You would definitely need a straw man to do this. Oh btw I hope to get a  
recent DMD into Gentoo again soonish.
Sep 14 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 09/14/2011 12:18 PM, Marco Leise wrote:

 Am 14.09.2011, 15:02 Uhr, schrieb Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org>:


 On 9/14/11 6:22 AM, bioinfornatics wrote:

 It is a good news. This news appear in same time where fedora add D2
 in Fedora 16:
 https://fedoraproject.org/wiki/Features/D2_programming

 :-)

 next step create powerfull free and open source library, more
 documentation and
 tutorial on the web


 I wonder whether it's okay to post this on reddit.

 Andrei


 You would definitely need a straw man to do this.


The thing about that piece is I'm not sure for how long D has been in 
Fedora, so I wouldn't know whether it's really news, and how noteworthy 
(e.g. is Fedora picky in adopting new packages etc).


 Oh btw I hope to get a
 recent DMD into Gentoo again soonish.


That would be great!


Andrei
Sep 14 2011
parent Kai Meyer <kai unixlords.com> writes:
On 09/14/2011 11:24 AM, Andrei Alexandrescu wrote:

 On 09/14/2011 12:18 PM, Marco Leise wrote:

 Am 14.09.2011, 15:02 Uhr, schrieb Andrei Alexandrescu
 <SeeWebsiteForEmail erdani.org>:


 On 9/14/11 6:22 AM, bioinfornatics wrote:

 It is a good news. This news appear in same time where fedora add D2
 in Fedora 16:
 https://fedoraproject.org/wiki/Features/D2_programming

 :-)

 next step create powerfull free and open source library, more
 documentation and
 tutorial on the web


 I wonder whether it's okay to post this on reddit.

 Andrei


 You would definitely need a straw man to do this.


 The thing about that piece is I'm not sure for how long D has been in
 Fedora, so I wouldn't know whether it's really news, and how noteworthy
 (e.g. is Fedora picky in adopting new packages etc).


 Oh btw I hope to get a
 recent DMD into Gentoo again soonish.


 That would be great!


 Andrei


Fedora will pick up just about any package that has a package maintainer 
committed to keep it up to date. It's pretty open. I think I remember 
LDC showing up in Fedora 15. With 9 month(ish) release cycles, I would 
guess that some interested party has been trying to get LDC into Fedora 
for at least a year at this point.
Sep 14 2011
prev sibling parent Samuel Lampa <samuel.lampa gmail.com> writes:

today.

[1] Still get it after doing GSoC for Semantic MediaWiki the other year...
[2] http://www.infoworld.com/print/172587

// Samuel


On 09/10/2011 07:50 AM, Paulo Pinto wrote:

 D just entered into the top 20!

 http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

 I know that TIOBE's value is relative, but this is the type of charts
 that managers usually understand.

 --
 Paulo
Sep 16 2011