www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Recent download statistics

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
I recently got the access logs for dmd downloads through August (and Sep 
1st). They show a sharp increase since July, probably following the 
Wired coverage and the recent release. This is raw data so it does not 
correct for failed downloads etc, but I assume it is a good proxy for 
the actual numbers and also good for looking at trends.

The relevant makefile rules are:

$(OUT)/downloads:
	grep -o 'GET [^ ]* ' \
	  $(DATADIR)/downloads/2013-* $(DATADIR)/downloads/2014-* \
	  | grep 'dmd' | sed -e 's/:GET//' -e 's|^.*downloads/||' \
	  | sed -e 's/ .*dmd/ dmd/' | sort >$ .tmp
	mv $ .tmp $ 

$(OUT)/downloads.monthly: $(OUT)/downloads
	sed -e 's/-.. .*//' $^ | uniq -c >>$ .tmp
	mv $ .tmp $ 

The log lines follow the pattern (some numbers changed for privacy):

f0a5c47d9741bd8c3fffbfc3dacfaeb7 downloads.dlang.org 
[31/Dec/2013:23:04:13 +0000] 181.16.226.282 - 61ED358EF78C3B10 
WEBSITE.GET.OBJECT releases/2013/dmd.2.064.2.zip "GET 
/releases/2013/dmd.2.064.2.zip HTTP/1.1" 200 - 33349115 33349115 331564 
266 "-" "NSIS_Inetc (Mozilla)" -

Going forward I think we should publish the logs and run a cron job on 
the server that updates them periodically.

July's 34979 is a 19 months (and possibly all-time) high. 
Congratulations to everybody involved in pushing the D language forward!


Andrei



5886 2013-01
5525 2013-02
22799 2013-03
11717 2013-04
6214 2013-05
9614 2013-06
11455 2013-07
16803 2013-08
20835 2013-09
19009 2013-10
20569 2013-11
15742 2013-12
18002 2014-01
20191 2014-02
18651 2014-03
19600 2014-04
21015 2014-05
20962 2014-06
34979 2014-07
34288 2014-08
1088 2014-09-01
Sep 03 2014
next sibling parent reply "Peter Alexander" <peter.alexander.au gmail.com> writes:
On Wednesday, 3 September 2014 at 09:20:44 UTC, Andrei 
Alexandrescu wrote:

 I recently got the access logs for dmd downloads through August 
 (and Sep 1st). [snip]


Nice! Will be interesting to see how much the recent increase 
contributes to sustained activity/growth.

Any idea what caused the spike in March last year?
Sep 03 2014
parent "Gary Willoughby" <dev nomad.so> writes:
On Wednesday, 3 September 2014 at 09:47:45 UTC, Peter Alexander 
wrote:

 Any idea what caused the spike in March last year?


Dconf 2013?
Sep 03 2014
prev sibling parent reply "Martin Nowak" <code dawg.eu> writes:
On Wednesday, 3 September 2014 at 09:20:44 UTC, Andrei 
Alexandrescu wrote:

 The relevant makefile rules are:

 $(OUT)/downloads:
 	grep -o 'GET [^ ]* ' \
 	  $(DATADIR)/downloads/2013-* $(DATADIR)/downloads/2014-* \
 	  | grep 'dmd' | sed -e 's/:GET//' -e 's|^.*downloads/||' \
 	  | sed -e 's/ .*dmd/ dmd/' | sort >$ .tmp
 	mv $ .tmp $ 

 $(OUT)/downloads.monthly: $(OUT)/downloads
 	sed -e 's/-.. .*//' $^ | uniq -c >>$ .tmp
 	mv $ .tmp $ 


Nice, might be worth to spend some time filtering out machine 
traffic, e.g. from running travis-ci. Also equally interesting is 
to group those numbers by DPL release and OS.
Oct 13 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 10/13/14, 9:07 AM, Martin Nowak wrote:

 On Wednesday, 3 September 2014 at 09:20:44 UTC, Andrei Alexandrescu wrote:

 The relevant makefile rules are:

 $(OUT)/downloads:
     grep -o 'GET [^ ]* ' \
       $(DATADIR)/downloads/2013-* $(DATADIR)/downloads/2014-* \
       | grep 'dmd' | sed -e 's/:GET//' -e 's|^.*downloads/||' \
       | sed -e 's/ .*dmd/ dmd/' | sort >$ .tmp
     mv $ .tmp $ 

 $(OUT)/downloads.monthly: $(OUT)/downloads
     sed -e 's/-.. .*//' $^ | uniq -c >>$ .tmp
     mv $ .tmp $ 


 Nice, might be worth to spend some time filtering out machine traffic,
 e.g. from running travis-ci.


How can those be identified?


 Also equally interesting is to group those numbers by DPL release and
 OS.


Yah.


Andrei
Oct 13 2014
parent "Martin Nowak" <code dawg.eu> writes:
On Monday, 13 October 2014 at 17:47:56 UTC, Andrei Alexandrescu
wrote:

 Nice, might be worth to spend some time filtering out machine 
 traffic,
 e.g. from running travis-ci.


 How can those be identified?


Good news for the new D support on Travis-CI. I prefixed the curl
User-Agent with Travis-CI, so you'll easily find those :) [1].
Otherwise (many people will continue to use their old scripts)
you can only filter for the IP range of BlueBox that currently
hosts most (if not all) of the Travis-CI boxes, see [2], [3].

https://github.com/MartinNowak/travis-build/commit/43286a1bf3865977461c3cb86882a8c35a964a9e

[1]:
https://github.com/MartinNowak/travis-build/commit/43286a1bf3865977461c3cb86882a8c35a964a9e
[2]:
http://ns.myip.ms/view/web_hosting/143013/Blue_Box_Group_Inc.html
[3]: https://github.com/travis-ci/travis-ci/issues/2580
Dec 11 2014