www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Digger 1.0

reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
Most notable change since DConf is that on Windows, Digger can 
now build D from source (including x64 versions) without 
requiring Git or Visual Studio to be installed. It achieves this 
by downloading and locally installing (unpacking) all the 
software it needs.

Windows binaries:

https://github.com/CyberShadow/Digger/releases/tag/1.0

Digger is a tool for working with D's source code and its 
history. It can build D (including older D versions), customize 
the build with pending pull requests or forks, and find the exact 
pull request which introduced a regression (or fixed a bug). It 
comes together with a web interface which makes building D from 
source trivial even for people new to D, Git or the command line.

https://github.com/CyberShadow/Digger
Sep 18 2014
next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Friday, 19 September 2014 at 01:36:54 UTC, Vladimir Panteleev 
wrote:

 Windows binaries:

 https://github.com/CyberShadow/Digger/releases/tag/1.0


Yet another release ruined by a DMD -inline wrong-code bug :(

Reuploaded new .zip file without -inline.
Sep 20 2014
parent =?UTF-8?B?Ik5vcmRsw7Z3Ig==?= <per.nordlow gmail.com> writes:
On Saturday, 20 September 2014 at 20:07:46 UTC, Vladimir 
Panteleev wrote:

 Yet another release ruined by a DMD -inline wrong-code bug :(


It seems like use of -inline is not recommended then?
Sep 23 2014
prev sibling parent reply Rainer Schuetze <r.sagitario gmx.de> writes:
On 19.09.2014 03:36, Vladimir Panteleev wrote:

 Most notable change since DConf is that on Windows, Digger can now build
 D from source (including x64 versions) without requiring Git or Visual
 Studio to be installed. It achieves this by downloading and locally
 installing (unpacking) all the software it needs.

 Windows binaries:

 https://github.com/CyberShadow/Digger/releases/tag/1.0

 Digger is a tool for working with D's source code and its history. It
 can build D (including older D versions), customize the build with
 pending pull requests or forks, and find the exact pull request which
 introduced a regression (or fixed a bug). It comes together with a web
 interface which makes building D from source trivial even for people new
 to D, Git or the command line.

 https://github.com/CyberShadow/Digger


I tried it on Windows and Digger does an amazing job at installing 
dependencies. I think we should recommend it as the first thing to run 
when trying to get your hands on building dmd/phobos.

In case someone starts creating patches: Would it be possible to also 
write a batch file into the repository folder to redo a build of dmd, 
druntime and phobos without checking all the dependencies?

How about running the test suite?
Sep 21 2014
next sibling parent "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Sunday, 21 September 2014 at 17:43:14 UTC, Rainer Schuetze 
wrote:

 How about running the test suite?


+1
Would make me far more happier of starting seriously getting into 
dmd bug fixing.
Sep 21 2014
prev sibling parent reply Nick Treleaven <ntrel-public yahoo.co.uk> writes:
On 21/09/2014 18:43, Rainer Schuetze wrote:

 I tried it on Windows and Digger does an amazing job at installing
 dependencies. I think we should recommend it as the first thing to run
 when trying to get your hands on building dmd/phobos.


+1


 In case someone starts creating patches: Would it be possible to also
 write a batch file into the repository folder to redo a build of dmd,
 druntime and phobos without checking all the dependencies?


Sometimes my Windows machine with 2 GB RAM gets OOM when trying to link 
phobos.lib (I have to close most programs and start again), it would be 
nice if there was a way to continue a failed build without starting from 
scratch.

I also hope there's a way of making the link take less memory, perhaps 
by making sub-libraries first and linking those together.


 How about running the test suite?


AFAICT the test suite needs a separate MSYS install from the one Git 
uses, e.g. for a newer version of 'diff'. Not sure if that makes it 
harder for Digger to support.
Sep 22 2014
next sibling parent reply simendsjo <simendsjo+dlang gmail.com> writes:
On 09/22/2014 12:50 PM, Nick Treleaven wrote:
(...)

 Sometimes my Windows machine with 2 GB RAM gets OOM when trying to link
 phobos.lib (I have to close most programs and start again), it would be
 nice if there was a way to continue a failed build without starting from
 scratch.


My guess is the average for developers is ~8GB. 2GB RAM is really not
enough for pretty much anything these days - the browser alone easily
chews 3-4GB on moderate use. I recommend you just upgrade your computer
(or compile on a better iron).
Sep 22 2014
next sibling parent reply ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Mon, 22 Sep 2014 15:24:55 +0200
simendsjo via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:


 My guess is the average for developers is ~8GB. 2GB RAM is really not
 enough for pretty much anything these days - the browser alone easily
 chews 3-4GB on moderate use. I recommend you just upgrade your
 computer (or compile on a better iron).


are your primary language is java? i'm asking just out of curiousity.
Sep 22 2014
next sibling parent simendsjo <simendsjo+dlang gmail.com> writes:
On 09/22/2014 07:28 PM, ketmar via Digitalmars-d-announce wrote:

 On Mon, 22 Sep 2014 15:24:55 +0200
 simendsjo via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:
 

 My guess is the average for developers is ~8GB. 2GB RAM is really not
 enough for pretty much anything these days - the browser alone easily
 chews 3-4GB on moderate use. I recommend you just upgrade your
 computer (or compile on a better iron).


 are your primary language is java? i'm asking just out of curiousity.
 



isn't exactly light on memory either.
Sep 22 2014
prev sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 22 September 2014 at 17:28:50 UTC, ketmar via 
Digitalmars-d-announce wrote:

 On Mon, 22 Sep 2014 15:24:55 +0200
 simendsjo via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:


 My guess is the average for developers is ~8GB. 2GB RAM is 
 really not
 enough for pretty much anything these days - the browser alone 
 easily
 chews 3-4GB on moderate use. I recommend you just upgrade your
 computer (or compile on a better iron).


 are your primary language is java? i'm asking just out of 
 curiousity.


That was uncalled for.

Firefox requires 4GB of memory to build.
Chromium requires 8GB of memory to build.
Android requires 16GB of memory to build.

If you want to work on big projects, you WILL need a decent 
computer.

I think 4GB for a modern programming language's implementation is 
not an unreasonable requirement, even if it could be brought down 
in the future. Especially considering that you can't even buy a 
new laptop today with less than 4GB of RAM, and 3GB is becoming 
the norm for smartphones.
Sep 22 2014
next sibling parent reply Nick Treleaven <ntrel-public yahoo.co.uk> writes:
On 22/09/2014 19:59, Vladimir Panteleev wrote:

 Firefox requires 4GB of memory to build.
 Chromium requires 8GB of memory to build.
 Android requires 16GB of memory to build.


Thanks for the info, I didn't realize.


 If you want to work on big projects, you WILL need a decent computer.

 I think 4GB for a modern programming language's implementation is not an
 unreasonable requirement, even if it could be brought down in the
 future. Especially considering that you can't even buy a new laptop
 today with less than 4GB of RAM, and 3GB is becoming the norm for
 smartphones.


OK. Perhaps I can upgrade my RAM (I can't afford a new computer). 
Linking phobos.lib is the first time I've got OOM, I use Firefox 
heavily. phobos.lib is only 10 MB, which is why I thought it odd that 
linking uses well over 1 GB.
Sep 23 2014
parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 23/09/2014 11:20, Nick Treleaven wrote:

 Linking phobos.lib is the first time I've got OOM, I use Firefox
 heavily. phobos.lib is only 10 MB, which is why I thought it odd that
 linking uses well over 1 GB.


I'm now building Phobos 'myself' with win32.mak rather than with Digger, 
and it seems to use under 1 GB (~750 KB). YMMV.

Also I think I was wrong to say just 'linking', it seems to be compiling 
Phobos as well, so it's understandable why it uses that amount of memory.
Sep 30 2014
parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Tuesday, 30 September 2014 at 12:19:05 UTC, Nick Treleaven 
wrote:

 On 23/09/2014 11:20, Nick Treleaven wrote:

 Linking phobos.lib is the first time I've got OOM, I use 
 Firefox
 heavily. phobos.lib is only 10 MB, which is why I thought it 
 odd that
 linking uses well over 1 GB.


 I'm now building Phobos 'myself' with win32.mak rather than 
 with Digger, and it seems to use under 1 GB (~750 KB). YMMV.

 Also I think I was wrong to say just 'linking', it seems to be 
 compiling Phobos as well, so it's understandable why it uses 
 that amount of memory.


I don't think Digger should use much more memory than when 
building things manually. Can you check what's using up memory 
when using Digger?
Sep 30 2014
next sibling parent reply Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 01/10/2014 04:51, Vladimir Panteleev wrote:

 On Tuesday, 30 September 2014 at 12:19:05 UTC, Nick Treleaven wrote:

 On 23/09/2014 11:20, Nick Treleaven wrote:

 Linking phobos.lib is the first time I've got OOM, I use Firefox
 heavily. phobos.lib is only 10 MB, which is why I thought it odd that
 linking uses well over 1 GB.


 I'm now building Phobos 'myself' with win32.mak rather than with
 Digger, and it seems to use under 1 GB (~750 KB). YMMV.

 Also I think I was wrong to say just 'linking', it seems to be
 compiling Phobos as well, so it's understandable why it uses that
 amount of memory.


 I don't think Digger should use much more memory than when building
 things manually. Can you check what's using up memory when using Digger?


OK, I'll try it again. I had been using an old-ish Git checkout of 
Digger, I've updated to 1.0, but I get this error:

$ rdmd --build-only digger
digger.d(6): Error: module wininet is in file 'ae\sys\net\wininet.d' 
which cannot be read
import path[0] = .
import path[1] = c:\D\dmd2\windows\bin\..\..\src\phobos
import path[2] = c:\D\dmd2\windows\bin\..\..\src\druntime\import
Failed: ["dmd", "-v", "-o-", "digger.d", "-I."]

There is no folder 'net' under ae/sys.
Oct 01 2014
parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Wednesday, 1 October 2014 at 15:17:09 UTC, Nick Treleaven 
wrote:

 OK, I'll try it again. I had been using an old-ish Git checkout 
 of Digger, I've updated to 1.0, but I get this error:

 $ rdmd --build-only digger
 digger.d(6): Error: module wininet is in file 
 'ae\sys\net\wininet.d' which cannot be read
 import path[0] = .
 import path[1] = c:\D\dmd2\windows\bin\..\..\src\phobos
 import path[2] = c:\D\dmd2\windows\bin\..\..\src\druntime\import
 Failed: ["dmd", "-v", "-o-", "digger.d", "-I."]

 There is no folder 'net' under ae/sys.


After updating a Git repository with submodules, you need to run 
"git submodule update" to update the submodules too (ae in this 
case).
Oct 01 2014
prev sibling parent Nick Treleaven <ntrel-pub mybtinternet.com> writes:
On 01/10/2014 04:51, Vladimir Panteleev wrote:

 On Tuesday, 30 September 2014 at 12:19:05 UTC, Nick Treleaven wrote:

 On 23/09/2014 11:20, Nick Treleaven wrote:

 Linking phobos.lib is the first time I've got OOM, I use Firefox
 heavily. phobos.lib is only 10 MB, which is why I thought it odd that
 linking uses well over 1 GB.


 I'm now building Phobos 'myself' with win32.mak rather than with
 Digger, and it seems to use under 1 GB (~750 KB). YMMV.




Here I meant 750 MB :-/


 Also I think I was wrong to say just 'linking', it seems to be
 compiling Phobos as well, so it's understandable why it uses that
 amount of memory.


 I don't think Digger should use much more memory than when building
 things manually. Can you check what's using up memory when using Digger?


With both Digger 1.0 and just win32.mak, building latest phobos.lib took 
about 792 MB. Sorry for the noise.
Oct 02 2014
prev sibling parent reply Marco Leise <Marco.Leise gmx.de> writes:
Am Mon, 22 Sep 2014 18:59:12 +0000
schrieb "Vladimir Panteleev" <vladimir thecybershadow.net>:


 On Monday, 22 September 2014 at 17:28:50 UTC, ketmar via 
 Digitalmars-d-announce wrote:

 On Mon, 22 Sep 2014 15:24:55 +0200
 simendsjo via Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:


 My guess is the average for developers is ~8GB. 2GB RAM is 
 really not
 enough for pretty much anything these days - the browser alone 
 easily
 chews 3-4GB on moderate use. I recommend you just upgrade your
 computer (or compile on a better iron).


 are your primary language is java? i'm asking just out of 
 curiousity.


 
 That was uncalled for.
 
 Firefox requires 4GB of memory to build.
 Chromium requires 8GB of memory to build.
 Android requires 16GB of memory to build.


I've been compiling Firefox and Chromium with 4GB of RAM + 4GB
of swap on Linux 64-bit in the past. I don't know if I had to
close any applications for that.


 I think 4GB for a modern programming language's implementation is 
 not an unreasonable requirement, even if it could be brought down 
 in the future. Especially considering that you can't even buy a 
 new laptop today with less than 4GB of RAM, and 3GB is becoming 
 the norm for smartphones.


So why would Apple be able to get away with 1GB on its just
released iPhone 6? Maybe 1048576 kilobytes is enough for
everyone?

-- 
Marco
Sep 30 2014
parent "Sean Kelly" <sean invisibleduck.org> writes:
On Tuesday, 30 September 2014 at 09:35:20 UTC, Marco Leise wrote:

 So why would Apple be able to get away with 1GB on its just
 released iPhone 6? Maybe 1048576 kilobytes is enough for
 everyone?


ARC is more memory efficient than mark & sweep GC like Javascript 
uses.  Though a lot of it is just that iOS developers are simply 
very careful about memory use.  Writing a performant game in iOS 
is really quite hard because of the memory constraints.
Oct 01 2014
prev sibling parent reply "Joakim" <dlang joakim.fea.st> writes:
On Monday, 22 September 2014 at 13:23:33 UTC, simendsjo wrote:

 My guess is the average for developers is ~8GB. 2GB RAM is 
 really not
 enough for pretty much anything these days - the browser alone 
 easily
 chews 3-4GB on moderate use.


You have to admit that this is ridiculous.  I updated to the 
64-bit Chrome on Windows when it came out and it is a huge memory 
hog.  Web browsers have grown out of control.

On Monday, 22 September 2014 at 18:59:13 UTC, Vladimir Panteleev 
wrote:

 Firefox requires 4GB of memory to build.
 Chromium requires 8GB of memory to build.


This is not a requirement for Chromium, merely a recommendation 
for faster builds.  I regularly built Chromium for FreeBSD with 2 
GBs of RAM up till a couple years ago.  Perhaps it has gotten 
much more bloated since or maybe just on Windows, but phobos 
shouldn't be in the same class.


 If you want to work on big projects, you WILL need a decent 
 computer.

 I think 4GB for a modern programming language's implementation 
 is not an unreasonable requirement, even if it could be brought 
 down in the future. Especially considering that you can't even 
 buy a new laptop today with less than 4GB of RAM, and 3GB is 
 becoming the norm for smartphones.


I'd say it's unreasonable from a technical standpoint, maybe not 
that much from an affordability standpoint, which is what you're 
pointing out.  My guess is the real problem is optlink on 
Windows, in which case I recommend that Nick try out the new 
32-bit MSVC toolchain support, if he can't use the existing 
64-bit Windows MSVC integration.

I regularly build git HEAD of dmd/druntime/phobos in a linux VM 
with 512 MB of RAM and about the same amount of swap and have 
never had a problem.  It's only when compiling the unit tests 
that I have to start increasing the allocated RAM.
Sep 23 2014
parent reply simendsjo <simendsjo+dlang gmail.com> writes:
On 09/23/2014 04:48 PM, Joakim wrote:

 On Monday, 22 September 2014 at 13:23:33 UTC, simendsjo wrote:

 My guess is the average for developers is ~8GB. 2GB RAM is really not
 enough for pretty much anything these days - the browser alone easily
 chews 3-4GB on moderate use.


 
 You have to admit that this is ridiculous.  I updated to the 64-bit
 Chrome on Windows when it came out and it is a huge memory hog.  Web
 browsers have grown out of control.


It's well beyond rediculous. After I log into my graphical environment
with everything started, ~200MB is used. Launch a browser, and suddenly
~3800MB is used. If it wasn't for everyone hailing the inner platform
effect (usually seen as a bad thing) as the best thing since sliced
bread, many laptops would probably not ship with 4GB standard. But if
your parents want Facebook and Instagram, you better give them a pretty
beefy computer.

Oh... And the CPU requirements is pretty steep too - even my Lenovo T520
(Core i7) is really slow at browsing the web!

Oh.. And did I forget? You have to run a dynamic programming language
that pushes most mistakes easily caught at compile-time to runtime errors.

I obviously don't think The Web™ has made the right choices.
The amount of Javascript hype I see makes me quite ill. An OS? Really?
In Javascript? Seriously?
Sep 23 2014
parent ketmar via Digitalmars-d-announce <digitalmars-d-announce puremagic.com> writes:
On Tue, 23 Sep 2014 18:29:17 +0200
simendsjo via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:


 But if your parents want Facebook and Instagram, you better give them
 a pretty beefy computer.


i'll give 'em opera 12. yes, it's dead, but it's the only browser that
can work month by month without restarting (no, i'm not kidding!) and
feel itself good in ~300-400 MB of RAM. no FOSS bloatware browser can
compete (alas).
Sep 23 2014
prev sibling parent "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Monday, 22 September 2014 at 10:50:51 UTC, Nick Treleaven 
wrote:

 AFAICT the test suite needs a separate MSYS install from the 
 one Git uses, e.g. for a newer version of 'diff'. Not sure if 
 that makes it harder for Digger to support.


It shouldn't be too hard. The difficult part is getting the 
environment right (e.g. the DMD test suite needs GNU make.exe but 
DigitalMars link.exe).

Downloading / installing more stuff should be easy. See e.g. the 
code for the Git installer:

https://github.com/CyberShadow/ae/blob/master/sys/install/git.d
Sep 22 2014