www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - 64bit DMD on Windows

reply 9il <ilyayaroshenko gmail.com> writes:
Hi,

I am migrating a project to Windows.
DMD fails with

Fatal error: out of memory

Splitting the project to dozen subpackages allows to workaround 
the issue. In other hand, dub test doesn't test subpackages.

It would be nice to have 64 bit DMD.

Best Regards,
Ilya Yaroshenko
Jun 29 2018
next sibling parent reply Seb <seb wilzba.ch> writes:
On Friday, 29 June 2018 at 14:39:29 UTC, 9il wrote:

 Hi,

 I am migrating a project to Windows.
 DMD fails with

 Fatal error: out of memory

 Splitting the project to dozen subpackages allows to workaround 
 the issue. In other hand, dub test doesn't test subpackages.

 It would be nice to have 64 bit DMD.

 Best Regards,
 Ilya Yaroshenko


You can grab one at AppVeyor 
https://ci.appveyor.com/project/greenify/dmd/build/1.0.4829/artifacts (it gets
build with every PR/commit at DMD).
I hope we manage to ship it with the official releases soon.
Jun 29 2018
parent reply bauss <jj_1337 live.dk> writes:
On Friday, 29 June 2018 at 16:11:56 UTC, Seb wrote:

 I hope we manage to ship it with the official releases soon.


I can't wait.

Honestly, a compiler should never run out of memory and it's a 
major issue that DMD does it.

Regardless of that it has fast compilation, it should never run 
out of memory.

I assume it's something that will never be fixed in DMD however, 
because of its memory model.
Jun 29 2018
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Friday, June 29, 2018 20:23:05 bauss via Digitalmars-d wrote:

 On Friday, 29 June 2018 at 16:11:56 UTC, Seb wrote:

 I hope we manage to ship it with the official releases soon.


 I can't wait.

 Honestly, a compiler should never run out of memory and it's a
 major issue that DMD does it.

 Regardless of that it has fast compilation, it should never run
 out of memory.

 I assume it's something that will never be fixed in DMD however,
 because of its memory model.


It could be fixed, even if it's just adding a compiler switch to indicate
that you want it to use the GC to collect memory, because you're going to
run out of memory if you don't. As I understand it, the compiler already can
have that turned on internally. It's just that it hurt compilation times
enough to do so that it's been left off - which makes sense for many, many
cases, but is a definite problem when your project is large enough or doing
enough CTFE that compiling it causes it to run out of memory. What the best
way to handle ti is, I don't know, since we do want the compilation speed in
the typical case, but we also don't want to run out of memory. A compiler
switch would be an easy out but less than ideal.

Regardless of all of that, I expect that newCTFE will have a fairly large
impact on memory consumption once it's finally ready, because the current
CTFE is an absolute memory hog (e.g. as I understand it, it allocates a new
object every time a variable is mutated, so doing something like
incrementing an integer in a loop in CTFE means that the compiler is doing a
_lot_ of allocating). So, depending on why folks run out of memory when
compiling, newCTFE alone may fix the problem for most folks.

Still, we really should have the 64-bit compiler binary available on
Windows. For most developers, there's really no reason to use a 32-bit
binary, and if it solves the running out of memory problem (even if it's
only by being able to throw more memory at the compiler), then that will be
a big help even if it doesn't fix the underlying problem that the compiler
isn't versatile enough to be able to free memory when it needs to without
paying the full cost of memory management when it doesn't need to.

- Jonathan M Davis
Jun 29 2018
next sibling parent reply SonicFreak94 <sonicfreak94 hotmail.com> writes:
On Friday, 29 June 2018 at 20:57:24 UTC, Jonathan M Davis wrote:

 [...]

 It could be fixed, even if it's just adding a compiler switch 
 to indicate that you want it to use the GC to collect memory, 
 because you're going to run out of memory if you don't. As I 
 understand it, the compiler already can have that turned on 
 internally. It's just that it hurt compilation times enough to 
 do so that it's been left off - which makes sense for many, 
 many cases, but is a definite problem when your project is 
 large enough or doing enough CTFE that compiling it causes it 
 to run out of memory. What the best way to handle ti is, I 
 don't know, since we do want the compilation speed in the 
 typical case, but we also don't want to run out of memory. A 
 compiler switch would be an easy out but less than ideal.

 [...]


I managed to get around this issue by using LDC to compile a test 
project, but the memory usage was insane. It used more than 4GB 
of memory. I would rather have slower compile times, personally. 
I bet letting the GC do its job and free memory is less time 
consuming than letting the compiler eventually run out of memory, 
halt, and leave me to scramble to free up system memory so that I 
can try again.
Jun 29 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Fri, Jun 29, 2018 at 11:41:41PM +0000, SonicFreak94 via Digitalmars-d wrote:

 On Friday, 29 June 2018 at 20:57:24 UTC, Jonathan M Davis wrote:

 [...]
 It could be fixed, even if it's just adding a compiler switch to
 indicate that you want it to use the GC to collect memory, because
 you're going to run out of memory if you don't. As I understand it,
 the compiler already can have that turned on internally. It's just
 that it hurt compilation times enough to do so that it's been left
 off - which makes sense for many, many cases, but is a definite
 problem when your project is large enough or doing enough CTFE that
 compiling it causes it to run out of memory.




This has been a showstopper for me on low-memory machines. If I hadn't
been using a high-memory system when I first tried out D, I might have
just walked away.

This *needs* to be an option.

Though the last time I tried compiling DMD with the GC enabled, it
segfaulted on me, so there may be bugs that need to be fixed in order
for this to work.


[...]

 I managed to get around this issue by using LDC to compile a test
 project, but the memory usage was insane. It used more than 4GB of
 memory. I would rather have slower compile times, personally. I bet
 letting the GC do its job and free memory is less time consuming than
 letting the compiler eventually run out of memory, halt, and leave me
 to scramble to free up system memory so that I can try again.


Absolutely agree.  As I said, this issue has been a showstopper for me
on low-memory machines; it basically makes D unusable. (And one of the
environments I was trying to use dmd in is a remote colocated server for
which adding additional memory is not a practical option.)


T

-- 
It only takes one twig to burn down a forest.
Jun 29 2018
parent reply 0xEAB <desisma heidel.beer> writes:
On Friday, 29 June 2018 at 23:57:51 UTC, H. S. Teoh wrote:

 This has been a showstopper for me on low-memory machines. If I 
 hadn't been using a high-memory system when I first tried out 
 D, I might have just walked away.


Confirmed. On a Raspberry Pi D compilers are completely useless.

For example, I can compile huge C projects like the PHP 
interpreter without a problem (it's horribly slow, though of 
course 😌), but I run out of memory with anything a bit bigger 
written in D.

Well, this applies to LDC, at least (I've got no plan if the LLVM 
backend frees memory). Haven't tried GDC yet (distro packages are 
as outdated as GDC's releases, and I'm too lazy to compile that 
thing myself). DMD is not available for ARM.


In my humble opinion, there should be a (compile-time?) GC switch 
for the frontend.

Sorry if this post has any wrong assumptions. Feel free to 
correct me :)


Kind regards,
Elias
Jun 30 2018
parent Joakim <dlang joakim.fea.st> writes:
On Saturday, 30 June 2018 at 22:38:13 UTC, 0xEAB wrote:

 On Friday, 29 June 2018 at 23:57:51 UTC, H. S. Teoh wrote:

 This has been a showstopper for me on low-memory machines. If 
 I hadn't been using a high-memory system when I first tried 
 out D, I might have just walked away.


 Confirmed. On a Raspberry Pi D compilers are completely useless.

 For example, I can compile huge C projects like the PHP 
 interpreter without a problem (it's horribly slow, though of 
 course 😌), but I run out of memory with anything a bit bigger 
 written in D.

 Well, this applies to LDC, at least (I've got no plan if the 
 LLVM backend frees memory).


LDC uses the same GC-less D frontend as DMD, that's likely the 
issue, not the llvm backend.
Jun 30 2018
prev sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 30/06/2018 8:57 AM, Jonathan M Davis wrote:

 It could be fixed, even if it's just adding a compiler switch to indicate
 that you want it to use the GC to collect memory


Currently leaks memory like crazy.
Jun 29 2018
prev sibling parent pedro6655 <pedro6655 yahoo.com> writes:
I think you should consult this matter with 
https://mailhelp.net/yahoo-support/. they will definitely solve 
your issue. Whenever I faced any issue, every time I consult with 
them. So I am suggesting their name to you.
Jun 29 2018