www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Modern C++ Lamentations

reply Walter Bright <newshound2 digitalmars.com> writes:
http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

Time to show off your leet D skilz and see how good we can do it in D!
Dec 29 2018
next sibling parent rikki cattermole <rikki cattermole.co.nz> writes:
On 29/12/2018 10:29 PM, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do it in D!
Reddit thread: https://www.reddit.com/r/programming/comments/aac4hg/modern_c_lamentations/
Dec 29 2018
prev sibling next sibling parent reply JN <666total wp.pl> writes:
On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do 
 it in D!
I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 . Here's another take on language complexity, related to Rust - https://graydon2.dreamwidth.org/263429.html
Dec 29 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do it in
 D!
I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
[...] Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds. Let me say that again. Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed. The mere act of using std.regex causes compilation times to TRIPLE. Let that sink in for a moment. These days, it has been really hard for me to boast about D compilation times with a straight face. True, if you write C-style D code, then compilation *is* lightning fast. But modern D has moved away from that style, and the kind of style that modern D code takes on these days is template- and CTFE-heavy, both areas of which are in the "extremely slow" category of D compilation. This seriously needs to improve if we're going to continue boasting about compilation times, because otherwise it's becoming more and more like false advertisement when we talk about D being fast to compile. I've been saying this ever since we adopted that cringe-worthy fast-fast-fast slogan on dlang.org, and it seems not much has improved since then. These days I'm almost ashamed to talk about D compilation times. :-/ And don't get me started on dmd's ridiculous memory usage requirements, which makes it basically unusable on low-memory systems. There is not even an option to trade off compilation speed for less memory usage. We're paying mandatory memory tax yet we're still unable to improve compilation speed of std.regex to within acceptable limits. And modern C++ compilers have improved enough that on said low-memory systems, compiling C++ can be actually *faster* than D because g++ stays within available RAM and therefore isn't thrashing on swap, whereas dmd thrashes on swap like crazy because it's such a memory hog. And that's if dmd even finishes compiling at all, before it gets hit by the OOM killer, otherwise we're looking at finite compilation time vs. infinite compilation time (because it never finishes). Don't get me wrong, I still love D for having the best power to ease of writing ratio, and I can't see myself going back to C++ in the foreseeable future (or ever again). But we seriously need to improve on these two areas before we start proclaiming how good D compilation times are, because that's no longer true. T -- A bend in the road is not the end of the road unless you fail to make the turn. -- Brian White
Dec 29 2018
next sibling parent reply Adam D. Ruppe <destructionator gmail.com> writes:
On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
 Yeah no kidding, recently I rewrote a whole bunch of code to 
 get *rid* of dependency on std.regex because it was too slow, 
 and project compilation time improved from about 7+ seconds
Ditto. (Basically). Rewriting the uri parser in cgi.d brought its build time from about 3 seconds down to 0.6. Which still feels slow, especially next to my minigui.d, which can do 0.3 since it broke off from Phobos entirely! (It was 2.5 seconds before).
 But we seriously need to improve on these two areas before we 
 start proclaiming how good D compilation times are, because 
 that's no longer true.
Amen. I have boxes that have infinite D build times because it is impossible to run the sloppy compiler!
Dec 29 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sat, Dec 29, 2018 at 03:57:57PM +0000, Adam D. Ruppe via Digitalmars-d wrote:
 On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
 Yeah no kidding, recently I rewrote a whole bunch of code to get
 *rid* of dependency on std.regex because it was too slow, and
 project compilation time improved from about 7+ seconds
Ditto. (Basically). Rewriting the uri parser in cgi.d brought its build time from about 3 seconds down to 0.6. Which still feels slow, especially next to my minigui.d, which can do 0.3 since it broke off from Phobos entirely! (It was 2.5 seconds before).
IME, Phobos isn't all bad. Some Phobos modules are pretty useful, esp. std.algorithm and std.range, and despite std.algorithm being pretty huge (its submodules are pretty huge even after I split up the original std/algorithm.d -- which was so bad I couldn't run its unittests locally anymore because it ran out of memory, on a machine with 4GB RAM) it actually compiles very fast, proportional to how much you use it. The bad apples are std.regex, std.format, and maybe a few others, that rely a little too heavily on recursive templates and CTFE.
 But we seriously need to improve on these two areas before we start
 proclaiming how good D compilation times are, because that's no
 longer true.
Amen. I have boxes that have infinite D build times because it is impossible to run the sloppy compiler!
Yeah, this is one reason I haven't dared propose using D at my day job. I'd be laughed out of the CTO's office if dmd ran out of memory on a trivial regex test. And my coworkers will hate me so much for making the already RAM-heavy build system soak up even more memory. Memory may be cheap these days, but it's still not free, and it's still a finite resource. T -- Customer support: the art of getting your clients to pay for your own incompetence.
Dec 29 2018
prev sibling parent Manu <turkeyman gmail.com> writes:
On Sat, Dec 29, 2018 at 8:00 AM Adam D. Ruppe via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
 Yeah no kidding, recently I rewrote a whole bunch of code to
 get *rid* of dependency on std.regex because it was too slow,
 and project compilation time improved from about 7+ seconds
Ditto. (Basically). Rewriting the uri parser in cgi.d brought its build time from about 3 seconds down to 0.6. Which still feels slow, especially next to my minigui.d, which can do 0.3 since it broke off from Phobos entirely! (It was 2.5 seconds before).
I ran into this the other day: https://github.com/dlang/druntime/pull/2398#issuecomment-445690050 That is `max`, can you imagine a theoretically simpler function? Phobos is chaos!
Jan 02
prev sibling parent reply Ivan Kazmenko <gassa mail.ru> writes:
On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
 On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d 
 wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
 wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can 
 do it in D!
I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
[...] Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds. Let me say that again. Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed. The mere act of using std.regex causes compilation times to TRIPLE. Let that sink in for a moment.
I thought I'd chime in with a similar experience. I have this small library, testlib.d, used to check text outputs for programming contest problems (used internally, no public repository right now). As one can imagine, a regular expression is sometimes handy to parse simple alphanumeric constructs and check their format. But after using std.regex a bit, I put the import down into one function and templated it, and mostly stopped using the function in new checkers' code, developing some not-so-nice workarounds instead. A typical checker is very short, a couple dozen to a couple hundred lines, and this change brings compile times from 5+ seconds to 2+ seconds. Sorry, I just can't stand having to wait 5+ seconds to compile a 50-liner, repeatedly, when the development time for a single checker is on the scale of minutes, not hours. The C++ analog library (https://github.com/MikeMirzayanov/testlib) is currently 4500+ lines long, and so also takes 5+ seconds to compile a trivial checker. This was actually one of the incentives for me to switch to D with a homebrew library for writing checkers. I know there was a move to make Phobos compilation faster a few months ago, and things seemed to improve a little then, but std.regex still adds seconds of compilation time. Ivan Kazmenko.
Dec 29 2018
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/29/18 7:18 PM, Ivan Kazmenko wrote:
 On Saturday, 29 December 2018 at 15:34:19 UTC, H. S. Teoh wrote:
 On Sat, Dec 29, 2018 at 10:35:05AM +0000, JN via Digitalmars-d wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 Time to show off your leet D skilz and see how good we can > do 
it in D! I don't know if D is a role model here, considering just importing std.regex adds three seconds to compile time - https://issues.dlang.org/show_bug.cgi?id=18378 .
[...] Yeah no kidding, recently I rewrote a whole bunch of code to get *rid* of dependency on std.regex because it was too slow, and project compilation time improved from about 7+ seconds to 2+ seconds. Let me say that again.  Removing dependency on std.regex (by writing equivalent functionality by hand) improved compilation times from more than SEVEN seconds to just over TWO seconds. That's almost TRIPLE the compilation speed.  The mere act of using std.regex causes compilation times to TRIPLE. Let that sink in for a moment.
I thought I'd chime in with a similar experience.  I have this small library, testlib.d, used to check text outputs for programming contest problems (used internally, no public repository right now).  As one can imagine, a regular expression is sometimes handy to parse simple alphanumeric constructs and check their format.  But after using std.regex a bit, I put the import down into one function and templated it, and mostly stopped using the function in new checkers' code, developing some not-so-nice workarounds instead.  A typical checker is very short, a couple dozen to a couple hundred lines, and this change brings compile times from 5+ seconds to 2+ seconds. Sorry, I just can't stand having to wait 5+ seconds to compile a 50-liner, repeatedly, when the development time for a single checker is on the scale of minutes, not hours.  The C++ analog library (https://github.com/MikeMirzayanov/testlib) is currently 4500+ lines long, and so also takes 5+ seconds to compile a trivial checker.  This was actually one of the incentives for me to switch to D with a homebrew library for writing checkers. I know there was a move to make Phobos compilation faster a few months ago, and things seemed to improve a little then, but std.regex still adds seconds of compilation time.
Hmmm, I thought Dmitry was successful at eliminating most overheads of importing and not using std.regex. Perhaps a second pass would be needed?
Dec 30 2018
parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 30, 2018 at 09:45:11AM -0500, Andrei Alexandrescu via Digitalmars-d
wrote:
[...]
 Hmmm, I thought Dmitry was successful at eliminating most overheads of
 importing and not using std.regex. Perhaps a second pass would be
 needed?
I think the cost of importing without actually using std.regex should have been fixed by now, but the complaint here is importing and *using* std.regex, which would be the important use case. :-D Understandably, some amount of cost would have to be paid to actually use the module, but given our fast-fast-fast slogan, should it really add *5 seconds* to compilation time just to use a couple of near-trivial regexes? T -- Why can't you just be a nonconformist like everyone else? -- YHL
Dec 30 2018
prev sibling next sibling parent Rubn <where is.this> writes:
 Issues with “Everything is a library” C++
That kind of mentality sounds familiar, can't put my finger on it though. Probably won't get anything as nice as what C# does: var triples = from z in Enumerable.Range(1, int.MaxValue) from x in Enumerable.Range(1, z) from y in Enumerable.Range(x, z) where x*x+y*y==z*z select (x:x, y:y, z:z); foreach (var t in triples.Take(100)) { Console.WriteLine($"({t.x},{t.y},{t.z})"); } At least not creating an "object" that represents a triple in that sense that is readable and doesn't involve mixins.
Dec 29 2018
prev sibling next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/29/18 4:29 AM, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do it in D!
Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one. I tried my best, but it kind of sucks: foreach(z, x, y; iota(size_t.max) .map!(a => zip(StoppingPolicy.shortest, a.repeat, iota(1, a))) .joiner .map!(t => zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0]))) .joiner .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2]) .take(100)) { writeln(x, " ", y, " ", z); } Now, a specialized range looks much better and more efficient to me. This is essentially what the author wrote. And as is typical for D, we can split the tasks into more generic pieces. For instance: struct PossiblePythags(T) { T z = 2; T x = 1; T y = 1; auto front() { return tuple!(T, "x", T, "y", T, "z")(x, y, z); } void popFront() { if(++y == z) { if(++x == z) { ++z; x = 1; } y = x; } } enum empty = false; } auto pythagrange(T = size_t)() { return PossiblePythags!T() .filter!(p => p.x * p.x + p.y * p.y == p.z * p.z); } No measuring of either compilation or runtime, but I would expect the specialized range version to be pretty close to the author's measurements. I'm wondering if some generic "emulate N nested loops" with given stopping and starting conditions might be a useful addition for std.range or std.algorithm. I'm thinking of other looping algorithms like Floyd Warshall that might benefit from such building blocks. -Steve
Dec 29 2018
next sibling parent Timon Gehr <timon.gehr gmx.ch> writes:
On 29.12.18 23:01, Steven Schveighoffer wrote:
 
 I'm wondering if some generic "emulate N nested loops" with given 
 stopping and starting conditions might be a useful addition for 
 std.range or std.algorithm. I'm thinking of other looping algorithms 
 like Floyd Warshall that might benefit from such building blocks.
 
 -Steve
cartesianProduct suffices for Floyd-Warshall: cartesianProduct(iota(n),iota(n),iota(n)) .each!((k,i,j){ d[i][j]=min(d[i][j],d[i][k]+d[k][j]); }); For loops where nested ranges depend on outer indices, 'then' goes a long way. It is also easy to do something like: mixin(comp!q{(i,j,k) | i in iota(n), j in iota(i), k in iota(j)}) This would expand to: iota(n).then!(i=>iota(i).then!(j=>iota(j).map!(k=>tuple(i,j,k)))) (Of course, right now, tuples are somewhat inconvenient to use.)
Dec 29 2018
prev sibling next sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 29 December 2018 at 22:01:58 UTC, Steven 
Schveighoffer wrote:
 On 12/29/18 4:29 AM, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do 
 it in D!
Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one. I tried my best, but it kind of sucks: foreach(z, x, y; iota(size_t.max) .map!(a => zip(StoppingPolicy.shortest, a.repeat, iota(1, a))) .joiner .map!(t => zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0]))) .joiner .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2]) .take(100)) { writeln(x, " ", y, " ", z); }
Or if you can bear the closures iota(1, size_t.max) .map!(z => iota(1, z + 1) .map!(x => iota(x, z + 1) .map!(y => tuple!("x", "y", "z")(x, y, z))) .joiner) .joiner .filter!(t => t.x^^2 + t.y^^2 == t.z^^2);
Dec 30 2018
parent Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 30 December 2018 at 12:22:01 UTC, John Colvin wrote:
 Or if you can bear the closures

     iota(1, size_t.max)
         .map!(z => iota(1, z + 1)
               .map!(x => iota(x, z + 1)
                     .map!(y => tuple!("x", "y", "z")(x, y, z)))
               .joiner)
         .joiner
         .filter!(t => t.x^^2 + t.y^^2 == t.z^^2);
But this is still unreadable if you are not a D annointed range ninja.
Dec 30 2018
prev sibling parent reply John Colvin <john.loughran.colvin gmail.com> writes:
On Saturday, 29 December 2018 at 22:01:58 UTC, Steven 
Schveighoffer wrote:
 On 12/29/18 4:29 AM, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do 
 it in D!
Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one. I tried my best, but it kind of sucks: foreach(z, x, y; iota(size_t.max) .map!(a => zip(StoppingPolicy.shortest, a.repeat, iota(1, a))) .joiner .map!(t => zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0]))) .joiner .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2]) .take(100)) { writeln(x, " ", y, " ", z); }
Isn't "StoppingPolicy.shortest" the default?
Dec 30 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/30/18 7:27 AM, John Colvin wrote:
 On Saturday, 29 December 2018 at 22:01:58 UTC, Steven Schveighoffer wrote:
 On 12/29/18 4:29 AM, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do it in D!
Ugh, ranges really aren't a good fit for emulating nested loops, unless you write a specialized one. I tried my best, but it kind of sucks:     foreach(z, x, y;     iota(size_t.max)         .map!(a =>              zip(StoppingPolicy.shortest, a.repeat, iota(1, a)))         .joiner         .map!(t =>              zip(StoppingPolicy.shortest, t[0].repeat, t[1].repeat, iota(t[1], t[0])))         .joiner         .filter!(t => t[0]*t[0] == t[1]*t[1] + t[2]*t[2])         .take(100))     {         writeln(x, " ", y, " ", z);     }
Isn't "StoppingPolicy.shortest" the default?
Maybe :) I didn't spend a lot of time examining the details. I also like your way, it's much more readable, but I still don't like the joiners. There has to be a way to just store the three ranges and iterate them properly, something like: struct Triples(ZRange, alias r2func, alias r3func) { ZRange z; typeof(r2func(z.front)) x; typeof(r3func(z.front, x.front)) y; auto front() { return tuple(z.front, x.front, y.front); } void popFront() { y.popFront; if(y.empty) { scope(exit) if(!x.empty) y = r3func(z.front, x.front) x.popFront; if(x.empty) { scope(exit) if(!z.empty) x = r2func(z.front); z.popFront; } } } bool empty() { return y.empty } } auto pythags = iota(size_t.max).triples!(z => iota(1 .. z), (z, x) => iota(x, z)).filter!(t => t[1]*t[1] + t[2]*t[2] == t[0]*t[0]); Maybe split it out into something like withloop, maybe we can make this kind of thing work: auto pythags = iota(size_t.max) .withLoop!(z => iota(1, z)) .withLoop!((z, x) => iota(x, z)) .filter!(t => t[1]*t[1] + t[2]*t[2] == t[0]*t[0]); Which is really similar to the loop design. -Steve
Dec 30 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/30/18 1:13 PM, Steven Schveighoffer wrote:
 Maybe split it out into something like withloop, maybe we can make this 
 kind of thing work:
 
There you go, nested looping with ranges, with no need for closures or joiners: https://run.dlang.io/is/hkqhvZ :) OK, spent enough time on this... I have to stop before it consumes my whole weekend. -Steve
Dec 30 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/30/18 1:47 PM, Steven Schveighoffer wrote:
 On 12/30/18 1:13 PM, Steven Schveighoffer wrote:
 Maybe split it out into something like withloop, maybe we can make 
 this kind of thing work:
There you go, nested looping with ranges, with no need for closures or joiners: https://run.dlang.io/is/hkqhvZ
BTW, this would be much nicer with first-class tuples... -Steve
Dec 30 2018
prev sibling next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 29.12.18 10:29, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
 
 Time to show off your leet D skilz and see how good we can do it in D!
Current D: --- import std.range, std.algorithm, std.typecons; import std.bigint, std.stdio; alias then(alias a)=(r)=>map!a(r).joiner; void main(){ auto triples=recurrence!"a[n-1]+1"(1.BigInt) .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z)))) .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2); triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } --- tuple-syntax branch at https://github.com/tgehr/dmd/tree/tuple-syntax --- import std.range, std.algorithm, std.typecons; import std.bigint, std.stdio; alias then(alias a)=(r)=>map!a(r).joiner; void main(){ auto triples=recurrence!"a[n-1]+1"(1.BigInt) .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>(x,y,z)))) .filter!((x,y,z)=>x^^2+y^^2==z^^2); triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } ---
Dec 29 2018
next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 12/29/18 9:03 PM, Timon Gehr wrote:
 On 29.12.18 10:29, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do it in D!
Current D: --- import std.range, std.algorithm, std.typecons; import std.bigint, std.stdio; alias then(alias a)=(r)=>map!a(r).joiner; void main(){     auto triples=recurrence!"a[n-1]+1"(1.BigInt)         .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))         .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2);     triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } --- tuple-syntax branch at https://github.com/tgehr/dmd/tree/tuple-syntax --- import std.range, std.algorithm, std.typecons; import std.bigint, std.stdio; alias then(alias a)=(r)=>map!a(r).joiner; void main(){     auto triples=recurrence!"a[n-1]+1"(1.BigInt)         .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>(x,y,z))))         .filter!((x,y,z)=>x^^2+y^^2==z^^2);     triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } ---
The "then" abstraction is pretty awesome. Thanks!
Dec 30 2018
parent reply Jacob Carlborg <doob me.com> writes:
On 2018-12-30 15:46, Andrei Alexandrescu wrote:
 On 12/29/18 9:03 PM, Timon Gehr wrote:
 alias then(alias a)=(r)=>map!a(r).joiner;
The "then" abstraction is pretty awesome. Thanks!
Isn't that usually called "flatMap"? -- /Jacob Carlborg
Dec 30 2018
parent reply sarn <sarn theartofmachinery.com> writes:
On Sunday, 30 December 2018 at 16:51:07 UTC, Jacob Carlborg wrote:
 On 2018-12-30 15:46, Andrei Alexandrescu wrote:
 On 12/29/18 9:03 PM, Timon Gehr wrote:
 alias then(alias a)=(r)=>map!a(r).joiner;
The "then" abstraction is pretty awesome. Thanks!
Isn't that usually called "flatMap"?
Yeah, in Haskell monad terminology, "then" means >>, but flatMap is >>= ("bind"). So flatMap is a less confusing name for some people.
Dec 30 2018
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 30.12.18 22:12, sarn wrote:
 On Sunday, 30 December 2018 at 16:51:07 UTC, Jacob Carlborg wrote:
 On 2018-12-30 15:46, Andrei Alexandrescu wrote:
 On 12/29/18 9:03 PM, Timon Gehr wrote:
 alias then(alias a)=(r)=>map!a(r).joiner;
The "then" abstraction is pretty awesome. Thanks!
Isn't that usually called "flatMap"?
Yeah, in Haskell monad terminology, "then" means >>, but flatMap is >>= ("bind").  So flatMap is a less confusing name for some people.
Pun intended. Welcome to D, where 'enum' means 'const', 'const' means 'readonly', 'lazy' means 'by name', 'assert' means 'assume' and 'real' does not mean 'real' (in fact, I really like the 'ireal' and 'creal' keywords, pity they are being phased out). :) The wider context is that I have argued many times that it makes sense to put 'trivial' one-liners like this one into Phobos even if for no other reason than to standardize their names.
Dec 31 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/31/2018 2:28 PM, Timon Gehr wrote:
 Welcome to D, where 'enum' means 'const', 'const' means 
 'readonly', 'lazy' means 'by name', 'assert' means 'assume' and 'real' does
not 
 mean 'real' (in fact, I really like the 'ireal' and 'creal' keywords, pity
they 
 are being phased out). :)
D's "by name" are the template alias parameters.
Dec 31
parent reply Timon Gehr <timon.gehr gmx.ch> writes:
On 01.01.19 07:21, Walter Bright wrote:
 On 12/31/2018 2:28 PM, Timon Gehr wrote:
 Welcome to D, where 'enum' means 'const', 'const' means 'readonly', 
 'lazy' means 'by name', 'assert' means 'assume' and 'real' does not 
 mean 'real' (in fact, I really like the 'ireal' and 'creal' keywords, 
 pity they are being phased out). :)
D's "by name" are the template alias parameters.
I think alias parameters are not "by name" they are something like "by symbol". ("by name" is a bit confusing of a name, the argument expression need not really have a "name".) The standard PL jargon is this: https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name https://en.wikipedia.org/wiki/Lazy_evaluation
Jan 03
parent Walter Bright <newshound2 digitalmars.com> writes:
On 1/3/2019 11:26 AM, Timon Gehr wrote:
 On 01.01.19 07:21, Walter Bright wrote:
 On 12/31/2018 2:28 PM, Timon Gehr wrote:
 Welcome to D, where 'enum' means 'const', 'const' means 'readonly', 'lazy' 
 means 'by name', 'assert' means 'assume' and 'real' does not mean 'real' (in 
 fact, I really like the 'ireal' and 'creal' keywords, pity they are being 
 phased out). :)
D's "by name" are the template alias parameters.
I think alias parameters are not "by name" they are something like "by symbol". ("by name" is a bit confusing of a name, the argument expression need not really have a "name".) The standard PL jargon is this: https://en.wikipedia.org/wiki/Evaluation_strategy#Call_by_name https://en.wikipedia.org/wiki/Lazy_evaluation
Thanks for the info. It seems the explanations are a little slippery, and D's usage doesn't precisely fit. For example, D's lazy parameters can be used to implement control structures, as it says lazy evaluation is for, but then seems a bit ambiguous about whether the lazy evaluation is done once or many times.
Jan 03
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/29/2018 6:03 PM, Timon Gehr wrote:
 Current D:
 ---
 import std.range, std.algorithm, std.typecons;
 import std.bigint, std.stdio;
 alias then(alias a)=(r)=>map!a(r).joiner;
 void main(){
      auto triples=recurrence!"a[n-1]+1"(1.BigInt)
          .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z))))
          .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2);
      triples.each!((x,y,z){ writeln(x," ",y," ",z); });
 }
 ---
I never would have thought of that. Thank you!
Dec 30 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do 
 it in D!
I'm on it. As I write, I'm timing (compile and run time) several C++, D, and Rust implementations and writing a blog post. I'm only on the 2nd implementation but D is winning... :) I'm going to shamelessly steal Timon's range code in this thread and the generator one posted on reddit.
Dec 30 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/30/2018 5:18 AM, Atila Neves wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do it in D!
I'm on it. As I write, I'm timing (compile and run time) several C++, D, and Rust implementations and writing a blog post. I'm only on the 2nd implementation but D is winning... :) I'm going to shamelessly steal Timon's range code in this thread and the generator one posted on reddit.
Wow! I'm looking forward to it.
Dec 30 2018
parent reply Atila Neves <atila.neves gmail.com> writes:
On Sunday, 30 December 2018 at 22:31:27 UTC, Walter Bright wrote:
 On 12/30/2018 5:18 AM, Atila Neves wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
 wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do 
 it in D!
I'm on it. As I write, I'm timing (compile and run time) several C++, D, and Rust implementations and writing a blog post. I'm only on the 2nd implementation but D is winning... :) I'm going to shamelessly steal Timon's range code in this thread and the generator one posted on reddit.
Wow! I'm looking forward to it.
Blog: https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-triples-in-c-d-and-rust/ Reddit: https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_triples_in_c_d_and_rust/ Hacker news something something dark side.
Dec 31 2018
next sibling parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/31/18 8:20 AM, Atila Neves wrote:
 On Sunday, 30 December 2018 at 22:31:27 UTC, Walter Bright wrote:
 On 12/30/2018 5:18 AM, Atila Neves wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do it in D!
I'm on it. As I write, I'm timing (compile and run time) several C++, D, and Rust implementations and writing a blog post. I'm only on the 2nd implementation but D is winning... :) I'm going to shamelessly steal Timon's range code in this thread and the generator one posted on reddit.
Wow! I'm looking forward to it.
Blog: https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-trip es-in-c-d-and-rust/ Reddit: https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_trip es_in_c_d_and_rust/ Hacker news something something dark side.
I tried the version I came up with with a dedicated looping range (https://run.dlang.io/is/hkqhvZ), cuts the range version down to 550ms (my timings are on my macbook, but my baseline was literally exactly what yours was, 154ms). In any case, it does seem like the optimizer/compiler is much better at dealing with nested loops than 3 ranges. This is probably a case where the low level code is so fast (doing 3 multiplications, an addition and an equality check) that the loop processing becomes significant. -Steve
Dec 31 2018
parent reply Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/31/18 10:15 AM, Steven Schveighoffer wrote:
 On 12/31/18 8:20 AM, Atila Neves wrote:
 On Sunday, 30 December 2018 at 22:31:27 UTC, Walter Bright wrote:
 On 12/30/2018 5:18 AM, Atila Neves wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/

 Time to show off your leet D skilz and see how good we can do it in D!
I'm on it. As I write, I'm timing (compile and run time) several C++, D, and Rust implementations and writing a blog post. I'm only on the 2nd implementation but D is winning... :) I'm going to shamelessly steal Timon's range code in this thread and the generator one posted on reddit.
Wow! I'm looking forward to it.
Blog: https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-trip es-in-c-d-and-rust/ Reddit: https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_trip es_in_c_d_and_rust/ Hacker news something something dark side.
I tried the version I came up with with a dedicated looping range (https://run.dlang.io/is/hkqhvZ), cuts the range version down to 550ms (my timings are on my macbook, but my baseline was literally exactly what yours was, 154ms). In any case, it does seem like the optimizer/compiler is much better at dealing with nested loops than 3 ranges. This is probably a case where the low level code is so fast (doing 3 multiplications, an addition and an equality check) that the loop processing becomes significant.
And the answer was provided by someone who examined the compiler output: https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_triples_in_c_d_and_rust/ecy6rqn/ And it fixes the problem with Timon's D version as well, new time down to 172ms (see my reply to that post). -Steve
Dec 31 2018
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/31/2018 8:05 AM, Steven Schveighoffer wrote:
 And the answer was provided by someone who examined the compiler output:
 
 https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_triples_in_c_d_and_rust/ecy6rqn/
For reference, the reddit comment is: "Looking at the generated code in compiler explorer, it looks like the Rust compiler is not hoisting the multiplications out of the loops, while the C++ compiler (I used clang) does. Furthermore, it seems like using `for x in y..=z` etc results in quite convoluted conditions. This code seems to perform the same as the C++: https://godbolt.org/z/-nzALh It looks like there's some things to fix in the rust compiler.."
 And it fixes the problem with Timon's D version as well, new time down to
172ms 
One thing to consider is optimizers are designed by looking at the code generated by popular programming languages and popular programming methods. Hence the traditional loop structure gets a heluva lot of attention. The range version is fairly new, and so it will likely do less well.
 (see my reply to that post).
Your reply is: "Ooh, that's interesting. Same issue with the D version. I had to work a bit on it, but this does work and is 172ms vs the ~1000ms: return recurrence!"a[n-1]+1"(1) .then!((z) { auto ztotal = z * z; return iota(1, z + 1).then!((x) { auto xtotal = x * x; return iota(x, z + 1) .filter!(y => y * y + xtotal == total) .map!(y => tuple(x,y,z)); }); }); " Atila, can you please update the blog post?
Dec 31 2018
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 12/31/18 1:43 PM, Walter Bright wrote:
 On 12/31/2018 8:05 AM, Steven Schveighoffer wrote:
  > (see my reply to that post).
 
 Your reply is:
 
 "Ooh, that's interesting. Same issue with the D version.
 I had to work a bit on it, but this does work and is 172ms vs the ~1000ms:
 
    return
      recurrence!"a[n-1]+1"(1)
      .then!((z) {
         auto ztotal = z * z;
         return iota(1, z + 1).then!((x) {
             auto xtotal = x * x;
             return iota(x, z + 1)
                .filter!(y => y * y + xtotal == total)
                .map!(y => tuple(x,y,z));
                });
         });
 "
 
 Atila, can you please update the blog post?
This isn't a fair comparison though -- I'm doing work here that in the other languages the compiler is doing (hoisting the multiplications outside the inner loops). It's not a straight port. I just wanted to point out that this accounts for how the rust and C++ versions are faster than the D range versions. It would be good to look into why the D compilers are not seeing that optimization possibility. -Steve
Dec 31 2018
prev sibling next sibling parent Jon Degenhardt <jond noreply.com> writes:
On Monday, 31 December 2018 at 13:20:35 UTC, Atila Neves wrote:
 Blog:

 https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-triples-in-c-d-and-rust/

 Reddit:

 https://www.reddit.com/r/programming/comments/ab71ag/comparing_pythagorean_triples_in_c_d_and_rust/


 Hacker news something something dark side.
LTO often helps optimize range code (and phobos code generally), so I tried it out on the variants in your repo. Indeed, the range variants improve, but not the others. Still doesn't bring the range version into line with other three variants. Compile lines: * No LTO: ldc2 -O2 * LTO: ldc2 -O2 -flto=thin -defaultlib=phobos2-ldc-lto,druntime-ldc-lto LDC 1.13.0; Macbook Pro, 16GB RAM. The runtime for simple, lambda, and generator variants came in at 120ms each for both LTO off and LTO on. The range version was 1080ms for no LTO, and 780ms with LTO. I bumped N up to 3000 to see if more differentiation would show up. There was some, but overall the results were still very consistent with N at 1000. --Jon
Dec 31 2018
prev sibling parent reply Dukc <ajieskola gmail.com> writes:
On Monday, 31 December 2018 at 13:20:35 UTC, Atila Neves wrote:
 Blog:

 https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-triples-in-c-d-and-rust/
Isn't the main problem with performance of the Timon's range loop that it uses arbitrary-sized integers (BigInts)? I took his example, and modified it to this: import std.experimental.all; import std.datetime.stopwatch : AutoStart, StopWatch; alias then(alias a)=(r)=>map!a(r).joiner; void main(){ auto sw = StopWatch(AutoStart.no); if (true) { sw.start; scope (success) sw.stop; auto triples=recurrence!"a[n-1]+1"(1.BigInt) .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z)))) .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2) .until!(t=>t[2] >= 500); triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } writefln("Big int time is %s microseconds", sw.peek.total!"usecs"); sw.reset; if (true) { sw.start; scope (success) sw.stop; auto triples=recurrence!"a[n-1]+1"(1L) .then!(z=>iota(1,z+1).then!(x=>iota(x,z+1).map!(y=>tuple(x,y,z)))) .filter!((t)=>t[0]^^2+t[1]^^2==t[2]^^2) .until!(t=>t[2] >= 500); triples.each!((x,y,z){ writeln(x," ",y," ",z); }); } writefln("Long int time is %s microseconds", sw.peek.total!"usecs"); return; } The output, LDC version being 1.11.0-beta2, with: dub --compiler=ldc2 --build=release was: 3 4 5 6 8 10 5 12 13 9 12 15 8 15 17 [...snip...] 155 468 493 232 435 493 340 357 493 190 456 494 297 396 495 Big int time is 4667925 microseconds 3 4 5 6 8 10 5 12 13 9 12 15 8 15 17 [...snip...] 155 468 493 232 435 493 340 357 493 190 456 494 297 396 495 Long int time is 951821 microseconds That is almost five times as fast. Assuming the factor would be the same in your blog, doesn't that account for most of the difference between performance of D ranged and other versions? The remaining difference might be explained with bounds-checking.
Jan 04
parent Steven Schveighoffer <schveiguy gmail.com> writes:
On 1/4/19 9:00 AM, Dukc wrote:
 On Monday, 31 December 2018 at 13:20:35 UTC, Atila Neves wrote:
 Blog:

 https://atilanevesoncode.wordpress.com/2018/12/31/comparing-pythagorean-trip
es-in-c-d-and-rust/ 
Isn't the main problem with performance of the Timon's range loop that it uses arbitrary-sized integers (BigInts)?
Atila's version of that code doesn't use bigints: https://github.com/atilaneves/pythagoras/blob/master/range.d#L24 The major problem with the D range implementation is that the compiler isn't able to find the optimization of hoisting the multiplication of the outer indexes out of the inner loop. See my responses to Atila in this thread. -Steve
Jan 04
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
It says:
 C++ compilation times have been a source of pain in every 
 non-trivial-size codebase I’ve worked on. Don’t believe me? Try 
 building one of the widely available big codebases (any of: 
 Chromium, Clang/LLVM, UE4 etc will do). Among the things I 
 really want out of C++, “solve compile times” is probably #1 on 
 the list, and has been since forever.
I agree, it has been #1 problem with C++ in years, and only getting worse. There is the theory (D builds fast) and the application (DUB often negate the advantage, you need to avoid templatitis).
Dec 30 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 30, 2018 at 01:25:33PM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
It says:
 C++ compilation times have been a source of pain in every
 non-trivial-size codebase I’ve worked on. Don’t believe me? Try
 building one of the widely available big codebases (any of:
 Chromium, Clang/LLVM, UE4 etc will do). Among the things I really
 want out of C++, “solve compile times” is probably #1 on the list,
 and has been since forever.
I agree, it has been #1 problem with C++ in years, and only getting worse. There is the theory (D builds fast) and the application (DUB often negate the advantage, you need to avoid templatitis).
D theory sounds all good and all, but in practice you have warts like dub (one big reason I stay away from it -- though based on what Sonke said recently, performance may have improved since I last checked), std.regex (after the last big refactor, something Really Bad happened to its compile times -- it didn't used to be this bad!), std.format (a big hairball I haven't dared to look too deeply into), and a couple of others, like various recursive templates elsewhere in Phobos. And also std.uni's large templated internal tables, which may be (part of?) the common cause of compile-time slowdowns in std.format and std.regex. There's also dmd's ridiculous memory usage policy, which is supposed to help compile times when you have ridiculous amounts of free RAM, but which causes anything from swap thrashing slowdowns to outright unusability on medium- to low-memory systems. T -- That's not a bug; that's a feature!
Dec 30 2018
next sibling parent reply Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy, which is 
 supposed to help compile times when you have ridiculous amounts 
 of free RAM, but which causes anything from swap thrashing 
 slowdowns to outright unusability on medium- to low-memory 
 systems.
Rainer an Martin Kinkelin have been working on that for LDC, they might upstream eventually.
Dec 30 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 30, 2018 at 01:58:56PM +0000, Nicholas Wilson via Digitalmars-d
wrote:
 On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy, which is supposed
 to help compile times when you have ridiculous amounts of free RAM,
 but which causes anything from swap thrashing slowdowns to outright
 unusability on medium- to low-memory systems.
Rainer an Martin Kinkelin have been working on that for LDC, they might upstream eventually.
Recently I noticed that LDC now compiles every function into their own section and runs LTO, including GC of unreferenced sections, by default. As a result, executable sizes are back down to where equivalent C/C++ code would be, as opposed to being a MB or so larger when compiled with DMD. It more-or-less nullifies most of the ill-effects of template bloat. Furthermore, LDC now tracks dmd releases very closely, almost on par, produces better code, has a far wider range of target archs, like Android/ARM, that I doubt DMD will ever support in the foreseeable future, and recently shows compile times pretty close to DMD (with things like std.regex or std.format making DMD dog-slow anyway, the extra time the LDC backend takes for codegen basically becomes roundoff error). Right now, I'm very tempted to drop dmd as my primary D compiler and use LDC instead. If this memory usage thing is fixed in LDC, I probably WILL drop dmd for good and use LDC instead. I've been itching to get D to compile on some of my low-memory systems but have been hampered until now -- it's been extremely frustrating. T -- "You are a very disagreeable person." "NO."
Dec 30 2018
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 12/30/2018 6:27 AM, H. S. Teoh wrote:
 Recently I noticed that LDC now compiles every function into their own
 section
DMD has always done that.
 and runs LTO, including GC of unreferenced sections, by default.
I've tried using the GC feature of ld, but it would produce executables that crashed. But that was years ago, perhaps ld has improved. If someone would like to turn that on with dmd, it would be a worthwhile experiment.
Dec 30 2018
next sibling parent Jacob Carlborg <doob me.com> writes:
On 2018-12-30 23:37, Walter Bright wrote:

 I've tried using the GC feature of ld, but it would produce executables 
 that crashed.
I think there might be some sections (perhaps the module info and similar) that need to be tagged (somehow) otherwise they will be removed. -- /Jacob Carlborg
Dec 31 2018
prev sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 30, 2018 at 02:37:59PM -0800, Walter Bright via Digitalmars-d wrote:
 On 12/30/2018 6:27 AM, H. S. Teoh wrote:
 Recently I noticed that LDC now compiles every function into their
 own section
DMD has always done that.
Then it must be doing something wrong, since running dmd with -L-gc-sections produces a 2 MB executable, but running ldc2 (without any special options) produces a 456 KB executable. Exactly the same set of source files. No dependencies on compiler-specific features in the code. To put this more in perspective, I re-tested this with a trivial Hello World program: import std.stdio; void main() { writeln("Hello world"); } Compile this with dmd: $ dmd -L--gc-sections test.d $ ls -l test -rwxrwxr-x 1 hsteoh hsteoh 967416 Dec 31 07:43 test Compile this with ldc2: $ ldc2 test.d hsteoh crystal:/tmp$ \ls -l test -rwxrwxr-x 1 hsteoh hsteoh 24632 Dec 31 07:44 test Note the order of magnitude difference in size, and that ldc2 achieves this by default, with no additional options needed. How do you make dmd produce the same (or comparable) output?
 and runs LTO, including GC of unreferenced sections, by default.
I've tried using the GC feature of ld, but it would produce executables that crashed. But that was years ago, perhaps ld has improved. If someone would like to turn that on with dmd, it would be a worthwhile experiment.
Maybe it's because certain required sections need to be marked in some way so that the linker doesn't discard them? T -- The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- Anonymous
Dec 31 2018
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/31/2018 7:51 AM, H. S. Teoh wrote:
 How do you make dmd produce the same (or comparable) output?
I don't know. To find out would take some time comparing the .o files.
Dec 31 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-12-31 16:51, H. S. Teoh wrote:

 Then it must be doing something wrong, since running dmd with
 -L-gc-sections produces a 2 MB executable, but running ldc2 (without any
 special options) produces a 456 KB executable. Exactly the same set of
 source files. No dependencies on compiler-specific features in the code.
 
 To put this more in perspective, I re-tested this with a trivial Hello
 World program:
 
 	import std.stdio;
 	void main() {
 		writeln("Hello world");
 	}
 
 Compile this with dmd:
 
 	$ dmd -L--gc-sections test.d
 	$ ls -l test
 	-rwxrwxr-x 1 hsteoh hsteoh 967416 Dec 31 07:43 test
 
 Compile this with ldc2:
 
 	$ ldc2 test.d
 	hsteoh crystal:/tmp$ \ls -l test
 	-rwxrwxr-x 1 hsteoh hsteoh 24632 Dec 31 07:44 test
 
 Note the order of magnitude difference in size, and that ldc2 achieves
 this by default, with no additional options needed.
 
 How do you make dmd produce the same (or comparable) output?
For me I get comparable output with DMD and LDC: LDC: 932 KB DMD (--gc-sections): 957 KB DMD: 988 KB This is running using Docker containers (which are running Ubuntu). Funny thing, on macOS it's the opposite of your experience: LDC: 5 MB LDC (-dead_strip): 957 KB DMD: 882 KB DMD (-dead_strip): 361 KB -- /Jacob Carlborg
Jan 01
parent reply kinke <noone nowhere.com> writes:
On Tuesday, 1 January 2019 at 09:44:40 UTC, Jacob Carlborg wrote:
 On 2018-12-31 16:51, H. S. Teoh wrote:
 Note the order of magnitude difference in size, and that ldc2 
 achieves
 this by default, with no additional options needed.
 
 How do you make dmd produce the same (or comparable) output?
For me I get comparable output with DMD and LDC
You guys are most likely comparing apples to oranges - H. S. using some distro-LDC preconfigured to link against shared druntime/Phobos, while LDC usually defaults to the static libs.
Jan 01
next sibling parent reply Rubn <where is.this> writes:
On Tuesday, 1 January 2019 at 11:57:26 UTC, kinke wrote:
 On Tuesday, 1 January 2019 at 09:44:40 UTC, Jacob Carlborg 
 wrote:
 On 2018-12-31 16:51, H. S. Teoh wrote:
 Note the order of magnitude difference in size, and that ldc2 
 achieves
 this by default, with no additional options needed.
 
 How do you make dmd produce the same (or comparable) output?
For me I get comparable output with DMD and LDC
You guys are most likely comparing apples to oranges - H. S. using some distro-LDC preconfigured to link against shared druntime/Phobos, while LDC usually defaults to the static libs.
Druntime and Phobos can be used as shared libraries? What is this magical feature? -Windows User
Jan 01
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 1, 2019 7:04:42 AM MST Rubn via Digitalmars-d wrote:
 On Tuesday, 1 January 2019 at 11:57:26 UTC, kinke wrote:
 On Tuesday, 1 January 2019 at 09:44:40 UTC, Jacob Carlborg

 wrote:
 On 2018-12-31 16:51, H. S. Teoh wrote:
 Note the order of magnitude difference in size, and that ldc2
 achieves
 this by default, with no additional options needed.

 How do you make dmd produce the same (or comparable) output?
For me I get comparable output with DMD and LDC
You guys are most likely comparing apples to oranges - H. S. using some distro-LDC preconfigured to link against shared druntime/Phobos, while LDC usually defaults to the static libs.
Druntime and Phobos can be used as shared libraries? What is this magical feature? -Windows User
LOL. Personally, it's a feature that I avoid like the plague, because I hate it when my programs stop working just because I updated dmd (which would be bad enough for a normal user, but it's particularly nasty when you're frequently following master and making local changes, because you're working on Phobos, druntime, and/or dmd). But yeah, when I saw the difference in size that H.S. Teoh was seeing, my first thought was that one was probably using Phobos as a shared library while the other was using it as a static one. - Jonathan M Davis
Jan 01
next sibling parent reply Rubn <where is.this> writes:
On Tuesday, 1 January 2019 at 19:28:36 UTC, Jonathan M Davis 
wrote:
 On Tuesday, January 1, 2019 7:04:42 AM MST Rubn via 
 Digitalmars-d wrote:
 On Tuesday, 1 January 2019 at 11:57:26 UTC, kinke wrote:
 On Tuesday, 1 January 2019 at 09:44:40 UTC, Jacob Carlborg

 wrote:
 On 2018-12-31 16:51, H. S. Teoh wrote:
 Note the order of magnitude difference in size, and that 
 ldc2
 achieves
 this by default, with no additional options needed.

 How do you make dmd produce the same (or comparable) 
 output?
For me I get comparable output with DMD and LDC
You guys are most likely comparing apples to oranges - H. S. using some distro-LDC preconfigured to link against shared druntime/Phobos, while LDC usually defaults to the static libs.
Druntime and Phobos can be used as shared libraries? What is this magical feature? -Windows User
LOL. Personally, it's a feature that I avoid like the plague, because I hate it when my programs stop working just because I updated dmd (which would be bad enough for a normal user, but it's particularly nasty when you're frequently following master and making local changes, because you're working on Phobos, druntime, and/or dmd). But yeah, when I saw the difference in size that H.S. Teoh was seeing, my first thought was that one was probably using Phobos as a shared library while the other was using it as a static one. - Jonathan M Davis
On linux the version should be part of the filename, and iirc it'll only try the shared library without a version if it can't find the one it needs. Not too familiar with how it is done on Windows as it isn't really standardized. Could just look at how it is done for the C++ runtime, as I usually have 20 different versions of it installed even for the same MSVC year revision. I just stick the dll in the same folder as the executable. I'd rather have a single shared library than having 20 instances of it loaded in my 20 dll files. I can dream though, that one day this fundamental feature will be added to D.
Jan 01
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 1, 2019 1:59:25 PM MST Rubn via Digitalmars-d wrote:
 On Tuesday, 1 January 2019 at 19:28:36 UTC, Jonathan M Davis
 wrote:
 On Tuesday, January 1, 2019 7:04:42 AM MST Rubn via
 Druntime and Phobos can be used as shared libraries? What is
 this magical feature?

 -Windows User
LOL. Personally, it's a feature that I avoid like the plague, because I hate it when my programs stop working just because I updated dmd (which would be bad enough for a normal user, but it's particularly nasty when you're frequently following master and making local changes, because you're working on Phobos, druntime, and/or dmd). But yeah, when I saw the difference in size that H.S. Teoh was seeing, my first thought was that one was probably using Phobos as a shared library while the other was using it as a static one. - Jonathan M Davis
On linux the version should be part of the filename, and iirc it'll only try the shared library without a version if it can't find the one it needs. Not too familiar with how it is done on Windows as it isn't really standardized. Could just look at how it is done for the C++ runtime, as I usually have 20 different versions of it installed even for the same MSVC year revision. I just stick the dll in the same folder as the executable. I'd rather have a single shared library than having 20 instances of it loaded in my 20 dll files. I can dream though, that one day this fundamental feature will be added to D.
The version number is in the file name for Phobos' shared library, but if you're building the development version, that really doesn't help you, because every time you rebuild it, you get a file with the same name but potentially different contents (at least until the next release gets tagged, and the version number gets bumped). If you're only using releases, you don't have that problem. However, unless you keep older versions of the shared library around, as soon as you update, all of your D programs break, because the version they need isn't there anymore. All in all, it's just way simpler to use Phobos as a static library. Sure, when you have a bunch of D programs statically linked, it takes up a few extra megabytes that way, but when systems have terabytes, that doesn't really matter. Anyone who wants to use the shared library version is free to do so, but I really don't think that it makes sense in most cases. I do agree that shared library support in general is something that we should have, but that doesn't mean that it really makes sense as the default for the standard library - especially when so much of it is templated anyway. I really have no clue what the situation is with dll support, because I pretty much only program on Windows when I'm forced to, but given how dlls work on Windows, I've never been a fan of using them except in cases where you have to. The whole nonsense where you have to rebuild your program, because _anything_ changed in the dll is just ridiculous IMHO. At least on *nix systems, shared libraries can be changed so long as the ABIs of the existing symbols aren't changed, making it so that you can actually update shared libraries without rebuilding everything. We just get screwed with D, because the way it's designed makes it very hard to maintain ABI compatibility if templates are involved, and no attempt is made to maintain ABI compatibility across versions of Phobos. So, for Phobos, using a shared library can be very problematic (whereas a shared library with minimal templates that was designed to maintain a fixed API and ABI would work just fine, just like it does with C). - Jonathan M Davis
Jan 01
parent reply Rubn <where is.this> writes:
On Tuesday, 1 January 2019 at 22:34:24 UTC, Jonathan M Davis 
wrote:
 The version number is in the file name for Phobos' shared 
 library, but if you're building the development version, that 
 really doesn't help you, because every time you rebuild it, you 
 get a file with the same name but potentially different 
 contents (at least until the next release gets tagged, and the 
 version number gets bumped). If you're only using releases, you 
 don't have that problem. However, unless you keep older 
 versions of the shared library around, as soon as you update, 
 all of your D programs break, because the version they need 
 isn't there anymore. All in all, it's just way simpler to use 
 Phobos as a static library. Sure, when you have a bunch of D 
 programs statically linked, it takes up a few extra megabytes 
 that way, but when systems have terabytes, that doesn't really 
 matter. Anyone who wants to use the shared library version is 
 free to do so, but I really don't think that it makes sense in 
 most cases. I do agree that shared library support in general 
 is something that we should have, but that doesn't mean that it 
 really makes sense as the default for the standard library - 
 especially when so much of it is templated anyway.
Why wouldn't you keep older versions of phobos? Why are you deleting the old ones when you install a new version of DMD. They are designed in such a way to keep them there for this very reason. I don't really understand your argument here, to me it just seems DMD is just doing something backwards in comparison to the rest of the platform (as usual). But we aren't, there's no shared library of phobos for Windows and the looks of it Mac OS. It's not about size, it's about easy of use. The way the garbage collector works it might not be as bad in comparison with the C runtime. Where allocating memory from a shared library then unloading it would effectively free all the memory that was allocated by that shared library. But at the same time since allocating memory from the GC
 The whole nonsense where you have to rebuild your program, 
 because _anything_ changed in the dll is just ridiculous IMHO.
What?! Where did you hear this non-sense from? I'm not surprised at the state of shared library support on windows anymore. Optlink and DMC are the greatest compilers and linkers on Windows, they don't cause ANY problems AT ALL, they are very well supported and shouldn't be removed. Sigh.
Jan 01
next sibling parent Neia Neutuladh <neia ikeran.org> writes:
On Wed, 02 Jan 2019 00:40:21 +0000, Rubn wrote:
 Why wouldn't you keep older versions of phobos? Why are you deleting the
 old ones when you install a new version of DMD.
Distributing binaries requires either static linking, shipping multiple files that must be kept together (which is strictly inferior to static linking), or a compatible library already installed on the client machine. Phobos doesn't have a stable ABI, so that means the client would need your specific version of Phobos. That's not super kind. You can't tell which versions of libphobos.so you need to keep around without running ldd on every executable on your computer. This would be moderately annoying with only official releases; for instance, I've got one tiny project that's old enough that I built it using dsss, and either that would be broken, or I'd have a few dozen versions of phobos running around. More than one per binary that I care about. Specific to Jonathan M Davis, if you're building dmd/druntime/phobos from git head, the previous problems are hugely compounded, plus you need to assign unique numbers to each built version and update the relevant dmd.conf to match.
Jan 01
prev sibling next sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 1, 2019 5:40:21 PM MST Rubn via Digitalmars-d wrote:
 On Tuesday, 1 January 2019 at 22:34:24 UTC, Jonathan M Davis

 wrote:
 The version number is in the file name for Phobos' shared
 library, but if you're building the development version, that
 really doesn't help you, because every time you rebuild it, you
 get a file with the same name but potentially different
 contents (at least until the next release gets tagged, and the
 version number gets bumped). If you're only using releases, you
 don't have that problem. However, unless you keep older
 versions of the shared library around, as soon as you update,
 all of your D programs break, because the version they need
 isn't there anymore. All in all, it's just way simpler to use
 Phobos as a static library. Sure, when you have a bunch of D
 programs statically linked, it takes up a few extra megabytes
 that way, but when systems have terabytes, that doesn't really
 matter. Anyone who wants to use the shared library version is
 free to do so, but I really don't think that it makes sense in
 most cases. I do agree that shared library support in general
 is something that we should have, but that doesn't mean that it
 really makes sense as the default for the standard library -
 especially when so much of it is templated anyway.
Why wouldn't you keep older versions of phobos? Why are you deleting the old ones when you install a new version of DMD. They are designed in such a way to keep them there for this very reason. I don't really understand your argument here, to me it just seems DMD is just doing something backwards in comparison to the rest of the platform (as usual).
Anyone using a package manager to install dmd is only ever going to end up with the version of Phobos that goes with that dmd. It would be highly abnormal for multiple versions of Phobos to be installed on a system. The only folks who would ever end up with multiple versions of Phobos on the same system are folks actively going to the effort of making it happen instead of using any of the normal install mechanisms. And ultimately, it's far simpler to just always use Phobos as a static library rather than trying to do anything with trying to keep older versions of the shared library around - especially if you're actually building development versions of Phobos. In general, the only downsides to statically linking are that the executables are slightly larger, and you don't get the benefit of security updates to libraries without rebuilding the binary. And with how easy it is to break ABI compatibility with D libraries (something as simple as a function attribute changing breaks it, and that's trivial if the code is inferring attributes), updating D shared libraries in a way that doesn't require rebuilding the program is quite problematic in general. In some situations, it's worth it, but in general, it really isn't. The place where shared libraries offer far more benefit is with plugins and the like which are loaded while the program is running rather than in trying to actually share code. The ABI problems still exist there, but plugin APIs are generally far more limited than full-on libraries (which reduces the problem), and you simply can't use static libraries in those situations, whereas if you're just linking your program against a library, static libraries work just fine. I'm by no means against having shared libraries in general work, and I think that full dll support should exist for D on Windows, but aside from plugins, in the vast majority of cases, I think that using shared libraries is far more trouble than it's worth (especially with how difficult it is to maintain ABI compatibility with D libraries).
 The whole nonsense where you have to rebuild your program,
 because _anything_ changed in the dll is just ridiculous IMHO.
What?! Where did you hear this non-sense from? I'm not surprised at the state of shared library support on windows anymore.
From working with dlls with C++. With dlls on Windows, your program links
against a static library associated with the dynamic library, and if any of the symbols are changed, the addresses change, and your program will be unable to load the newer version of the library without being rebuilt against the new version of the static library. This is in stark contrast to *nix where the linking works in such a way that as long as the symbols still exist with the same ABI in the newly built library, they're found when the program loads, and it's not a problem. The addresses aren't hard-coded in the way that happens with dlls on Windows. dlls on Windows allow you to share code so long as the programs are all built against exactly the same version of the dll (and if they're not, then you need separate copies of the dll, and you get into dll hell), whereas with *nix, you can keep updating the shared library as much as you like without changing the executable as long as the API and ABI of the existing symbols don't change. - Jonathan M Davis
Jan 01
next sibling parent reply Rubn <where is.this> writes:
On Wednesday, 2 January 2019 at 02:04:24 UTC, Jonathan M Davis 
wrote:
 I'm by no means against having shared libraries in general 
 work, and I think that full dll support should exist for D on 
 Windows, but aside from plugins, in the vast majority of cases, 
 I think that using shared libraries is far more trouble than 
 it's worth (especially with how difficult it is to maintain ABI 
 compatibility with D libraries).
You've obviously haven't used shared libraries that much then -- that is shared libraries that link statically to the runtime. Having multiple instances of Phobos/druntime loaded at the same time in one process has it's own can of warms. I'm not surprised at all that in general people don't even use shared libraries.
 The whole nonsense where you have to rebuild your program, 
 because _anything_ changed in the dll is just ridiculous 
 IMHO.
What?! Where did you hear this non-sense from? I'm not surprised at the state of shared library support on windows anymore.
From working with dlls with C++. With dlls on Windows, your 
program links
against a static library associated with the dynamic library, and if any of the symbols are changed, the addresses change, and your program will be unable to load the newer version of the library without being rebuilt against the new version of the static library. This is in stark contrast to *nix where the linking works in such a way that as long as the symbols still exist with the same ABI in the newly built library, they're found when the program loads, and it's not a problem. The addresses aren't hard-coded in the way that happens with dlls on Windows. dlls on Windows allow you to share code so long as the programs are all built against exactly the same version of the dll (and if they're not, then you need separate copies of the dll, and you get into dll hell), whereas with *nix, you can keep updating the shared library as much as you like without changing the executable as long as the API and ABI of the existing symbols don't change. - Jonathan M Davis
On Wednesday, 2 January 2019 at 04:09:29 UTC, H. S. Teoh wrote:
 On Tue, Jan 01, 2019 at 07:04:24PM -0700, Jonathan M Davis via 
 Digitalmars-d wrote: [...]
 From working with dlls with C++. With dlls on Windows, your 
 program links against a static library associated with the 
 dynamic library, and if any of the symbols are changed, the 
 addresses change, and your program will be unable to load the 
 newer version of the library without being rebuilt against the 
 new version of the static library.
Wow. That makes me glad I'm not programming on Windows...
Mother of god, these two comments. Rip D on Windows, there's clearly only one competent Windows user on the D development team. It's too bad most of his work is focused on maintaining the VS plugin but from the looks of it if he didn't no one would. Everyone else is obliviously tucking incompetent. If it worked the way you think it does (it doesn't) then every Windows update would literally break EVERY SINGLE executable file on the face of the earth. They would all need to be recompiled as the system DLL libraries are updated. You can only link to the system libraries dynamically virtually and every executable uses them. Yet somehow Windows is able to maintain backwards compatibility with old executable files better than Linux does. The more worrying thing about this is even though you said it yourself you barely use Windows yourself, rather than spending the 5 mins it would take for you to google search this yourself, you continue to go on with your misinformation that might have been true with Windows DOS 40 years ago. God damn.
Jan 02
parent 12345swordy <alexanderheistermann gmail.com> writes:
On Wednesday, 2 January 2019 at 21:04:25 UTC, Rubn wrote:
 Mother of god, these two comments. Rip D on Windows, there's 
 clearly only one competent Windows user on the D development 
 team. It's too bad most of his work is focused on maintaining 
 the VS plugin but from the looks of it if he didn't no one 
 would.
Some of us would. (Manu most noticeably) VS is simply that good. Alex
Jan 02
prev sibling parent reply rjframe <dlang ryanjframe.com> writes:
On Tue, 01 Jan 2019 19:04:24 -0700, Jonathan M Davis wrote:

 From working with dlls with C++. With dlls on Windows, your program
 links
 against a static library associated with the dynamic library, and if any
 of the symbols are changed, the addresses change, and your program will
 be unable to load the newer version of the library without being rebuilt
 against the new version of the static library.
That's not necessarily true; Windows supports "implicit linking" and "explicit linking"; for implicit linking you do need to statically link against an import library, but for explicit linking you don't even need to know the DLL's name until runtime. With explicit linking you load the library by calling LoadLibrary/ LoadLibraryEx, then call GetProcAddress with the name of your desired function to get the function pointer. If you watch the filesystem for the DLL to change, you could live-update by reloading the DLL (which you typically wouldn't do outside debugging or maybe if offering plugin support). Most people just do implicit linking because it's less work. Any DLL can be loaded in both ways, though if there's a DllMain there may be problems if the library author doesn't support both methods; for implicit linking, DllMain is run before the program entry point, but for explicit linking its called by LoadLibrary in the context of the thread that calls it. --Ryan
Jan 03
next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, January 3, 2019 4:52:56 AM MST rjframe via Digitalmars-d wrote:
 On Tue, 01 Jan 2019 19:04:24 -0700, Jonathan M Davis wrote:
 From working with dlls with C++. With dlls on Windows, your program
 links
 against a static library associated with the dynamic library, and if any
 of the symbols are changed, the addresses change, and your program will
 be unable to load the newer version of the library without being rebuilt
 against the new version of the static library.
That's not necessarily true; Windows supports "implicit linking" and "explicit linking"; for implicit linking you do need to statically link against an import library, but for explicit linking you don't even need to know the DLL's name until runtime. With explicit linking you load the library by calling LoadLibrary/ LoadLibraryEx, then call GetProcAddress with the name of your desired function to get the function pointer. If you watch the filesystem for the DLL to change, you could live-update by reloading the DLL (which you typically wouldn't do outside debugging or maybe if offering plugin support). Most people just do implicit linking because it's less work. Any DLL can be loaded in both ways, though if there's a DllMain there may be problems if the library author doesn't support both methods; for implicit linking, DllMain is run before the program entry point, but for explicit linking its called by LoadLibrary in the context of the thread that calls it.
*nix has the same distinction. It's a fundamentally different situation from linking your executable against the library. You're really dynamically loading rather than dynamically linking (though unfortunately, the terminology for the two is not particularly distinct, and they're often referred to the same way even though they're completely different). Loading libraries that way is what you do when you do stuff like plugins, because those aren't known when you build your program. But it makes a lot less sense as an alternative to linking your program against the library if you don't actually need to load the library like that. The COFF vs OMF mess on Windows makes it make slightly more sense on Windows (at least with D, where dmd uses OMF by default, unlike most of the C/C++ world at this point), because then it doesn't matter whether COFF or OMF was used (e.g. IIRC, Derelict is designed to be loaded that way for that reason), but in general, it's an unnecessarily complicated way to use a library. And if Windows' eccentricities make it more desirable than it is on *nix systems, then that's just yet another black mark against how Windows does dynamic linking IMHO. - Jonathan M Davis
Jan 03
prev sibling parent Manu <turkeyman gmail.com> writes:
On Thu, Jan 3, 2019 at 5:50 AM Jonathan M Davis via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Thursday, January 3, 2019 4:52:56 AM MST rjframe via Digitalmars-d wrote:
 On Tue, 01 Jan 2019 19:04:24 -0700, Jonathan M Davis wrote:
 From working with dlls with C++. With dlls on Windows, your program
 links
 against a static library associated with the dynamic library, and if any
 of the symbols are changed, the addresses change, and your program will
 be unable to load the newer version of the library without being rebuilt
 against the new version of the static library.
That's not necessarily true; Windows supports "implicit linking" and "explicit linking"; for implicit linking you do need to statically link against an import library, but for explicit linking you don't even need to know the DLL's name until runtime. With explicit linking you load the library by calling LoadLibrary/ LoadLibraryEx, then call GetProcAddress with the name of your desired function to get the function pointer. If you watch the filesystem for the DLL to change, you could live-update by reloading the DLL (which you typically wouldn't do outside debugging or maybe if offering plugin support). Most people just do implicit linking because it's less work. Any DLL can be loaded in both ways, though if there's a DllMain there may be problems if the library author doesn't support both methods; for implicit linking, DllMain is run before the program entry point, but for explicit linking its called by LoadLibrary in the context of the thread that calls it.
*nix has the same distinction. It's a fundamentally different situation from linking your executable against the library. You're really dynamically loading rather than dynamically linking (though unfortunately, the terminology for the two is not particularly distinct, and they're often referred to the same way even though they're completely different). Loading libraries that way is what you do when you do stuff like plugins, because those aren't known when you build your program. But it makes a lot less sense as an alternative to linking your program against the library if you don't actually need to load the library like that. The COFF vs OMF mess on Windows makes it make slightly more sense on Windows (at least with D, where dmd uses OMF by default, unlike most of the C/C++ world at this point), because then it doesn't matter whether COFF or OMF was used (e.g. IIRC, Derelict is designed to be loaded that way for that reason), but in general, it's an unnecessarily complicated way to use a library. And if Windows' eccentricities make it more desirable than it is on *nix systems, then that's just yet another black mark against how Windows does dynamic linking IMHO.
Sorry, I don't think you know what you're talking about WRT Windows DLL's and import libs. Linking a Windows import lib is the same as `-lSharedLib.so`; it links(/generates) a small stub at entry that loads the DLL, and resolves the symbols in the import table to local function pointers. You certainly do NOT need to rebuild your exe if the DLL is updated, assuming no breaking changes to the ABI. The import lib includes little stub's for the import functions that call through the resolved pointer into the DLL. It's nothing more than a convenience, and it's also possible to *generate* an import lib from a .dll, which is effectively identical to linking against a .so.
Jan 03
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 01, 2019 at 07:04:24PM -0700, Jonathan M Davis via Digitalmars-d
wrote:
[...]
 Anyone using a package manager to install dmd is only ever going to
 end up with the version of Phobos that goes with that dmd. It would be
 highly abnormal for multiple versions of Phobos to be installed on a
 system.
Wait, what? Where did you get that idea from? Since about a half decade ago, Linux distros like Debian have had the possibility of multiple versions of the same shared library installed at the same time. It's pretty much impossible to manage a distro otherwise, since being unable to do so would mean that you cannot upgrade a shared library until ALL upstream code has been updated to use the new version. Now granted, most of the time new library versions are ABI compatible with older versions, so you don't actually need to keep *every* version of the library around just because some executable somewhere needs them. And granted, splitting a library package into multiple simultaneous versions is only done when necessary. But the mechanisms for doing so have been in place since a long time ago, and any decent distro management would include setting up the proper mechanisms for multiple versions of Phobos to be installable simultaneously. Otherwise you have the untenable situation that Phobos cannot be upgraded without breaking every D program currently installed on the system. Of course, for this to work, the soname needs to be set properly and a sane versioning system (encoded in the soname) needs to be in place. Basically, *every* ABI incompatibility (and I do mean *every*, even those with no equivalent change in the source code) needs to be reflected by a soname change. Which is likely not being done with the current makefiles in git. Which would explain your observations. But it is certainly *possible*, and often *necessary*, to install multiple versions of the same shared library simultaneously.
 The only folks who would ever end up with multiple versions of Phobos
 on the same system are folks actively going to the effort of making it
 happen instead of using any of the normal install mechanisms.
I have no idea what you mean by "normal install mechanisms", because it makes no sense to me to use any system-wide installation mechanism that *doesn't* support multiple versions per shared library. I mean, how do you even get a sane operating environment at all? It would be DLL hell compounded with .so hell, episode II. OTOH, if you're installing stuff by hand or by 3rd party installers (which is generally a bad idea on Linux distros with a distro-specific packaging system -- since colliding assumptions made by either side means endless headaches on the user end), then you're on your own. Maybe that's where you're coming from.
 And ultimately, it's far simpler to just always use Phobos as a static
 library rather than trying to do anything with trying to keep older
 versions of the shared library around - especially if you're actually
 building development versions of Phobos.
Yes, for development builds, I'd tend to agree. [...]
 From working with dlls with C++. With dlls on Windows, your program
 links against a static library associated with the dynamic library,
 and if any of the symbols are changed, the addresses change, and your
 program will be unable to load the newer version of the library
 without being rebuilt against the new version of the static library.
Wow. That makes me glad I'm not programming on Windows...
 This is in stark contrast to *nix where the linking works in such a
 way that as long as the symbols still exist with the same ABI in the
 newly built library, they're found when the program loads, and it's
 not a problem. The addresses aren't hard-coded in the way that happens
 with dlls on Windows. dlls on Windows allow you to share code so long
 as the programs are all built against exactly the same version of the
 dll (and if they're not, then you need separate copies of the dll, and
 you get into dll hell), whereas with *nix, you can keep updating the
 shared library as much as you like without changing the executable as
 long as the API and ABI of the existing symbols don't change.
[...] Yes, though the library authors / distributors will have to be extremely careful to bump the soname for every ABI incompatibility. All too often people forget to do so, esp. when the ABI incompatibility is not something directly caused by a source code change (the fallacious assumption being "I didn't touch the source code, the ABI can't have changed, right?"). And more often than people would like to admit, sonames fail to get bumped even in the face of source code change, and that's where things start getting ugly. T -- Государство делает вид, что платит нам зарплату, а мы делаем вид, что работаем.
Jan 01
prev sibling parent Rubn <where is.this> writes:
Wouldn't be the least bit surprised if DMD didn't follow Linux 
conventions though.
Jan 01
prev sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Jan 01, 2019 at 11:57:26AM +0000, kinke via Digitalmars-d wrote:
 On Tuesday, 1 January 2019 at 09:44:40 UTC, Jacob Carlborg wrote:
 On 2018-12-31 16:51, H. S. Teoh wrote:
 Note the order of magnitude difference in size, and that ldc2
 achieves this by default, with no additional options needed.
 
 How do you make dmd produce the same (or comparable) output?
For me I get comparable output with DMD and LDC
You guys are most likely comparing apples to oranges - H. S. using some distro-LDC preconfigured to link against shared druntime/Phobos, while LDC usually defaults to the static libs.
Aha! I think you hit the nail on the head. `ldd` revealed that the LDC binary has dynamic dependencies on libphobos2-ldc-shared.so and libdruntime-ldc-shared.so. So that explains that. So that's one less reason to ditch dmd... but still, I'm tempted, because of overall better codegen with ldc, esp. with compute-intensive code. T -- What are you when you run out of Monet? Baroque.
Jan 01
prev sibling parent Jon Degenhardt <jond noreply.com> writes:
On Sunday, 30 December 2018 at 14:27:56 UTC, H. S. Teoh wrote:
 Recently I noticed that LDC now compiles every function into 
 their own section and runs LTO, including GC of unreferenced 
 sections, by default. As a result, executable sizes are back 
 down to where equivalent C/C++ code would be, as opposed to 
 being a MB or so larger when compiled with DMD. It more-or-less 
 nullifies most of the ill-effects of template bloat.
At DConf I showed executable size reductions on my apps resulting from applying LDC's LTO. They are on slide 14 here: https://github.com/eBay/tsv-utils/blob/master/docs/dconf2018.pdf. I didn't have a basis for comparison to equivalent C/C++ though. --Jon
Dec 30 2018
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 D theory sounds all good and all, but in practice you have 
 warts like dub (one big reason I stay away from it -- though 
 based on what Sonke said recently, performance may have 
 improved since I last checked), std.regex (after the last big 
 refactor, something Really Bad happened to its compile times -- 
 it didn't used to be this bad!), std.format (a big hairball I 
 haven't dared to look too deeply into), and a couple of others, 
 like various recursive templates elsewhere in Phobos. And also 
 std.uni's large templated internal tables, which may be (part 
 of?) the common cause of compile-time slowdowns in std.format 
 and std.regex.

 There's also dmd's ridiculous memory usage policy, which is 
 supposed to help compile times when you have ridiculous amounts 
 of free RAM, but which causes anything from swap thrashing 
 slowdowns to outright unusability on medium- to low-memory 
 systems.


 T
Phobos has a few modules like this, but I believe that all of them should be fixable without help from the compiler, given enough effort. On the other hand, hopefully soon we'll have the option to turn on the GC for the frontend. See: https://github.com/ldc-developers/ldc/pull/2916 As for Dub, we really ought to add a lower-level dependency graph interface for describing non-trivial builds. There are already a couple of (meta-)build systems written in D, so we have to come up with a good design and integrate one of them.
Dec 30 2018
parent welkam <wwwelkam gmail.com> writes:
On Sunday, 30 December 2018 at 13:59:57 UTC, Petar Kirov 
[ZombineDev] wrote:
 On the other hand, hopefully soon we'll have the option to turn 
 on the GC for the frontend. See: 
 https://github.com/ldc-developers/ldc/pull/2916
GC might not be needed here. Compiler compiles your program in steps and memory that were used in previous step could be deallocated later in one place. We could just use a couple different allocators for different parts of compiler and call free on them when we are sure that nothing will need that data anymore. For example we can deallocate file buffers after we parsed those files. A person more familiar with compiler could find more cases where this kind of strategy is applicable.
Dec 30 2018
prev sibling next sibling parent Laeeth Isharc <laeeth laeeth.com> writes:
On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 On Sun, Dec 30, 2018 at 01:25:33PM +0000, Guillaume Piolat via 
 Digitalmars-d wrote:
 On Saturday, 29 December 2018 at 09:29:30 UTC, Walter Bright 
 wrote:
 http://aras-p.info/blog/2018/12/28/Modern-C-Lamentations/
It says:
 C++ compilation times have been a source of pain in every
 non-trivial-size codebase I’ve worked on. Don’t believe me? 
 Try
 building one of the widely available big codebases (any of:
 Chromium, Clang/LLVM, UE4 etc will do). Among the things I 
 really
 want out of C++, “solve compile times” is probably #1 on the 
 list,
 and has been since forever.
I agree, it has been #1 problem with C++ in years, and only getting worse. There is the theory (D builds fast) and the application (DUB often negate the advantage, you need to avoid templatitis).
D theory sounds all good and all, but in practice you have warts like dub (one big reason I stay away from it -- though based on what Sonke said recently, performance may have improved since I last checked), std.regex (after the last big refactor, something Really Bad happened to its compile times -- it didn't used to be this bad!), std.format (a big hairball I haven't dared to look too deeply into), and a couple of others, like various recursive templates elsewhere in Phobos. And also std.uni's large templated internal tables, which may be (part of?) the common cause of compile-time slowdowns in std.format and std.regex. T
Perhaps we should implement CyberShadow's idea into the build infrastructure. Works quite nicely with etsy statsd - see library from Burner. https://blog.thecybershadow.net/2015/05/05/is-d-slim-yet/ Laeeth
Dec 30 2018
prev sibling next sibling parent reply Guillaume Piolat <first.last gmail.com> writes:
On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy, which is 
 supposed to help compile times when you have ridiculous amounts 
 of free RAM, but which causes anything from swap thrashing 
 slowdowns to outright unusability on medium- to low-memory 
 systems.
Don't get me started :) My VPS host has 512 mb RAM, and it's stressful when a D compiler can't build a few files. Upping this memory limit costs significantly more money.
Dec 30 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Sun, Dec 30, 2018 at 03:10:17PM +0000, Guillaume Piolat via Digitalmars-d
wrote:
 On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy, which is supposed
 to help compile times when you have ridiculous amounts of free RAM,
 but which causes anything from swap thrashing slowdowns to outright
 unusability on medium- to low-memory systems.
 
Don't get me started :) My VPS host has 512 mb RAM, and it's stressful when a D compiler can't build a few files. Upping this memory limit costs significantly more money.
Exactly the same problem I've been facing. Not to mention some workstations at my day job that don't have the luxury of having gobs of free RAM available. Which is why I haven't dared to mention D at work yet... it would become the laughing stock of my coworkers if dmd were to die with an OOM error on the simplest of programs. T -- What's an anagram of "BANACH-TARSKI"? BANACH-TARSKI BANACH-TARSKI.
Dec 30 2018
prev sibling parent Mengu <mengukagan gmail.com> writes:
On Sunday, 30 December 2018 at 15:10:17 UTC, Guillaume Piolat 
wrote:
 On Sunday, 30 December 2018 at 13:46:46 UTC, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy, which is 
 supposed to help compile times when you have ridiculous 
 amounts of free RAM, but which causes anything from swap 
 thrashing slowdowns to outright unusability on medium- to 
 low-memory systems.
Don't get me started :) My VPS host has 512 mb RAM, and it's stressful when a D compiler can't build a few files. Upping this memory limit costs significantly more money.
i was shocked when i encountered the exact same thing on an ec2 micro instance. it was a bit upsetting and disappointing.
Dec 30 2018
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 12/30/2018 5:46 AM, H. S. Teoh wrote:
 There's also dmd's ridiculous memory usage policy,
Now that D is written in D, one can compile it with: -profile=gc and see where the memory usage is coming from.
Dec 30 2018
prev sibling next sibling parent Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Tuesday, January 1, 2019 9:09:29 PM MST H. S. Teoh via Digitalmars-d 
wrote:
 On Tue, Jan 01, 2019 at 07:04:24PM -0700, Jonathan M Davis via
 Digitalmars-d wrote: [...]

 Anyone using a package manager to install dmd is only ever going to
 end up with the version of Phobos that goes with that dmd. It would be
 highly abnormal for multiple versions of Phobos to be installed on a
 system.
Wait, what? Where did you get that idea from? Since about a half decade ago, Linux distros like Debian have had the possibility of multiple versions of the same shared library installed at the same time. It's pretty much impossible to manage a distro otherwise, since being unable to do so would mean that you cannot upgrade a shared library until ALL upstream code has been updated to use the new version. Now granted, most of the time new library versions are ABI compatible with older versions, so you don't actually need to keep *every* version of the library around just because some executable somewhere needs them. And granted, splitting a library package into multiple simultaneous versions is only done when necessary. But the mechanisms for doing so have been in place since a long time ago, and any decent distro management would include setting up the proper mechanisms for multiple versions of Phobos to be installable simultaneously. Otherwise you have the untenable situation that Phobos cannot be upgraded without breaking every D program currently installed on the system. Of course, for this to work, the soname needs to be set properly and a sane versioning system (encoded in the soname) needs to be in place. Basically, *every* ABI incompatibility (and I do mean *every*, even those with no equivalent change in the source code) needs to be reflected by a soname change. Which is likely not being done with the current makefiles in git. Which would explain your observations. But it is certainly *possible*, and often *necessary*, to install multiple versions of the same shared library simultaneously.
Sure, you could have separate packages for separate versions of Phobos, but from what I've seen, there's always one version of dmd with one version of Phobos, and distros don't provide multiple versions. Now, I haven't studied every distro there is, so maybe there is a distro out there that provides separate packages for old versions of Phobos, but in my experience, you're lucky if the distro has any package for dmd, let alone for it to be trying to provide a way to have multiple versions of Phobos installed. And while yes, distros are set up in a way that you can have multiple versions of a library if multiple packages exist for them, and sometimes they do that, in the vast majority of cases, the solution is that all packages for the distro are built for a specific version of a library, and when that library is upgraded, all the packages that depend on it get rebuilt and need to be reinstalled. That's part of why it usually works so poorly to distribute closed source programs for Linux. Distros tend to be put together with the idea that all of the packages on the system are built for that system with whatever version of the libraries they're currently using, and having multiple versions of a library is the exception rather than the rule. But regardless of what is typically done with libraries for stuff like C++ or python, unless distros are specifically providing packages for older versions of Phobos, then no one installing dmd and Phobos via a package manager is going to end up with multiple versions of Phobos installed. So, while it may be theoretically possible to have multiple versions of Phobos installed via a package manager, from what I've seen, that simply doesn't happen in practice, because the packages for dmd and Phobos aren't set up that way. - Jonathan M Davis
Jan 01
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, January 3, 2019 12:19:59 PM MST Manu via Digitalmars-d wrote:
 On Thu, Jan 3, 2019 at 5:50 AM Jonathan M Davis via Digitalmars-d

 <digitalmars-d puremagic.com> wrote:
 On Thursday, January 3, 2019 4:52:56 AM MST rjframe via Digitalmars-d 
wrote:
 On Tue, 01 Jan 2019 19:04:24 -0700, Jonathan M Davis wrote:
 From working with dlls with C++. With dlls on Windows, your program
 links
 against a static library associated with the dynamic library, and if
 any
 of the symbols are changed, the addresses change, and your program
 will
 be unable to load the newer version of the library without being
 rebuilt
 against the new version of the static library.
That's not necessarily true; Windows supports "implicit linking" and "explicit linking"; for implicit linking you do need to statically link against an import library, but for explicit linking you don't even need to know the DLL's name until runtime. With explicit linking you load the library by calling LoadLibrary/ LoadLibraryEx, then call GetProcAddress with the name of your desired function to get the function pointer. If you watch the filesystem for the DLL to change, you could live-update by reloading the DLL (which you typically wouldn't do outside debugging or maybe if offering plugin support). Most people just do implicit linking because it's less work. Any DLL can be loaded in both ways, though if there's a DllMain there may be problems if the library author doesn't support both methods; for implicit linking, DllMain is run before the program entry point, but for explicit linking its called by LoadLibrary in the context of the thread that calls it.
*nix has the same distinction. It's a fundamentally different situation from linking your executable against the library. You're really dynamically loading rather than dynamically linking (though unfortunately, the terminology for the two is not particularly distinct, and they're often referred to the same way even though they're completely different). Loading libraries that way is what you do when you do stuff like plugins, because those aren't known when you build your program. But it makes a lot less sense as an alternative to linking your program against the library if you don't actually need to load the library like that. The COFF vs OMF mess on Windows makes it make slightly more sense on Windows (at least with D, where dmd uses OMF by default, unlike most of the C/C++ world at this point), because then it doesn't matter whether COFF or OMF was used (e.g. IIRC, Derelict is designed to be loaded that way for that reason), but in general, it's an unnecessarily complicated way to use a library. And if Windows' eccentricities make it more desirable than it is on *nix systems, then that's just yet another black mark against how Windows does dynamic linking IMHO.
Sorry, I don't think you know what you're talking about WRT Windows DLL's and import libs. Linking a Windows import lib is the same as `-lSharedLib.so`; it links(/generates) a small stub at entry that loads the DLL, and resolves the symbols in the import table to local function pointers. You certainly do NOT need to rebuild your exe if the DLL is updated, assuming no breaking changes to the ABI. The import lib includes little stub's for the import functions that call through the resolved pointer into the DLL. It's nothing more than a convenience, and it's also possible to *generate* an import lib from a .dll, which is effectively identical to linking against a .so.
From the last time I worked with Windows dlls, I remember quite distinctly
that doing anything like adding a symbol to the library meant that it was incompatible with executables previously built with it (which is not true for shared libraries on *nix - they only break if the ABI for the existing symbols changes). So, if that's not the case, I don't know what we were doing wrong, but I have an extremely sour taste in my mouth from dealing with Windows dlls. It was my experience that Linux stuff generally just worked, whereas we kept having to deal with junk on Windows to make them work (e.g. adding special attributes to functions just so that they would be exported), and I absolutely hated it. I have nothing good to say about dlls on Windows. It's quite possible that some of it isn't as bad if you know more about it than the team I was working on did, but it was one part of why making our libraries cross-platform was not at all fun. - Jonathan M Davis
Jan 03
next sibling parent Rubn <where is.this> writes:
On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis 
wrote:
 *nix has the same distinction. It's a fundamentally different 
 situation from linking your executable against the library. 
 You're really dynamically loading rather than dynamically 
 linking (though unfortunately, the terminology for the two is 
 not particularly distinct, and they're often referred to the 
 same way even though they're completely different). Loading 
 libraries that way is what you do when you do stuff like 
 plugins, because those aren't known when you build your 
 program. But it makes a lot less sense as an alternative to 
 linking your program against the library if you don't actually 
 need to load the library like that. The COFF vs OMF mess on 
 Windows makes it make slightly more sense on Windows (at least 
 with D, where dmd uses OMF by default, unlike most of the C/C++ 
 world at this point), because then it doesn't matter whether 
 COFF or OMF was used (e.g. IIRC, Derelict is designed to be 
 loaded that way for that reason), but in general, it's an 
 unnecessarily complicated way to use a library. And if Windows' 
 eccentricities make it more desirable than it is on *nix 
 systems, then that's just yet another black mark against how 
 Windows does dynamic linking IMHO.

 - Jonathan M Davis

 From the last time I worked with Windows dlls, I remember quite 
 distinctly
 that doing anything like adding a symbol to the library meant 
 that it was incompatible with executables previously built with 
 it (which is not true for shared libraries on *nix - they only 
 break if the ABI for the existing symbols changes). So, if 
 that's not the case, I don't know what we were doing wrong, but 
 I have an extremely sour taste in my mouth from dealing with 
 Windows dlls. It was my experience that Linux stuff generally 
 just worked, whereas we kept having to deal with junk on 
 Windows to make them work (e.g. adding special attributes to 
 functions just so that they would be exported), and I 
 absolutely hated it. I have nothing good to say about dlls on 
 Windows. It's quite possible that some of it isn't as bad if 
 you know more about it than the team I was working on did, but 
 it was one part of why making our libraries cross-platform was 
 not at all fun.

 - Jonathan M Davis
Yah you should just stop and never talk about how Windows DLLs work again until you've actually learned how they actually work. "I used some piece of software that I have no idea how it works, I condemn it cause it works this way even though I'm sure if it even works that way". I already know the D team has no idea what they are doing on Windows, but just stop.
Jan 03
prev sibling parent reply Ethan <gooberman gmail.com> writes:
On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis 
wrote:
From the last time I worked with Windows dlls, I remember quite 
distinctly
 that doing anything like adding a symbol to the library meant 
 that it was incompatible with executables previously built with 
 it (which is not true for shared libraries on *nix - they only 
 break if the ABI for the existing symbols changes).
That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal. All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.
Jan 03
next sibling parent Rubn <where is.this> writes:
On Thursday, 3 January 2019 at 21:22:03 UTC, Ethan wrote:
 On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis 
 wrote:
From the last time I worked with Windows dlls, I remember quite 
distinctly
 that doing anything like adding a symbol to the library meant 
 that it was incompatible with executables previously built 
 with it (which is not true for shared libraries on *nix - they 
 only break if the ABI for the existing symbols changes).
That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal. All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.
Even if you used ordinal values instead, you need to assign the symbol a specific ordinal value. The only reason you would have to relink the executable is if you changed the ordinal values to be incompatible with the previous layout, eg assigning a new function to have a ordinal value the same to one that was previously assigned to different function. The equivalent to removing a symbol resolving by name on *nix/Windows.
Jan 03
prev sibling parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Thursday, January 3, 2019 2:22:03 PM MST Ethan via Digitalmars-d wrote:
 On Thursday, 3 January 2019 at 19:36:52 UTC, Jonathan M Davis

 wrote:
From the last time I worked with Windows dlls, I remember quite
distinctly

 that doing anything like adding a symbol to the library meant
 that it was incompatible with executables previously built with
 it (which is not true for shared libraries on *nix - they only
 break if the ABI for the existing symbols changes).
That would surely break if you were looking up symbols by ordinal instead of by name. I didn't even know you could do that until I read the documentation for GetProcAddress closely - turns out the name parameter can be used as an ordinal. All exportable symbols are exported by their mangled name in to the DLL's export table. Resolving by mangle will give you a similar experience to *nix there.
That would be a significant improvement. Thanks for pointing that out. If/when I have to deal with Windows dlls again, I'll have to look into that - though honestly, in general, I don't see any reason to use shared libraries for anything other than plugins (whether you're talking about *nix or Windows). If you're dealing with system libraries that are shared by a ton of programs on the system (e.g. openssl on *nix systems), then using shared libraries can definitely be worth it (especially when the library is likely to need security updates), but if you're distributing a program, static libraries are far easier, and if a library isn't used by many programs, then I think that the simplicity of static libraries is worth far more than the small space gain of using a shared library. It's just with plugins that you really don't have any choice. Regardless, D should have full shared library support on all of its supported platforms, and for those who care particularly about Windows, hopefully, Benjamin Thaut's work can be finished in a timely manner. IIRC, he did make a post about it a few months back looking for folks to try out his branch of dmd so that any issues could be worked out. I have no clue how long it will be before that work gets merged, but I expect that it will happen at some point. - Jonathan M Davis
Jan 03
parent reply Rubn <where is.this> writes:
On Friday, 4 January 2019 at 03:48:49 UTC, Jonathan M Davis wrote:
 though honestly, in general, I don't see any reason to use 
 shared libraries for anything other than plugins (whether 
 you're talking about *nix or Windows). If you're dealing with 
 system libraries that are shared by a ton of programs on the 
 system (e.g. openssl on *nix systems), then using shared 
 libraries can definitely be worth it (especially when the 
 library is likely to need security updates), but if you're 
 distributing a program, static libraries are far easier, and if 
 a library isn't used by many programs, then I think that the 
 simplicity of static libraries is worth far more than the small 
 space gain of using a shared library. It's just with plugins 
 that you really don't have any choice.
Here we go again. Really? You don't think there is any reason to using shared libraries in anything other than plugins? You then go on to list two other valid reasons in your next run on sentence. I'll add some more to your list: treating code as data is another reason. Hot loading code so you don't have to restart your program which can be time consuming. Would not work without a shared library. Modifying applications, though I guess from your perspective this might be niche. It would not be as elegant without shared libraries. You keep bringing up this size saving rationale, no one has said this other than you.
Jan 04
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Fri, 04 Jan 2019 12:54:12 +0000, Rubn wrote:
 Here we go again. Really? You don't think there is any reason to using
 shared libraries in anything other than plugins? You then go on to list
 two other valid reasons in your next run on sentence. I'll add some more
 to your list
I'll give another: linking times. With GTKD, static linking costs something like 6-10 seconds more than dynamic linking, which is huge.
Jan 04
parent reply Ethan <gooberman gmail.com> writes:
On Friday, 4 January 2019 at 16:28:32 UTC, Neia Neutuladh wrote:
 On Fri, 04 Jan 2019 12:54:12 +0000, Rubn wrote:
 Here we go again. Really? You don't think there is any reason 
 to using shared libraries in anything other than plugins? You 
 then go on to list two other valid reasons in your next run on 
 sentence. I'll add some more to your list
I'll give another: linking times. With GTKD, static linking costs something like 6-10 seconds more than dynamic linking, which is huge.
And to add to this. *ANY* large project benefits from reducing to smaller DLLs and a corresponding static .lib to load them. The MSVC team has come around in the last five years to love game developers specifically because we create massive codebases that tax the compiler. Remember how I said our binary sizes were something up near 100MB on Quantum Break during one of my DConf talks? Good luck linking that in one hit.
Jan 04
parent Ethan <gooberman gmail.com> writes:
On Friday, 4 January 2019 at 22:33:57 UTC, Ethan wrote:
 And to add to this. *ANY* large project benefits from reducing 
 to smaller DLLs and a corresponding static .lib to load them.

 The MSVC team has come around in the last five years to love 
 game developers specifically because we create massive 
 codebases that tax the compiler. Remember how I said our binary 
 sizes were something up near 100MB on Quantum Break during one 
 of my DConf talks? Good luck linking that in one hit.
Adding to myself. Pretty sure the default for the Windows UCRT (Universal C Runtime) is to be dynamically linked these days.
Jan 04